Planet
navi homePPSaboutscreenshotsdownloaddevelopmentforum

source: downloads/boost_1_34_1/libs/serialization/doc/implementation.html @ 33

Last change on this file since 33 was 29, checked in by landauf, 17 years ago

updated boost from 1_33_1 to 1_34_1
File size: 22.3 KB
Line 
1<!doctype HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
2<html>
3<!--
4(C) Copyright 2002-4 Robert Ramey - http://www.rrsd.com .
5Use, modification and distribution is subject to the Boost Software
6License, Version 1.0. (See accompanying file LICENSE_1_0.txt or copy at
7http://www.boost.org/LICENSE_1_0.txt)
8-->
9<head>
10<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
11<link rel="stylesheet" type="text/css" href="../../../boost.css">
12<link rel="stylesheet" type="text/css" href="style.css">
13<title>Serialization - Implementation Notes</title>
14</head>
15<body link="#0000ff" vlink="#800080">
16<table border="0" cellpadding="7" cellspacing="0" width="100%" summary="header">
17  <tr> 
18    <td valign="top" width="300"> 
19      <h3><a href="../../../index.htm"><img height="86" width="277" alt="C++ Boost" src="../../../boost.png" border="0"></a></h3>
20    </td>
21    <td valign="top"> 
22      <h1 align="center">Serialization</h1>
23      <h2 align="center">Implementation Notes</h2>
24    </td>
25  </tr>
26</table>
27<hr>
28<dl class="page-index">
29  <dt><a href="#functiontemplateordering">Partial Function Template Ordering</a>
30  <dt><a href="#charencoding">Character Encoding</a>
31  <dt><a href="#tempatesyntax">Template Invocation syntax</a>
32  <dt><a href="#partialtemplatespecialization">Partial Template Specialization</a>
33  <dt><a href="#othercompilerissues">Specific Compiler/Library Issues</a>
34  <dl class="page-index">
35    <dt><a href="#gcc3x">GCC 3.X, 4.X</a>
36    <dt><a href="#gcc295">GCC 2.95</a>
37    <dt><a href="#intel80">Intel 8.0</a>
38    <dt><a href="#vc80">Visual C++ 8.0</a>
39    <dt><a href="#vc71">Visual C++ 7.1</a>
40    <dt><a href="#vc70">Visual C++ 7.0</a>
41    <dt><a href="#vc6">Visual C++ 6.0</a>
42    <dt><a href="#borland">Borland 5.64 and 5.51</a>
43    <dt><a href="#comeau">Comeau 4.3.3</a>
44    <dt><a href="#codewarrior9">Code Warrior 9.x</a>
45    <dt><a href="#codewarrior">Code Warrior 8.3</a>
46    <dt><a href="#tru64">TRU64</a>
47    <dt><a href="#dinkumware">Dinkumware Library</a>
48    <dt><a href="#stlport">STLPort 4.5.3</a>
49  </dl>
50</dl>
51
52<h3><a name="functiontemplateordering">Partial Function Template Ordering</a></h3>
53Not all C++ compilers correctly support partial function template ordering (PFTO).
54For these compilers, the following code will fail to compile:
55<pre><code>
56template&lt;class Archive, class T&gt;
57void serialize(
58    Archive & ar,
59    T & t,
60    const unsigned int file_version
61){
62    ...
63}
64
65template&lt;class Archive, class T&gt;
66void serialize(
67    Archive & ar,
68    my_template&lt;T&gt & t,
69    const unsigned int file_version
70){
71    ...
72}
73</pre></code>
74The serialization library works around this issue by using a different
75default definition of the first template:
76<pre><code>
77template&lt;class Archive, class T&gt;
78void serialize(
79    Archive & ar,
80    T & t,
81    const unsigned long int file_version  // Note: change to long
82){
83    ...
84}
85</pre></code>
86Now, the second template is not matched with the first one so there
87is no PFTO and no compile error.  When the serialization library invokes
88<pre><code>
89serialize(ar, t, 0);
90</pre></code>
91the function declaration is first matched against templates with
92an integer for the third argument.  If there is a match, the matching
93template is instantiated and later invoked.  If there is no match,
94an attempt is made to match other templates by converting arguments to other types.
95In this case the third argument can be converted to long to match
96the first template - which is the default.  So in this case, the first
97template will be instantiated and later invoked.  We have managed to
98use function overloading to achieve the same effect as PFTO
99were it correctly implemented.
100<p>
101This depends upon undefined behavior of a compiler already
102determined to be non-conforming.  In other words, there is no
103guarantee that this will work on all compilers.  If a compiler does not
104correctly support PFTO and this method cannot be used to workaround
105it, non-intrusive serialization cannot be supported for that compiler.
106As of this writing, such a compiler has not been encountered.
107<p>
108It turns out that using this "trick" can create problems with
109compilers that DO correctly support PFTO.  For this reason we
110define a macro <code style="white-space: normal">BOOST_PTFO</code> which
111is defined to be <code style="white-space: normal">long</code>
112for non-conforming compilers and nothing for conforming ones. So
113the default definition is really:
114The serialization library works around this issue by using a different
115default definition of the first template:
116<pre><code>
117template&lt;class Archive, class T&gt;
118void serialize(
119    Archive & ar,
120    T & t,
121    const unsigned BOOST_PFTO int file_version  // Note: change to BOOST_PFTO
122){
123    ...
124}
125</pre></code>
126
127<h3><a name="charencoding">Character Encoding</a></h3>
128The whole question of character encoding combined with wide characters
129is much more complicated than it would seem to be. The current library
130defines in 3 formats (text, binary, and XML), wide and narrow characters,
131an attempts to be portable between compiler libraries.  The results of
132a rather long consideration of all these factors has been to set
133default encoding according to the following rules.
134<ul>
135  <li>All text archives (i.e. <code style="white-space: normal">text_?archive</code>) will produce
136      text output in the current stream <code style="white-space: normal">locale</code>.  Generally this will
137      produce no changes in string data.
138  <li>To produce binary output with Microsoft compilers, the stream
139      will have to be opened with mode <code style="white-space: normal">ios::binary</code>.
140      Failure to do so will result in 0x0d characters (carriage-return)
141      characters being removed from the input stream if they are followed
142      by a 0x0a character (line-feed).  This could corrupt the input
143      and make the file unreadable.  On UNIX systems the <code style="white-space: normal">ios::binary</code>
144      is not required and is ignored if used.
145  <li>character XML archives (i.e. xml_oarchive) will produce XML output
146      with characters encoded according to the current stream <code style="white-space: normal">locale</code>.
147  <li>wide character XML archives (i.e. xml_woarchive) will produce
148      files encoded in UTF-8.
149</ul>
150This character encoding is implemented by changing the <code style="white-space: normal">locale</code> of the
151i/o stream used by an archive when the archive is constructed, the stream
152local is changed back to its original value. This action can be overridden
153by specifying <code style="white-space: normal">boost::archive::no_codecvt</code>
154when the archive is opened.  In this case, the stream <code style="white-space: normal">locale</code> will
155not be changed by the serialization library.
156<p>
157Note that the code conversion included for wide character text and XML
158archives could alter <code style="white-space: normal">std::string</code> data stored in archives.
159Suppose a normal (multi-byte) character string
160is written to a wide character stream.  Our system uses the current <code style="white-space: normal">locale</code>
161to translate it to a wide character string before writing it out. 
162Upon reading, it is translated back to a (multi-byte)string.
163If the <code style="white-space: normal">locale</code> on the platform that reads the archive is different than
164the <code style="white-space: normal">locale</code> on the platform that wrote the stream, the actual string data
165may be altered by the serialization process. To avoid this, either
166avoid usage of <code style="white-space: normal">locale</code> dependent multi-byte strings or be sure that
167the <code style="white-space: normal">locale</code> is set correctly before reading the archive.
168<p>
169To produce wide character text output (i.e. 16 bit characters on Win32 systems),
170do the following.
171<ul>
172  <li>Open a wide character stream.
173  <li>Alter the stream <code style="white-space: normal">locale</code> to use
174     <code style="white-space: normal">boost::archive::codecvt_null&lt;OStream::char_type&gt;</code>
175  <li>Create the archive with the flag <code style="white-space: normal">no_codecvt</code>.
176</ul>
177Naturally, the input process has to be symmetrical.
178<h3><a name="partialtemplatespecialization">Partial Template Specialization</a></h3>
179Compilers which fail to support partial template specialization will fail to compile
180the following code.  To make this compiler, the <code style="white-space: normal">const</code> has to be removed.
181<pre><code>
182void f(A const* a, text_oarchive& oa)
183{
184  oa << a;
185}
186</code></pre>
187<h3><a name="tempatesyntax">Template Invocation syntax</a></h3>
188Some compilers may not recognize the syntax:
189<pre><code>
190ar.template register_type&lt;T&gt;();
191</code></pre>
192for "registering" derived pointers of polymorphic classes.  The actual
193function prototype is:
194<pre><code>
195template&lt;T&gt;
196void register_type(T * t = NULL);
197</code></pre>
198so that one may write <code style="white-space: normal">ar.register_type(static_cast&lt;T *&gt;(NULL))</code> instead of
199the syntax described above.
200</ul>
201<h3><a name="othercompilerissues">Specific Compiler/Library Issues</a></h3>
202
203<h4><a name="gcc3x">GCC 3.X, 4.X</a></h4>
204<ul>
205    <li>GCC versions for Cygwin and MinGW fail to support wide character I/O. 
206    So all tests using wide char I/O fail.  Note that if wide character I/O support
207    is added with STLPort, all tests complete successfully.
208    <li>This compiler generates long warning messages related to the usage of
209    non virtual destructors in polymorphic classes.  These warnings have been
210    carfully considered and the code that generates these warning has been
211    unchanged.  In this case the warning should should be ignored as in certain
212    usages of the library, making the destructors virtual could lead to problems.
213    As an alternative, base class destructors have been make "protected" to
214    address the concerns that motivate these warning messages.  When building
215    the serialization library and tests with bjam, these warnings are suppressed.
216    When building one's own applications, these warnings can be suppressed by
217    adding the following to the compiler command line:
218    <pre><code>
219    -Wno-non-virtual-dtor
220    -Wno-ctor-dtor-privacy
221    </code></pre>
222</ul>
223<h4><a name="gcc295">GCC 2.95</a></h4>
224All of the above plus:<br>
225<ul>
226    <li>The serialization library depends on the templeted stream implemention
227    to function properly.  So STLPort must be used to build the library.
228    <li>Polymorphic archive tests fail.
229    <li>XML serialization only works with version 1.6x of spirit. In order to build
230    and use this library with this compiler, one must use version 1.6x rather than the
231    latest version shipped with boost. See <a href="release.html">Release Notes</a>.
232</ul>
233<h4><a name="Intel80">Intel C++ 8.0</a></h4>
234No known issues. All tests compile and run in debug and release modes.
235
236<h4><a name="vc80">Visual C++ 8.0</a></h4>
237This compiler emits warnings for calls to functions from the standard
238library which are deemed security risks.  The serialization depends upon
239making some of these calls so programs which use the serialization library
240will get warning messages.  These messages can be suppressed from the command
241line by including the following switch:
242    <pre><code>
243    /wd4996
244    </code></pre>
245
246<h4><a name="vc71">Visual C++ 7.1</a></h4>
247Derivation from an archive class defined in a DLL as described in ... will not work.
248This is due to the way that VC++ handles templeted code with __decl(dllexport) and
249__decl(dllimport) specifications.  Basically, this compiler requires that all the
250instantiations have the same specification - even though they have different
251template arguments. The example <code style="white-space: normal">
252demo_portable_iarchive.cpp</code> would have to reformulated as a library or dll
253similar to the pre-defined archives in order to function.
254<p>
255This compiler does not have RTTI or exception handling turned on by default.  Although
256they are not strictly necessary to use the serialization package, the example and test
257program presume that they are enabled.  So be sure your command line or IDE settings
258enable these features if you want
259these switches are enabled if you want to build and run these programs.
260
261<h5>Using the Visual C++ IDE</h5>
262The library includes a VC++ 7.1 "Solution" - <code style="white-space: normal">BoostSerializationLibrary</code>
263along with of a set of project files - one for each demo and test. Consider the following if you
264decided to use these configurations.
265<ul>
266    <li>The projects assume that the tests have been built with bjam using the default
267    locations.  This will result in a <code style="white-space: normal">bin</code> subdirectory
268    within one's main boost directory.  Below this there is a whole structure which maintains
269    object and library files according to the type of build.  The easiest way to build this is to
270    invoke the runtest script which uses bjam. (see below)  If the libraries are not in these locations,
271    the projects will have to be modified accordingly.
272    <li>There are project configurations for all the combinations of build variants that boost
273    supports. That is for release, debug, static, static multi-threading, etc..
274    <li>If you want to use/debug the DLL versions of libraries and corresponding tests, alter
275    the project file to define <code style="white-space: normal">BOOST_ALL_DYN_LINK=1</code>.
276    Note that for the executables to run, the <code style="white-space: normal">PATH</code>
277    environmental variable will have to include the directories that contain the DLL versions of
278    the boost libraries.
279    <li>If you have difficulties building your own projects and linking with the boost libraries,
280    compare the project settings of your own projects with the ones here.  VC sometimes requires
281    consistent settings between projects and the libraries they use in order to link properly.
282    In particular, check support for exceptions, runtime typing(RTTI), and intrinsic support for
283    wide characters.  The standard version of this library presumes that these facilities are
284    enabled.  Projects generated by the IDE wizard do not have these features enabled by default.
285    <li>Frequently when trying to build a project or view project properties, one is presented with
286    a message box with the message "unspecified error".  This seems to occur when one changes the
287    build configuration selection.  It turns out this can be "fixed" by going to the "Build"
288    menu item, selecting "Configuration Manager" and selecting a build configuration for the project
289    you're working with.
290    <li>To test that boost libraries are built correctly, one can build and test them the way we do.
291    This entails:
292    <ol>
293        <li>downloading a copy of bjam.exe
294        <li>building process_jam_log
295        <li>building compiler_status
296        <li>invoking runtest.bat
297    </ol>
298    This will build the serialization library and run the tests on your system. If there are more than a
299    a couple of test failures, you likely won't be able to get your own projects working.  If most of the
300    tests pass, you can be confident that your own projects will work once you get your project settings
301    in sync with those included here.
302</ul>
303
304<h4><a name="vc70">Visual C++ 7.0</a></h4>
305<ul>
306    <li>The "pimpl" demo fails to link.  Cause and workaround for this is unknown
307    <li>XML serialization only works with version 1.6x of spirit.  In order to build and use this
308    library with this compiler, one must use version 1.6x rather than the latest version
309    shipped with boost.  See <a href="release.html">Release Notes</a>.
310    <li>This compiler does not support partial template specialization. 
311    The implementation of the new <code>shared_ptr</code> serialization depends upon
312    compiler support for partial template specialization.  This compiler doesn't implement this
313    feature. In order to serialize <code style="white-space: normal">shared_ptr&lt;A&gt;</code>,
314    invoke the macro <code style="white-space: normal">BOOST_SERIALIZATION_SHARED_PTR(A)</code>
315    in the header code.
316    <li>Lack of support for partial template specialization also creates problems for
317    serialization of <code style="white-space: normal">std::map</code>.  In order to serialize
318    instances of this type include the invocation of BOOST_BROKEN_COMPILER_TYPE_TRAITS_SPECIALIZATION
319    for the key type of the map.
320</ul>
321<h4><a name="vc6">Visual C++ 6.5</a></h4>
322all the above issues for Visual C++ 7.0 plus:
323<ul>
324    <li>Out of line template definitions are not recognized and fail with a confusing
325    error message.  To function save/load/serialize member function templates must be defined
326    within the class definition.  This feature is essential to <code style="white-space: normal">demo_pimpl</code>. Hence,
327    this program will fail to compile.  In this case the problem can't be worked around and
328    still demonstrate this facility.
329    <li>This compiler does not support <code style="white-space: normal">wchar_t</code> as a separate type.  It defines
330    <code style="white-space: normal">wchar_t</code> as an alias for <code style="white-space: normal">short int</code>.  In general things will still
331    function.  However certain customization, such as overloading archive operators for
332    saving/loading wide character arrays would produce surprises in this environment.
333    <li>Under certain circumstances, a program will fail to link with the message:
334    LIN1179 - "invalid or corrupt file: duplicate comdat".  According
335    to <a href="http://groups.google.com/groups?th=8a05c82c4ffee280">
336    http://groups.google.com/groups?th=8a05c82c4ffee280
337    </a> (look for P78)
338    A LNK1179 error occurs when:
339    <ul>
340        <li>The template class takes at least two arguments.
341        <li>The template is used at least two times with identical first
342        and different second arguments.
343        <li>The static member variable is of an object type with at least one
344        base class. (In another scenario it also occurred using a member
345        without a base class.)
346    </ul>
347    Working around this in the implementation of the library for this compiler
348    entailed a ridiculous amount of effort.  Even so, the effort wasn't entirely succesful.
349    With this compiler, this message will still appear under the following conditions:
350    <ul>
351        <li>When serializing a class with multiple base classes.  This problem causes two
352        failure in the test suite.  I have been unable to divise a way to work around this.
353        <li>Using more than one kind of archive in the same code module. This should be easy
354        to work around in practice.
355    </ul>
356    <li>Code modules exceeding some undetermined size that use the library will fail with
357    <i>fatal error C1204: compiler limit : internal structure overflow</i>.  This can be addressed
358    by dividing the module into smaller ones.
359</ul>
360<h4><a name="borland">Borland 5.64 and 5.51</a></h4>
361<ul>
362    <li><code style="white-space: normal">enum</code> data members cannot be serialized.
363    Conversion to/from integers will work around the problem.
364    <li>If class serialize functions are not accessable either by making them public or by
365    including <code style="white-space: normal">friend</code> declarations as described in
366    <a href="serialization.html#member">Class Serialization - Member Function</a>, the
367    will compile but fail at runtime.
368    <li>Tests using custom extended type which doesn't use rtti fails. (5.64 only !).
369    <li>Tests built in release mode fail. This seems to be an issue with the boost test system
370    with this compiler.
371    <li>XML serialization only works with version 1.6x of spirit. In order to build
372    and use this library with this compiler, one must use version 1.6x rather than the
373    latest version shipped with boost. See <a href="release.html">Release Notes</a>.
374</ul>
375<h4><a name="comeau">Comeau 4.3.3</a></h4>
376<ul>
377    <li>This compiler fails to make a DLL with export under windows.
378    <li>The associated library - libcomo fails when using a codecvt facet.
379    This generates a failure with all wide character archives.
380    <li>the test_set fails by going into an infinite memory leak.
381</ul>
382
383<h4><a name="codewarrior9">Code Warrior 9.x</a></h4>
384<ul>
385    <li>Some tests and demos demos fail - still under investigation
386</ul>
387
388<h4><a name="codewarrior">Code Warrior 8.3</a></h4>
389all the above issues for Code Warrior 9.x plus:
390<ul>
391    <li>This compiler only supports templated streams with the static library version.
392    <li>The above inhibits the build of DLL versions of the library.
393    <li>Some demos fail - still under investigation
394</ul>
395
396<h4><a name="tru64">TRU64</a></h4>
397All tests and demos pass except for test_variant.  Boost Variant doesn't function
398wih this compiler
399
400<h4><a name="dinkumware">Dinkumware Library</a></h4>
401Several compilers, including Visual C++ 6.0, use an older dinkumware library. 
402These platforms have several issues:
403<ul>
404    <li>The dinkumware library shipped with this compiler does not change the locale facet
405    of an i/o stream unless the <code style="white-space: normal">imbue</code> function is called before the the
406    stream is opened.  In order to use this library with this environment to generate UTF-8
407    files, one cannot depend on the "automatic" setting of locale that archives implement. The
408    stream locale must be set explicitly on the stream before an archive is opened on it.  The
409    archive should be opened with the <code style="white-space: normal">no_codecvt</code> flag.  Note this problem will
410    occur on all compilers shipped with this library.
411    <li>Other issues have been worked around in the file.
412    <a href="../../../boost/archive/dinkumware.hpp" target="dinkumware_hpp">dinkumware.hpp</a>
413</ul>
414
415<h4><a name="stlport">STLPort 4.5.3</a></h4>
416<ul>
417    <li>when built to use dynamic linking versions of C++ runtime code (<runtime-link>dynamic)
418    all tests fail to link.  This is due to a missing symbol in the stlport library related
419    to custom codecvt facets.
420    <li>the test_set fails to run correctly. It seems the hashed set interator doesn't
421    implement the ++ operator correctly.  This causes the test to fail by consuming all available
422    memory.  Given this, this test is commented out.
423</ul>
424
425<hr>
426<p>Revised 1 November, 2004
427<p><i>&copy; Copyright <a href="http://www.rrsd.com">Robert Ramey</a> 2002-2004.
428Distributed under the Boost Software License, Version 1.0. (See
429accompanying file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
430</i></p>
431</body>
432</html>
Note: See TracBrowser for help on using the repository browser.