| 1 | <html> |
|---|
| 2 | |
|---|
| 3 | <head> |
|---|
| 4 | <meta http-equiv="Content-Type" |
|---|
| 5 | content="text/html; charset=iso-8859-1"> |
|---|
| 6 | <meta name="GENERATOR" content="Microsoft FrontPage 4.0"> |
|---|
| 7 | <title>Boost Tokenizer Overview</title> |
|---|
| 8 | <!-- |
|---|
| 9 | -- Copyright (c) Jeremy Siek 2000 |
|---|
| 10 | -- |
|---|
| 11 | -- Permission to use, copy, modify, distribute and sell this software |
|---|
| 12 | -- and its documentation for any purpose is hereby granted without fee, |
|---|
| 13 | -- provided that the above copyright notice appears in all copies and |
|---|
| 14 | -- that both that copyright notice and this permission notice appear |
|---|
| 15 | -- in supporting documentation. Jeremy Siek makes no |
|---|
| 16 | -- representations about the suitability of this software for any |
|---|
| 17 | -- purpose. It is provided "as is" without express or implied warranty. |
|---|
| 18 | --> |
|---|
| 19 | </head> |
|---|
| 20 | |
|---|
| 21 | <body bgcolor="#FFFFFF" text="#000000" link="#0000EE" |
|---|
| 22 | vlink="#551A8B" alink="#FF0000"> |
|---|
| 23 | |
|---|
| 24 | <p><img src="../../boost.png" alt="C++ Boost" width="277" |
|---|
| 25 | height="86"> <br> |
|---|
| 26 | </p> |
|---|
| 27 | |
|---|
| 28 | <h1 align="center">Table Of Contents</h1> |
|---|
| 29 | |
|---|
| 30 | <p align="left"> </p> |
|---|
| 31 | |
|---|
| 32 | <h2 align="left"><a href="introduc.htm">Introduction</a></h2> |
|---|
| 33 | |
|---|
| 34 | <h2 align="left">Containers and Iterators</h2> |
|---|
| 35 | |
|---|
| 36 | <ul> |
|---|
| 37 | <li><h3 align="left"><a href="tokenizer.htm">tokenizer</a></h3> |
|---|
| 38 | </li> |
|---|
| 39 | <li><h3 align="left"><a href="token_iterator.htm">token |
|---|
| 40 | iterator</a></h3> |
|---|
| 41 | </li> |
|---|
| 42 | </ul> |
|---|
| 43 | |
|---|
| 44 | <h2><a href="tokenizerfunction.htm">TokenizerFunction Concept</a></h2> |
|---|
| 45 | |
|---|
| 46 | <h2>TokenizerFunction Models</h2> |
|---|
| 47 | |
|---|
| 48 | <ul> |
|---|
| 49 | <li><h3><a href="char_separator.htm">char_separator</a></h3> |
|---|
| 50 | </li> |
|---|
| 51 | <li><h3><a href="escaped_list_separator.htm">escaped_list_separator</a></h3> |
|---|
| 52 | </li> |
|---|
| 53 | <li><h3><a href="offset_separator.htm">offset_separator</a></h3> |
|---|
| 54 | </li> |
|---|
| 55 | <li><font color="red">Deprecated:</font> <a href="char_delimiters_separator.htm">char_delimiters_separator</a></a> |
|---|
| 56 | </li> |
|---|
| 57 | </ul> |
|---|
| 58 | |
|---|
| 59 | <h2> </h2> |
|---|
| 60 | <h2>Acknowledgements</h2> |
|---|
| 61 | <p>I wish to thank the members of the boost mailing list, whose comments, |
|---|
| 62 | compliments, and criticisms during both the development and formal review helped |
|---|
| 63 | make the Tokenizer library what it is. I especially wish to thank Aleksey |
|---|
| 64 | Gurtovoy for the idea of using a pair of iterators to specify the input, instead |
|---|
| 65 | of a string. I also wish to thank Jeremy Siek for his idea of providing a |
|---|
| 66 | container interface for the token iterators and for simplifying the template |
|---|
| 67 | parameters for the TokenizerFunctions. He and Daryle Walker also emphasized the |
|---|
| 68 | need to separate interface and implementation. Gary Powell sparked the idea of |
|---|
| 69 | using the isspace and ispunct as the defaults for char_delimiters_separator. |
|---|
| 70 | Jeff Garland provided ideas on how to change to order of the template parameters |
|---|
| 71 | in order to make tokenizer easier to declare. Thanks to Douglas Gregor who |
|---|
| 72 | served as review manager and provided many insights both on the boost list and |
|---|
| 73 | in e-mail on how to polish up the implementation and presentation of Tokenizer. |
|---|
| 74 | Finally, thanks to Beman Dawes who integrated the final version into the boost |
|---|
| 75 | distribution.</p> |
|---|
| 76 | |
|---|
| 77 | <hr> |
|---|
| 78 | |
|---|
| 79 | <p>© Copyright John R. Bandela 2001. Permission to copy, use, |
|---|
| 80 | modify, sell and distribute this document is granted provided |
|---|
| 81 | this copyright notice appears in all copies. This document is |
|---|
| 82 | provided "as is" without express or implied warranty, |
|---|
| 83 | and with no claim as to its suitability for any purpose.</p> |
|---|
| 84 | |
|---|
| 85 | <p> </p> |
|---|
| 86 | |
|---|
| 87 | <p> </p> |
|---|
| 88 | </body> |
|---|
| 89 | </html> |
|---|
