使用不同类型的参数处理std :: basic_string <>

Question

I want to implement a c++ library, and like many other libs I need to take string arguments from the user and giving back strings. The current standard defines std::string and std::wstring (I prefer wstring). Theoretically I have to implement methods with string arguments twice:

virtual void foo(std::string &) = 0; // convert internally from a previous defined charset to unicode
virtual void foo(std::wstring &) = 0;

C++0x doesn't make life easier, for char16_t and char32_t I need:

virtual void foo(std::u16string &) = 0;
virtual void foo(std::u32string &) = 0;

Handle such different types internally - for example putting all into a private vector member - requires conversion, wrappers... it's horrible.

Another problem is if a user (or myself) wants to work with custom allocators or customized trait classes: everthing results in a completely new type. For example, to write custom codecvt specializations for multibyte charsets, the standard says I have to introduce a custom state_type - which requires a custom trait class which results in a new std::basic_ifstream<> type - and that's completely incompatible to interfaces expecting std::ifstream& as an argument.

One -possible- solution is to construct each library class as a template that manages the value_type, traits and allocators specified by the user. But that's overkill and makes abstract base classes (interfaces) impossible.

Another solution is to just specify one type (eg u32string) as default, every user must pass data using this type. But now think about a project which uses 3 libraries, and the first lib uses u32string, the second lib u16string and the thirth lib wstring -> HELL.

What I really want is to declare a method just as void foo(put_unicode_string_here) - without introduce my own UnicodeString or UnicodeStream class.

Answer 1

There is always choice that has to be made if you don't want to support everything, but I personnally feel restricting input to UTF-8 is the easiest of all. Just use plain old std::string and everyone's happy. In practice, the user (of your library) will only have to convert to UTF-8 if he's on Windows, but there's a plethora of ways to do that simple task.

UPDATE : on the other hand, you could template all of your code and leave the std::basic_string<T> as a template throughout your code. This only gets messy if you do different things dependent on the size of the template argument.

Answer 2

char_traits is indeed a hopelessly awful wastebin of random traits. Should every string pre-specify the largest supported file size, case-sensitivity, and (ugh) state type of the encoding mechanism itself? NO.

However, what you ask is impossible even with well-designed traits. string and wstring are meaningfully different because the size of the internal character type differs. To run any kind of algorithm, you will need to query the object for char_t . That requires RTTI or virtual functions because basic_string doesn't (and shouldn't) maintain that info at runtime.

One -possible- solution is to construct each library class as a template that manages the value_type, traits and allocators specified by the user. But that's overkill and makes abstract base classes (interfaces) impossible.

This is the only complete solution. Templates actually do play well with abstract base classes: a number of templates can derive from a non-template abstract base, or the base can also be templated. However, it is difficult if not untenable because of the sensitivity and tedium of writing perfectly generic code.

Another solution is to just specify one type (eg u32string) as default, every user must pass data using this type. But now think about a project which uses 3 libraries, and the first lib uses u32string, the second lib u16string and the thirth lib wstring -> HELL.

This is why I'm scared by C++11's "improved" Unicode support. It simplifies direct interaction with file data and discourages abstraction to a common wchar_t internal format. It would have been better to require specific codecvts for UTF-16 and UTF-32 and specify that wchar_t must be at least 21 bits. Whereas before there were only "dumb" char and "smart" wchar_t libraries among clean C++ interfaces, we may have to contend with additional widths — and char16_t is just an instant red flag.

But, that's down the road.

If you really end up using a number of incompatible libraries, and the problem is shuttling data between functions requiring different formats, then write a ScopeGuard-style utility to convert from and back to your chosen common format, such as wstring . This utility can be a template with an explicit specialization for each incompatible format you need, or a non-templated set of classes.