Conversion from char16_t(utf16) to utf8 doesn't work in visual studio 2015 and 2017

Question

I want to convert the data type char16_t to utf8, and I have written the following code to do that:

// utf16 String
char16_t *wchs = u"UTF16 string";
// A converter object
wstring_convert<codecvt_utf8_utf16<char16_t>, char16_t> conv;
// Convert from utf16 to utf8
string str = conv.to_bytes(wchs);
// print the UTF-16 string after converting to UTF-8
cout << str << endl;

The previous code doesn't work in visual studio whether 2015 or 2017, specifically with "char16_t" data type and gives me an error:
error LNK2001: unresolved external symbol "public: static class std::locale::id std::codecvt<char16_t,char,struct _Mbstatet>::id" (?id@?$codecvt@_SDU_Mbstatet@@@std@@2V0locale@2@A) ,
but works well in the GCC compiler.

Why that happened and how to solve that problem ?

Answer 1

You might be out of luck as this seems to be a Visual Studio bug since there are at least 3 posts on the VS Developer Community page which describe the given problem

• https://developercommunity.visualstudio.com/content/problem/539181/stdwstring-convert-char16-t-cant-work-in-vs2017.html

• https://developercommunity.visualstudio.com/content/problem/65248/unresolved-external-symbol-static-class-stdlocalei.html

• https://developercommunity.visualstudio.com/content/problem/246272/msconnect-3118643-codecvt-library-issue.html

where all of them state that it has been

fixed in: visual studio 2019 version 16.2

with no indication of a backport to VS 2015 or even VS 2017.

Answer 2

Here is a simple function how you can convert from std::u16string to std::string. It works with VS 2017.

std::string utf16_to_utf8( std::u16string&& utf16_string )
{
   std::wstring_convert<std::codecvt_utf8_utf16<int16_t>, int16_t> convert;
   auto p = reinterpret_cast<const int16_t *>( utf16_string.data() );
   return convert.to_bytes( p, p + utf16_string.size() );
}

The reason is the declaration of wstring_convert<codecvt_utf8_utf16<char16_t>, char16_t> conv; with char16_t template parameter does not compile under VS 2015 and VS2017 due of missing initialization of static locale::id id; in a.cpp file. The int16_t type is working.

Under Linux with gcc or clang compiler suites it is a little bit easier to convert from std::u16string. There is no issue with static locale::id id; .

std::string utf16_to_utf8( std::u16string&& utf16_string )
{
   std::wstring_convert<std::codecvt_utf8_utf16<char16_t>, char16_t> convert;
   return convert.to_bytes( utf16_string );
}