TStringList.LoadFromFile Unicode

Question

I am attempting to open a txt file to a StringList but if I open a UTF-8 format it fails to load, this is confusing because I have Unicode XE2, am I missing something stupid here?

Simple Sample

Sl := tStringList.Create;

SL.LoadFromFile(sFilePath);

For i =0 to SL.Count -1 do
  foo

but the String does not load when the txt file is UTF-8 but works fine when its in ANSI format.

Answer 1

TStringList.LoadFromFile will attempt to infer the encoding from the file's byte order mark (BOM). If no BOM is present then ANSI encoding is assumed.

In your case it seems clear that there is no BOM, so you must tell LoadFromFile which encoding to use. Do that by specifying the encoding as the second argument passed to LoadFromFile :

SL.LoadFromFile(sFilePath, TEncoding.UTF8);

Answer 2

If your UTF-8 file does have a BOM, then loading a UTF-8 file which contains an invalid UTF-8 byte sequence will produce an empty result, with no exception or indication of the failure. This is a 'feature' of the Delphi file handling. So if you see this result and your file has a valid BOM, check the content.