Does SuperObject have UTF-8 support

Question

I have been using superobject for all my json parsing needs and today I ran into a bit of a problem that I cannot seem to fix. I downloaded a json file that had an entry in it that looked like this: "place" : "café" and when I tried to parse the file and show it in a messagebox the word café turned out like this: café which tells me that the there is some kind of conversion failure going on when the file was parsed using superobject so before I invest any more time in this library, I would like to know if it supports UTF-8 and if so, how would I go about enabling it.

BTW, The pseudo code I am using to parse the file looks something like this:

uses 
SuperObject
...

const
jsonstr = '{ "Place" : "café" }';
...

var
  SupOB : ISuperObject;
begin
  SupOB := SO(jsonstr);
  ShowMessage(SupOB['Place'].AsString);
end;

Is the conversion failing because I am casting the object as a string? I tried also using AsJson to see if that would have any effect, but it did not so I am not sure what is needed to make objects like these display as they are intended and would appreciate some help. Finally, I have checked and verified that the original file that is being parsed is indeed encoded as UTF-8.

Answer 1

You say you are parsing a file , but your example is parsing a string . That makes a big difference, because if you are reading file data into a string first, you are likely not reading the file data correctly. Remember that Delphi strings use UTF-16 in Delphi 2009 and later, but use ANSI in earlier versions. Either way, not UTF-8. So if your input file is UTF-8 encoded, you must decode its data to the proper string encoding before you can then parse it. café is the UTF-8 encoded form of café being mis-interpreted as ANSI.

Answer 2

Reading and writing files json encoded utf8. Tested on Delphi 2007.

function ReadSO(const aFileName: string): ISuperObject;
var
  input: TFileStream;
  output: TStringStream;
begin
  input := TFileStream.Create(aFileName, fmOpenRead, fmShareDenyWrite);
  try
     output := TStringStream.Create('');
     try
       output.CopyFrom(input, input.Size);
       Result := TSuperObject.ParseString(PWideChar(UTF8ToUTF16(output.DataString)), true, true);
     finally
       output.Free;
     end;

  finally
    input.Free;
  end;
end;

procedure WriteSO(const aFileName: string; o: ISuperObject);
var
  output: TFileStream;
  input: TStringStream;
begin
  input := TStringStream.Create(UTF16ToUTF8(o.AsJSon(true)));
  try
     output := TFileStream.Create(aFileName, fmOpenWrite or fmCreate, fmShareDenyWrite);
     try
       output.CopyFrom(input, input.Size);
     finally
       output.Free;
     end;
  finally
    input.Free;
  end;
end;

Functions UTF8ToUTF16 and UTF16ToUTF8 from unit JclConversions http://sourceforge.net/projects/jcl/ .