Serialize Lua Table from C++ (via JSON)

Question

I'd like to communicate complex data from a C++ service to a Lua application. This communication takes place over the network. For simplicity and speed in the Lua application I would prefer to send literal Lua table literals (no need for a separate parser) instead of XML or JSON or YAML or such.

While there exist things like C++ libraries that write JSON, I cannot find an existing C++ library for creating serialized Lua. My idea, then, is to use an existing JSON library for C++ and then convert the string to Lua.

So, for example, I'd like to convert this string:

{
  "hello":42,
  "array":[1,2,{"more":false},null,true],
  "worst":"still [null]: got it?"
}

into this string:

{
  ["hello"]=42,
  ["array"]={1,2,{["more"]=false},nil,true},
  ["worst"]="still [null]: got it?"
}

A naive replace_all converting to : to = , [] to {} , and null to nil will destroy content inside of strings. How can I perform this conversion?

_{To avoid the problems of an XY problem I have included my end motivation at the top and in the title, in case the JSON->Lua string conversion is the wrong choice.}

Answer 1

I would code that Lua -format serializing library by myself. You could choose a free software Json C++ library (eg jsoncpp or libjson ) and adapt its code (to your Lua-format) quite easily.

^{Of course you should obey the license of that library, and I strongly suggest you to make your Lua-format serialization library a free software itself, eg on github and/or freecode and/or sourceforge ...}

The point is that JSON (and hopefully your Lua format) is simple enough to make quite easy its parsing or printing... Adapting an existing library to your format is probably simpler and certainly faster than "post-processing" its output ...

Answer 2

Although I'm not locating it readily today, I recall that there was discussion a year or so ago on the Lua list of the merits of defining a limited subset of Lua table literals analogous to JSON, dubbed "LSON" for the sake of discussion. IIRC, the consensus developed that there wasn't enough benefit to be had over just using an established standard lightweight format like JSON, but I know some experiments were conducted.

This Github Gist for lson.lua demonstrates a simple LSON writer and reader in pure Lua. The writer could be transformed to C or C++ with only moderate effort based on that code. A key feature of that code is that it provides some protection against circular references, and against data types that can be stored in tables but which have no reasonable mechanism for writing as source code ( userdata , and thread are both highly problematic to serialize in any form). Of course, for data originating as plain old data in C with only lightweight structure, you won't have any of the problematic data types anyway. It also protects against circular references. If serializing lists or trees from C, circular references may be impossible by construction. If not, you will need to deal with them yourself.

Note that using Lua's own parser does potentially introduce security issues. The most glaring issue is that just writing assert(loadstring('return '..Input))() allows the imported text access to your entire current environment. While there is some protection resulting from applying the return keyword outside of the input text, that still won't prevent clever use of any functions that can be called from an expression. For best safety, you will want to read about sandboxes and possibly even apply some clever tricks to restrict the compiled bytecode before blindly executing it.

The security issues may be a strong argument in favor of qualifying and using a JSON parser. Even javascipt applications often prefer to use JSON parsers rather than just letting the javascript engine execute untrusted content.