JSON parsing very slow in Google Cloud Endpoints

Question

Google Cloud Endpoints suggests the use of GsonFactory or JacksonFactory to do the JSON parsing.

I have found that parsing a few 1000 java objects from my endpoint (each with only a few fields of longs and strings and a GeoPt) takes very long, about 15 seconds on a Galaxy Note 2.

As suggested by Google, I use:

myBuilder = new MyDbEndpoint.Builder(
                AndroidHttp.newCompatibleTransport(),
                new GsonFactory(),
                credential);
myEndpoint = myBuilder.build();

...

List<myDb> beings = myEndpoint.myDBEndpoint().someMethod().execute().getItems();

I get the same performance with JacksonFactory().

I've done some profiling and see that all the time is spent in JSON parsing.

Can anyone suggest any direction here for speeding this up, other than "paging" my data fetching?

Thanks.

Answer 1

It strikes me as a bit surprising that you didn't see a difference in speed between the implementations - I thought Jackson was supposed to be faster.

Leaving that aside, I wouldn't really expect this whole way of doing things to be fast. As someone who comes from a C/C++ background, the Google/GAE/Endpoints way of communicating data has always struck me as crazy inefficient - converting your binary data into text and then embedding it into a structured text representation, and then parsing all that data on the other end (eg compared to struct's).

Now, if you are just sending a few objects back and forth it doesn't really matter, but you are talking about parsing a few 1000 objects.

But you are not stuck with the inefficient methods just because you are using Java. Java also supports NIO (and I think GAE supports some of it), and Google also created protocol buffers, which strikes a balance of being much more efficient then JSON/XML, while still being language neutral. https://developers.google.com/protocol-buffers/docs/overview

So, if parsing 'a few 1000 java objects' is going to be normal for your app, I would suggest you consider a different representation for your objects.