Having problems parsing strings into objects in javascript and JSON

Question

I am querying a Cassandra database and then, trying to convert one of its BLOB entries into a javascript object.

Currently there is one line that gets the data/blob I am interested:

data[x][0]

When I console.log it with:

console.log("data[x][0]: ", data[x][0]);

The output is:

data[x][0]:  <Buffer 7b 22 70 72 65 66 69 78 22 3a 20 22 65 6e 77 69 6b 69 22 2c 20 22 74 69 74 6c 65 22 3a 20 22 22 41 67 68 6e 61 64 61 72 72 61 67 68 22 22 7d>

So I concluded its some hexadecimal buffer object (which I've seen before), so I tried to make it into a string with the toString() method and then after that maybe we can parse it and make it into a javascript object!

So first, I tested the toString() method:

console.log("data[x][0].toString(): ", data[x][0].toString());

The following is the output:

data[x][0].toString():  {"prefix": "enwiki", "title": ""Aghnadarragh""}

Which looks perfect! That is the object I am trying to create! (Though right now is a string)

So...why not parse the dam thing with JSON? It looks like JSON...

console.log( "JSON.string(data[x][0]): ", JSON.parse( data[x][0].toString() ) );

Well, if I do that I get an error thrown by the JSON library:

events.js:72
        throw er; // Unhandled 'error' event
              ^
SyntaxError: Unexpected token A
    at Object.parse (native)
    at CassandraBackend.queryCB2 (/home/brandomiranda/Documents/6.S194/testreduce/CassandraBackend.js:831:70)
    at Object.callback (/home/brandomiranda/Documents/6.S194/testreduce/node_modules/node-cassandra-cql/index.js:248:16)
    at Connection.handleFrame (/home/brandomiranda/Documents/6.S194/testreduce/node_modules/node-cassandra-cql/lib/connection.js:244:15)
    at FrameParser.<anonymous> (/home/brandomiranda/Documents/6.S194/testreduce/node_modules/node-cassandra-cql/lib/connection.js:45:12)
    at FrameParser.EventEmitter.emit (events.js:92:17)
    at emitReadable_ (_stream_readable.js:408:10)
    at emitReadable (_stream_readable.js:404:5)
    at readableAddChunk (_stream_readable.js:165:9)
    at FrameParser.Readable.push (_stream_readable.js:127:10)

Which from the error message, I would guess the string I am giving it forms some token that cannot be parsed properly (probably because it doesn't obey the JSON grammar would be my guess).

However, if its not JSON, then what is it? Its also hard to know what grammar it comes from because, I don't really know what type of object the original thing is:

Doing:

console.log(typeof data[x][0]);

Doesn't yield useful information:

object

How do I make that string into a Javascript object?

---

NOTICE: this project is using Cassandra 2.0.5

This comes from a open source project, the exact line were this code is generated can be found at:

https://github.com/gwicke/testreduce/blob/master/CassandraBackend.js#L830

A more careful look into the whole open source we can confirm that the data[x][0] indeed comes from Cassandra directly and its a blob.

The data[x] indexes a specific row of the Cassandra query. data[x][0] indexes the column. Further at the top we can see that the rows its fetching from the db are specified by query:

query: 'select test, score from test_by_score where commit = ?'

Thus, index 0 gets the column "test" from the table test_by_score . We can confirm its a blob from:

    CREATE TABLE IF NOT EXISTS test_by_score (
        commit blob, 
        delta int, 
        score int, 
        test blob, 
    PRIMARY KEY(commit, delta, test) );

So my deductions are that either Cassandra is giving something wrong and not produce correct JSON objects or...whatever was inserted originally at test was not inserted as a JSON object correctly...which one might it be? let's explore some more!

---

Made a gitissue for this bug on the open source repo

For reference, I have also made a gitissue on github explaining the problem and the things that have been done to address it (along with any discussion that happen to address this bug.

https://github.com/gwicke/testreduce/issues/43

The gitissue also provides a reference for people to know how the issue is being addressed on the original repo.

Answer 1

I think the problem is in the framework: since it seems to use double quote notation to insert values into Cassandra. In JSON the preferred one would be single quote notation.

You have two ways dealing with this problem: either you use single quotation for your upcoming upserts (old data must be truncated as it is not readable) or you fix the problem with a workaround: the comment from 'Aadit M Shah' to drop the inner double quotation marks may work -- but maybe does your application need them. So I think the problem needs some more complited RegExp since you wish to replace the inner double quotes only.

JSON.parse(data[x][0].toString().replace(/\"\"\(.*)\"\"/g, /\"\'$1\'\"/)

may solve the problem.

Also consider changing the BLOB to VARCHAR as discussed here: Thinking of storing serialized java objects into cassandra as JSON. What is the catch?