OrientDB ETL with JSON data file

Question

I am not seeing good documentation on how to use the OrientDB ETL function to load a JSON data file.

I am running this command: ./oetl.sh ../template_etl.json
The contents of template_etl.json looks like this:

{
    "config": {
        "log": "debug"
    },
    "begin": [
    ],
    "source" : {
        "file": {"path": "../repos.json", "lock" : true }
    },
    "extractor" : {
        "row": {}
    },
    "transformers" : [
        {"json"},
        { "vertex": { "class": "V" } }
    ],
    "loader" : {
        "orientdb": {
            "dbURL": "plocal../databases/template",
            "dbUser": "admin",
            "dbPassword": "admin",
            "dbAutoCreate": true,
            "tx": false,
            "batchCommit": 1000,
            "dbType": "graph"
        }
    }
}

I took this example from a csv example from https://www.udemy.com/orientdb-getting-started/#/lecture/1998370 where this line: {"json"}, was originally: {"csv": {"separator": ",", "multiValue": "NULL", "skipFrom": 1, "skipTo": 1 } },

The Error I am getting is: orientdb-community-2.0/bin$ ./oetl.sh ../template_etl.json

OrientDB etl v.2.0 (build @BUILD@) www.orientechnologies.com
Exception in thread "main" com.orientechnologies.orient.core.exception.OSerializationException: Error on unmarshalling JSON content for record: "config": {
        "log": "debug"
    },
    "begin": [
    ],
    "source" : {
        "file": {"path": "../repos.json", "lock" : true }
    },
    "extractor" : {
        "row": {}
    },
    "transformers" : [
        {"json"},
        { "vertex": { "class": "V" } }
    ],
    "loader" : {
        "orientdb": {
            "dbURL": "plocal../databases/template",
            "dbUser": "admin",
            "dbPassword": "admin",
            "dbAutoCreate": true,
            "tx": false,
            "batchCommit": 1000,
            "dbType": "graph"
        }
    }

    at   com.orientechnologies.orient.core.serialization.serializer.record.string.ORecordSerializerJSON.fromString(ORecordSerializerJSON.java:304)
        at com.orientechnologies.orient.core.record.ORecordAbstract.fromJSON(ORecordAbstract.java:165)
        at com.orientechnologies.orient.core.record.impl.ODocument.fromJSON(ODocument.java:1712)
        at com.orientechnologies.orient.etl.OETLProcessor.main(OETLProcessor.java:147)
    Caused by: com.orientechnologies.orient.core.exception.OSerializationException: Error on unmarshalling JSON content: wrong format ""json"". Use <field> : <value>
        at com.orientechnologies.orient.core.serialization.serializer.record.string.ORecordSerializerJSON.fromString(ORecordSerializerJSON.java:181)
        at com.orientechnologies.orient.core.serialization.serializer.record.string.ORecordSerializerJSON.getValueAsRecord(ORecordSerializerJSON.java:595)
        at com.orientechnologies.orient.core.serialization.serializer.record.string.ORecordSerializerJSON.getValueAsObjectOrMap(ORecordSerializerJSON.java:565)
        at com.orientechnologies.orient.core.serialization.serializer.record.string.ORecordSerializerJSON.getValue(ORecordSerializerJSON.java:413)
        at com.orientechnologies.orient.core.serialization.serializer.record.string.ORecordSerializerJSON.parseCollection(ORecordSerializerJSON.java:677)
        at com.orientechnologies.orient.core.serialization.serializer.record.string.ORecordSerializerJSON.getValueAsEmbeddedCollection(ORecordSerializerJSON.java:659)
        at com.orientechnologies.orient.core.serialization.serializer.record.string.ORecordSerializerJSON.getValueAsCollection(ORecordSerializerJSON.java:638)
        at com.orientechnologies.orient.core.serialization.serializer.record.string.ORecordSerializerJSON.getValue(ORecordSerializerJSON.java:415)
        at com.orientechnologies.orient.core.serialization.serializer.record.string.ORecordSerializerJSON.fromString(ORecordSerializerJSON.java:249)
        ... 3 more

I'm hoping there is a way to load a JSON data file directly into OreintDB.

Thanks

Answer 1

The json is not valid. Try to validate with www.jsonlint.com. Try replacing:

{"json"},

With:

{"json": {} },

Answer 2

I'm no expert like Lvca but your source file has a json extension. Which means your extractor must be replaced with ("json": {}) and there are no "json" transformers.

"extractor" : {
    "json": {}
},
"transformers" : [
    { "vertex": { "class": "V" } }
],

http://orientdb.com/docs/last/Transformer.html