JSON的Google云数据存储对象树

Question

I'm probably missing something obvious, but I can't find a way to store a JSON object with descendants in such a way that the Datastore DB has a hierarchy of entities that can be queried. Example code:

const Datastore = require('@google-cloud/datastore');
const datastore = new Datastore({
  projectId: 'MY_PROJECT_ID',
});

const ot1 = {
  // The kind and ID for the new entity
  key: datastore.key(['OT', '1']),
  data: {
    obj: [
      {
        var1: 'var1',
        var2: 'var2',
      },
      {
        var3: [{obj2:'var3'}, 'var5'],
        var4: 'var4',
      },
    ] // obj
  } // data
};

datastore.save(ot1);

If I now check my Datastore, it contains one entity 'obj' of type Array (as expected), but the value is one big JSON blob representing the array, ie I cannot drill down into descendant objects (which are obj[0] , obj[1] , obj[1].var3 , and obj[1].var3[0] ) from the Datastore console (whereas with AWS DynamoDB, and same JSON, I am able to drill down -- as though it handles the conversion from JS objects to DB format automatically). Yet, the docs indicate that the parent of an entity cannot be changed after entity created, so the parent entity has to be created before the child entity. It seems like the API requires that I save in multiple steps:

1. Save root object, with empty list
2. Save each child of root, setting their parent to root ( obj ); not clear how to add each child to the array; obj[1] would have empty Array as var3 property value
3. Save each element of obj[1].var3 array, setting their parent to obj[1] ; same issue about array

This is a lot of work, surely there is a library function that does this automatically, but I am unable to find one. And I'm not sure what this would look like in Datastore console, I suspect you would end up with 4 objects, ie hierarchy would only be indirectly known by observing the parent properties.

Update : I suspect the answer is to use an ORM like js-data.

Answer 1

IMHO it's not possible to automatically store the json data in a structured format without specifying that structured format - how would it be decided if a particular data portion should be indeed a separate entity or plain dumped in the same entity? - either one can be desirable depending on the application.

Not entirely sure about go (I'm not a go user), but python has support for Structured Properties which allow querying below the property level.

I see that go supports Structured Properties as well, but I found no good reference about querying and bulk-filling them. From Properties and value types :

You can also use a struct or slice to aggregate properties. See the Cloud Datastore reference for more details.

This would be a somehow similar question, but for python (it does involve specifying the structured format): What is the best practice to populate a StructuredProperty through the ndb.Model constructor? Maybe a similar approach can be applied in go.

I think this may be relevant, too: datastore: retrieving an entity's key as a struct field #453

Answer 2

You said:

"the parent of an entity cannot be changed after entity created, so the parent entity has to be created before the child entity"

But I think it is not exactly like this since documentation says...

"Because it is part of the entity's key, the identifier is associated permanently with the entity and cannot be changed ." [1]

and also:

"When you create an entity, you can optionally designate another entity as its parent; the new entity is a child of the parent entity (note that unlike in a file system, the parent entity need not actually exist )." [2]

Anyway, if you can't work like you want in your language (Node.js) and the solutions provided by Dan Cornilescu are not enough for you (or even if they are), you always can open a future request here about your Cloud Datastore concerns in the Google's issuetracker with component "Public Trackers > Cloud Platform > Storage and Databases > Cloud Datastore", (like this [3] )

1.- https://cloud.google.com/datastore/docs/concepts/entities#kinds_and_identifiers

2.- https://cloud.google.com/datastore/docs/concepts/entities#ancestor_paths

3.- https://issuetracker.google.com/issues/new?component=187197&template=1010240