[英]Saving to Google Datastore from Google Dataflow
I am trying to save to Google Datastore from Google Dataflow Job it gives me this error我正在尝试从 Google Dataflow Job 保存到 Google Datastore 它给了我这个错误
My Code inside the DoFN is我在 DoFN 中的代码是
Datastore datastore= DatastoreOptions.getDefaultInstance().getService();
TrackingRequest rq = gson.fromJson(c.element().toString(), TrackingRequest.class);
Query<Entity> query = Query.entityQueryBuilder().kind("Post").filter(PropertyFilter.eq("postid", rq.postid))
.build();
QueryResults<Entity> posts = datastore.run(query);
if (posts == null || !posts.hasNext()) {
KeyFactory keyFactory = datastore.newKeyFactory().setKind("Post");
Key key = keyFactory.newKey(rq.postid);
Entity entity = Entity.newBuilder(key)
.set("appid", rq.appid)
.set("postid", rq.postid)
.set("title", rq.title)
.build();
datastore.put(entity);
// c.output(((FullEntity<IncompleteKey>)entity).toPb());
}
The error is :错误是:
exception: "java.lang.NoSuchMethodError: com.google.datastore.v1.Entity$Builder.putProperties(Ljava/lang/String;Lcom/google/datastore/v1/Value;)Lcom/google/datastore/v1/Entity$Builder;
at com.google.cloud.datastore.BaseEntity.toPb(BaseEntity.java:683)
at com.google.cloud.datastore.DatastoreImpl.put(DatastoreImpl.java:337)
at com.google.cloud.datastore.DatastoreHelper.put(DatastoreHelper.java:55)
at com.google.cloud.datastore.DatastoreImpl.put(DatastoreImpl.java:315)
at com.kryptonz.proccess.KryptonzArchive$RawToObjectConverter.processElement(KryptonzArchive.java:80)
at com.google.cloud.dataflow.sdk.util.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:49)
at
Faced the same issue and so thought the way out might help others too.面临同样的问题,因此认为出路也可能对其他人有所帮助。
I had google-cloud-datastore
(version 0.11.2-beta
) as my dependency which come up with datastore-v1-protos-1.0.1
.我有
google-cloud-datastore
(版本0.11.2-beta
)作为我的依赖项,它提出了datastore-v1-protos-1.0.1
。 And I believe the root cause for the mentioned problem was this libs specific class com.google.datastore.v1.Entity.Builder
, which didn't support neither putProperties
nor getPropertiesMap
.我相信上述问题的根本原因是这个库特定类
com.google.datastore.v1.Entity.Builder
,它既不支持putProperties
也不支持getPropertiesMap
。
Knowing the problem statement in a better way, helped me to sort the issue conventionally.以更好的方式了解问题陈述,帮助我按常规对问题进行排序。 I had to just make sure to get a new dependency of
datastore-v1-protos
which supports those missing APIs.我只需要确保获得支持那些缺少的 API 的
datastore-v1-protos
的新依赖项。 Hence just added below dependency in my pom.xml
因此刚刚在我的
pom.xml
添加了以下依赖项
<dependency>
<groupId>com.google.cloud.datastore</groupId>
<artifactId>datastore-v1-protos</artifactId>
<version>1.3.0</version>
</dependency>
This change might create some protobuf regression, so include below dependency in your pom.xml
to avoid, if there is any such.此更改可能会导致一些 protobuf 回归,因此请在
pom.xml
包含以下依赖项以避免(如果有)。
<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java</artifactId>
<version>3.2.0</version>
</dependency>
Check your dependency of datastore in pom.xml and the library you import.检查 pom.xml 中数据存储的依赖关系以及您导入的库。
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-datastore</artifactId>
<version>1.90.0<</version>
</dependency>
import com.google.cloud.datastore.Entity;
import com.google.cloud.datastore.Key;
And if the req.post id is the id/Name in the datastore entity then you can directly create key and check if it present in the datastore.如果 req.post id 是数据存储实体中的 id/Name,那么您可以直接创建密钥并检查它是否存在于数据存储中。
Datastore datastore =
Datastore datastore= DatastoreOptions.getDefaultInstance().getService();
Key key = datastore.newKeyFactory().setKind("Post").newKey(rq.postid);
if(key == null){
Entity entity = Entity.newBuilder(key)
.set("appid", rq.appid)
.set("postid", rq.postid)
.set("title", rq.title)
.build();
datastore.put(entity);
}
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.