[英]Reading streaming data using Google Cloud Dataflow SDK
I am working on an IOT application where I need to read streaming data from a PubSub topic. 我正在开发需要从PubSub主题读取流数据的IOT应用程序。 I want to read this data using Google Cloud Dataflow SDK. 我想使用Google Cloud Dataflow SDK读取此数据。 I am using Java 1.8 我正在使用Java 1.8
I am using the trial version of Google Cloud Platform. 我正在使用Google Cloud Platform试用版。 When I was using PubSubIO.Read method to read the streaming data, I was getting errors in the log file that my project does not have enough CPU quota to run the app. 当我使用PubSubIO.Read方法读取流数据时,在日志文件中收到错误消息,表明我的项目没有足够的CPU配额来运行该应用程序。
So I want to read the streaming data using Google Cloud Dataflow SDK. 因此,我想使用Google Cloud Dataflow SDK读取流数据。
Can someone please let me know where can I find complete examples of reading the streaming data using Google Cloud Dataflow SDK. 有人可以告诉我在哪里可以找到使用Google Cloud Dataflow SDK读取流数据的完整示例。
Thanks in advance. 提前致谢。
A number of complete examples are available in Cloud Dataflow documentation under Complete Examples and, linked from there, on Github, also under Complete Examples . Cloud Dataflow文档中“ 完整示例”下提供了许多完整示例,并且在Github上“ 完整示例”下也提供了许多完整示例 。
According to your error message, you indeed have not enough CPU quota to run the default 3x 4-threaded ( n1-standard-4
) workers. 根据错误消息,您确实没有足够的CPU配额来运行默认的3x 4线程( n1-standard-4
)工作线程。 The CPU quota for Google Cloud trial is 8 CPUs. Google Cloud试用版的CPU配额为 8个CPU。
You can configure your job to require fewer CPUs, eg by using fewer workers (eg --numWorkers=1
) or a different machine type ( --workerMachineType=n1-standard-1
) 您可以将作业配置为需要更少的CPU,例如通过使用更少的工作程序(例如--numWorkers=1
)或其他机器类型( --workerMachineType=n1-standard-1
)。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.