[英]Specify worker pool for Apache TinkerPop's Spark-Gremlin - DataStax-Enterprise Graph-Analytics
I need to designate a specific worker pool to run gremlin olap queries.我需要指定一个特定的工作池来运行 gremlin olap 查询。 When I run gremlin olap queries using gremlin console or datastax studio it runs under the default pool (which is not what I want).
当我使用 gremlin 控制台或 datastax studio 运行 gremlin olap 查询时,它在默认池下运行(这不是我想要的)。 I want to run the gremlin olap queries under a specific worker pool eg gremlin_olap or be able to specify the memory and executors.
我想在特定的工作池下运行 gremlin olap 查询,例如 gremlin_olap 或者能够指定 memory 和执行程序。 I tried a few settings in dse.yaml (in the location resources/dse/conf) and olap.properties (in the location resources/graph/conf), I re-started the cluster still not able to force to use gremlin_olap worker pool.
我在 dse.yaml(在位置资源/dse/conf)和 olap.properties(在位置资源/graph/conf)中尝试了一些设置,我重新启动集群仍然无法强制使用 gremlin_olap 工作池.
olap.properties olap.properties
spark.scheduler.pool=gremlin_olap
spark.executor.cores=2
spark.executor.memory=2g
dse.yaml dse.yaml
resource_manager_options:
worker_options:
cores_total: 0.7
memory_total: 0.6
workpools:
- name: alwayson_sql
cores: 0.25
memory: 0.25
- name: gremlin_olap
cores: 0.25
memory: 0.25
Gremlin console bin/dse gremlin-console Gremlin 控制台bin/dse gremlin-console
\,,,/
(o o)
-----oOOo-(3)-oOOo-----
plugin activated: tinkerpop.server
plugin activated: tinkerpop.tinkergraph
gremlin> :remote config alias g identity.a
==>g=identity.a
gremlin> g.V().groupCount().by(label)
==>{identity=50000}
gremlin>
Am I missing something?我错过了什么吗?
These directions should help:这些指示应该有助于:
https://docs.datastax.com/en/dse/6.8/dse-dev/datastax_enterprise/graph/graphAnalytics/graphAnalyticsSparkGraphComputer.html#SettingSparkpropertiesfromGremlin https://docs.datastax.com/en/dse/6.8/dse-dev/datastax_enterprise/graph/graphAnalytics/graphAnalyticsSparkGraphComputer.html#SettingSparkpropertiesfromGremlin
This doesn't exactly create a Spark resource pool — but it does affect the resources that the Gremlin OLAP Spark application will use — and the way it works in DSE Graph is that there will only ever be one of these applications spun up, so it has the same effect as having a Spark resource pool.这并不完全创建一个 Spark 资源池——但它确实会影响 Gremlin OLAP Spark 应用程序将使用的资源——它在 DSE Graph 中的工作方式是这些应用程序中只会有一个启动,所以它与拥有 Spark 资源池的效果相同。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.