[英]Can we use spark pool to process data from dedicated SQL pool and is that a good architecture?
Is it a good design to use spark pool for processing data which comes in dedicated SQL pool and again write back to dedicated SQL pool and to adls.使用火花池处理专用 SQL 池中的数据并再次写回专用 SQL 池和 adls 是否是一个好的设计。
As of now everything we r doing with dedicated SQL pool so if we add spark pool so will it be more efficient or it will just be burden to existing dedicated SQL pool.截至目前,我们 r 使用专用 SQL 池所做的一切,所以如果我们添加火花池,它会更有效率,否则只会成为现有专用 SQL 池的负担。
Yes, you can use spark pool to process data from dedicated SQL pool and is that a good architecture as there it is recommended and directly support by Microsoft Officials.是的,您可以使用 spark 池来处理来自专用 SQL 池的数据,这是一个很好的体系结构,因为 Microsoft 官方推荐并直接支持它。
The Synapse Dedicated SQL Pool Connector is an API that efficiently moves data between Apache Spark runtime and Dedicated SQL pool in Azure Synapse Analytics.
Synapse 专用 SQL 池连接器是一个 API,可在 Apache Spark 运行时和 Azure Synapse Analytics 中的专用 SQL 池之间高效移动数据。 This connector is available in Scala.
此连接器在 Scala 中可用。
If your project required large scale streaming you can definitely go for Apache Spark.如果您的项目需要大规模流式传输,您绝对可以使用 go 来使用 Apache Spark。 There won't be any burden on existing architecture.
不会对现有架构造成任何负担。 You will get expected results.
你会得到预期的结果。
Refer: Azure Synapse Dedicated SQL Pool connector for Apache Spark参考: Azure Synapse Dedicated SQL Pool connector for Apache Spark
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.