[英]Azure Stream Analytics Job - Transformation Query - correct formatting in ARM template
When editing a Stream Analytics transformation query in the Portal, you can format it for readability across multiple lines...eg在门户中编辑 Stream Analytics 转换查询时,您可以对其进行格式化以便跨多行阅读...例如
SELECT
INTO [Output1]
FROM [Input1]
PARTITION BY PartitionId
WHERE etc etc etc
When putting this into an ARM template for CI/CD, this is entered as one massive long string and would end up displaying in the portal as...当将其放入 CI/CD 的 ARM 模板时,它作为一个巨大的长字符串输入,最终将在门户中显示为...
SELECT * INTO [Output1] FROM [Input1] PARTITION BY PartitionId WHERE etc etc etc to infinity....
The official documentation is pretty useless and doesn't give any clues for the query part of the template, just that it is a "string"...官方文档非常无用,没有为模板的查询部分提供任何线索,只是它是一个“字符串”......
https://learn.microsoft.com/en-us/azure/templates/microsoft.streamanalytics/2016-03-01/streamingjobs/transformations https://learn.microsoft.com/en-us/azure/templates/microsoft.streamanalytics/2016-03-01/streamingjobs/transformations
There is a Microsoft sample template that is the only example I could find with a transform query specified... https://github.com/Azure/azure-quickstart-templates/blob/master/101-streamanalytics-create/azuredeploy.json ...and it looks like it is trying to do spacing...有一个 Microsoft 示例模板,这是我可以找到的唯一一个指定转换查询的示例... https://github.com/Azure/azure-quickstart-templates/blob/master/101-streamanalytics-create/azuredeploy。 json ...看起来它正在尝试间距...
"query": "SELECT\r\n *\r\nINTO\r\n [YourOutputAlias]\r\nFROM\r\n [YourInputAlias]"
...but failing badly - see screenshot ...但失败得很严重 - 请参阅屏幕截图
Has anyone managed to do this?有没有人设法做到这一点?
Also does anyone know why you can see the transformation query in the Azure Resource Explorer ( https://resources.azure.com/ )?还有人知道为什么您可以在 Azure 资源浏览器 ( https://resources.azure.com/ ) 中看到转换查询吗? Or that it cannot be exported from the portal with the rest of the Stream Job?
还是无法使用 Stream 作业的 rest 从门户导出? (done at Resource Group level)
(在资源组级别完成)
Thanks in advance提前致谢
I know it is a full year later and perhaps you've figured this out already, however, this is what I did:我知道这是一整年之后,也许你已经想通了,但是,这就是我所做的:
In my Parameters file, I used an array of strings, for example:在我的参数文件中,我使用了一个字符串数组,例如:
"StreamAnalyticsJobQueryMultiline": {
"value": [
"WITH allData AS ( ",
" SELECT ",
" *, ",
" GetMetadataPropertyValue([%%INPUTSTREAMNAME%%], '[User].[EventNamespace]') AS EventNamespace ",
" FROM [%%INPUTSTREAMNAME%%] Partition By PartitionId ",
"SELECT ",
" *, ",
" 'EventHubWriterv1' AS EventType ",
"INTO ",
" [%%OUTPUTSTREAMNAME%%] ",
"FROM ",
" allData Partition By PartitionId "
]
When the array is concatenated, and output as a string, it produces something like this, where each item in that array is still enclosed by the quotation marks and the entire thing is contained within square braces (see: https://learn.microsoft.com/en-us/azure/azure-resource-manager/templates/template-functions-string#concat )当数组连接起来时,output 作为一个字符串,它会产生这样的东西,其中该数组中的每个项目仍然用引号括起来,整个东西都包含在方括号中(参见: https://learn.microsoft .com/en-us/azure/azure-resource-manager/templates/template-functions-string#concat )
["Foo","Bar","Blah"]
[“富”,“酒吧”,“等等”]
So additional transformation in order to turn it into something readable in the Stream Analytics output is required.因此,需要进行额外的转换以将其转换为 Stream 分析 output 中可读的内容。
Also note here the %%INPUTSTREAMNAME%% and %%OUTPUTSTREAMNAME%%, as both my input and output streams are also parameters, and using the typical inline [parameter('ParamName')] did not work nicely with the rest of the transformation needed.还要注意这里的 %%INPUTSTREAMNAME%% 和 %%OUTPUTSTREAMNAME%%,因为我的输入流和 output 流也是参数,并且使用典型的内联 [parameter('ParamName')] 不能很好地处理转换的 rest需要。
In my Template file, I take the StreamAnalyticsJobQueryMultiline parameter and use the variables field to do this transformation:在我的模板文件中,我使用StreamAnalyticsJobQueryMultiline参数并使用变量字段来执行此转换:
"QueryStringRaw": "[string(concat(parameters('StreamAnalyticsJobQueryMultiline')))]",
// Update the end-of-lines by removing the doublequote and comma, and replacing it with a newline character
"QueryStringIter1": "[replace(variables('QueryStringRaw'), '\",', '\n')]",
// Update the beginning-of-lines by removing the doublequote
"QueryStringIter2": "[replace(variables('QueryStringIter1'), '\"', '')]",
// Update the InputStreamName and OutputStreamName values
"QueryStringIter3": "[replace(variables('QueryStringIter2'), '%%INPUTSTREAMNAME%%', parameters('InputStreamName'))]",
"QueryStringIter4": "[replace(variables('QueryStringIter3'), '%%OUTPUTSTREAMNAME%%', parameters('OutputStreamName'))]",
// Produce the final output for the query string by trimming the leading and trailing square brackets
"QueryStringFinal": "[substring(variables('QueryStringIter4'), 1, sub(length(variables('QueryStringIter4')), 2))]"
Then I reference that in the transformation portion of the Microsoft.StreamAnalytics/streamingjobs properties:然后我在 Microsoft.StreamAnalytics/streamingjobs 属性的转换部分引用它:
"transformation": {
"name": "Transformation",
"properties": {
"streamingUnits": "[parameters('StreamAnalyticsStreamingUnitsScale')]",
"query": "[variables('QueryStringFinal')]"
}
}
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.