[英]Getting error on null and empty string while copying a csv file from blob container to Azure SQL DB
I tried all combination on the datatype of my data but each time my data factory pipeline is giving me this error:我尝试了所有数据类型的组合,但每次我的数据工厂管道都给我这个错误:
{ "errorCode": "2200", "message": "ErrorCode=UserErrorColumnNameNotAllowNull,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Empty or Null string found in Column Name 2. Please make sure column name not null and try again.,Source=Microsoft.DataTransfer.Common,'", "failureType": "UserError", "target": "xxx", "details": [] } { "errorCode": "2200", "message": "ErrorCode=UserErrorColumnNameNotAllowNull,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=在列名 2 中找到空或空字符串。请确保列名不为空再试一次。,Source=Microsoft.DataTransfer.Common,'", "failureType": "UserError", "target": "xxx", "details": [] }
My Copy data source code is something like this: { "name": "xxx", "description": "uuu", "type": "Copy", "dependsOn": [], "policy": { "timeout": "7.00:00:00", "retry": 0, "retryIntervalInSeconds": 30, "secureOutput": false, "secureInput": false }, "userProperties": [], "typeProperties": { "source": { "type": "DelimitedTextSource", "storeSettings": { "type": "AzureBlobStorageReadSettings", "recursive": true, "wildcardFileName": "*" }, "formatSettings": { "type": "DelimitedTextReadSettings" } }, "sink": { "type": "AzureSqlSink" }, "enableStaging": false, "translator": { "type": "TabularTranslator", "mappings": [ { "source": { "name": "populationId", "type": "Guid" }, "sink": { "name": "PopulationID", "type": "String" } }, { "source": { "name": "inputTime", "type": "DateTime" }, "sink": { "name": "inputTime", "type": "DateTime" } }, { "source": { "name": "inputCount", "type": "Decimal" }, "sink": { "name": "inputCount", "type": "Decimal" } }, { "source": { "name": "inputBiomass", "type": "Decimal" }, "sink": { "name": "inputBiomass", "type": "Decimal" } }, { "source": { "name": "inputNumber", "type": "Decimal" }, "sink": { "name": "inputNumber", "type": "Decimal" } }, { "source": { "name": "utcOffset", "type": "String" }, "sink": { "name": "utcOffset", "type": "Int32" } }, { "source": { "name": "fishGroupName", "type": "String" }, "sink": { "name": "fishgroupname", "type": "String" } }, { "source": { "name": "yearClass", "type": "String" }, "sink": { "name": "yearclass", "type": "String" } } ] } }, "inputs": [ { "referenceName": "DelimitedTextFTDimensions", "type": "DatasetReference" } ], "outputs": [ { "referenceName": "AzureSqlTable1", "type": "DatasetReference" } ] }
我的复制数据源代码是这样的: { "name": "xxx", "description": "uuu", "type": "Copy", "dependsOn": [], "policy": { "timeout": "7.00:00:00", "retry": 0, "retryIntervalInSeconds": 30, "secureOutput": false, "secureInput": false }, "userProperties": [], "typeProperties": { "source": { "type": "DelimitedTextSource", "storeSettings": { "type": "AzureBlobStorageReadSettings", "recursive": true, "wildcardFileName": "*" }, "formatSettings": { "type": "DelimitedTextReadSettings" } }, "sink": { "type": "AzureSqlSink" }, "enableStaging": false, "translator": { "type": "TabularTranslator", "mappings": [ { "source": { "name": "populationId", "type": "Guid" }, "sink": { "name": "PopulationID", "type": "String" } }, { "source": { "name": "inputTime", "type": "DateTime" }, "sink": { "name": "inputTime", "type": "DateTime" } }, { "source": { "name": "inputCount", "type": "Decimal" }, "sink": { "name": "inputCount", "type": "Decimal" } }, { "source": { "name": "inputBiomass", "type": "Decimal" }, "sink": { "name": "inputBiomass", "type": "Decimal" } }, { "source": { "name": "inputNumber", "type": "Decimal" }, "sink": { "name": "inputNumber", "type": "Decimal" } }, { "source": { "name": "utcOffset", "type": "String" }, "sink": { "name": "utcOffset", "type": "Int32" } }, { "source": { "name": "fishGroupName", "type": "String" }, "sink": { "name": "fishgroupname", "type": "String" } }, { "source": { "name": "yearClass", "type": "String" }, "sink": { "name": "yearclass", "type": "String" } } ] } }, "inputs": [ { "referenceName": "DelimitedTextFTDimensions", "type": "DatasetReference" } ], "outputs": [ { "referenceName": "AzureSqlTable1", "type": "DatasetReference" } ] }
{ "name": "xxx", "description": "uuu", "type": "Copy", "dependsOn": [], "policy": { "timeout": "7.00:00:00", "retry": 0, "retryIntervalInSeconds": 30, "secureOutput": false, "secureInput": false }, "userProperties": [], "typeProperties": { "source": { "type": "DelimitedTextSource", "storeSettings": { "type": "AzureBlobStorageReadSettings", "recursive": true, "wildcardFileName": "*" }, "formatSettings": { "type": "DelimitedTextReadSettings" } }, "sink": { "type": "AzureSqlSink" }, "enableStaging": false, "translator": { "type": "TabularTranslator", "mappings": [ { "source": { "name": "populationId", "type": "Guid" }, "sink": { "name": "PopulationID", "type": "String" } }, { "source": { "name": "inputTime", "type": "DateTime" }, "sink": { "name": "inputTime", "type": "DateTime" } }, { "source": { "name": "inputCount", "type": "Decimal" }, "sink": { "name": "inputCount", "type": "Decimal" } }, { "source": { "name": "inputBiomass", "type": "Decimal" }, "sink": { "name": "inputBiomass", "type": "Decimal" } }, { "source": { "name": "inputNumber", "type": "Decimal" }, "sink": { "name": "inputNumber", "type": "Decimal" } }, { "source": { "name": "utcOffset", "type": "String" }, "sink": { "name": "utcOffset", "type": "Int32" } }, { "source": { "name": "fishGroupName", "type": "String" }, "sink": { "name": "fishgroupname", "type": "String" } }, { "source": { "name": "yearClass", "type": "String" }, "sink": { "name": "yearclass", "type": "String" } } ] } }, "inputs": [ { "referenceName": "DelimitedTextFTDimensions", "type": "DatasetReference" } ], "outputs": [ { "referenceName": "AzureSqlTable1", "type": "DatasetReference" } ] }
Can anyone please help me understand the issue. { "name": "xxx", "description": "uuu", "type": "Copy", "dependsOn": [], "policy": { "timeout": "7.00:00:00", "retry": 0, "retryIntervalInSeconds": 30, "secureOutput": false, "secureInput": false }, "userProperties": [], "typeProperties": { "source": { "type": "DelimitedTextSource", "storeSettings": { "type": "AzureBlobStorageReadSettings", "recursive": true, "wildcardFileName": "*" }, "formatSettings": { "type": "DelimitedTextReadSettings" } }, "sink": { "type": "AzureSqlSink" }, "enableStaging": false, "translator": { "type": "TabularTranslator", "mappings": [ { "source": { "name": "populationId", "type": "Guid" }, "sink": { "name": "PopulationID", "type": "String" } }, { "source": { "name": "inputTime", "type": "DateTime" }, "sink": { "name": "inputTime", "type": "DateTime" } }, { "source": { "name": "inputCount", "type": "Decimal" }, "sink": { "name": "inputCount", "type": "Decimal" } }, { "source": { "name": "inputBiomass", "type": "Decimal" }, "sink": { "name": "inputBiomass", "type": "Decimal" } }, { "source": { "name": "inputNumber", "type": "Decimal" }, "sink": { "name": "inputNumber", "type": "Decimal" } }, { "source": { "name": "utcOffset", "type": "String" }, "sink": { "name": "utcOffset", "type": "Int32" } }, { "source": { "name": "fishGroupName", "type": "String" }, "sink": { "name": "fishgroupname", "type": "String" } }, { "source": { "name": "yearClass", "type": "String" }, "sink": { "name": "yearclass", "type": "String" } } ] } }, "inputs": [ { "referenceName": "DelimitedTextFTDimensions", "type": "DatasetReference" } ], "outputs": [ { "referenceName": "AzureSqlTable1", "type": "DatasetReference" } ] }
任何人都可以帮我理解这个问题。 I see in some blogs they ask me use treatnullasempty but I am not allowed to modify the JSON.我在一些博客中看到他们要求我使用treatnullasempty,但我不允许修改JSON。 is there a way to do that??有没有办法做到这一点??
I suggest to using Data Flow DerivedColumn , DerivedColumn can help you build expression to replace the null column.我建议使用Data Flow DerivedColumn , DerivedColumn 可以帮助您构建表达式来替换空列。
Derived Column, if Column_2
is null = true
, return 'dd' :派生列,如果Column_2
为 null = true
,则返回 'dd' :
iifNull(Column_2,'dd')
Reference: Data transformation expressions in mapping data flow参考: 映射数据流中的数据转换表达式
Hope this helps.希望这可以帮助。
修复了它。这是一个简单的修复,因为我在目的地中的一列被标记为非空,我将其更改为空并且它起作用了。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.