Getting error on null and empty string while copying a csv file from blob container to Azure SQL DB

Question

I tried all combination on the datatype of my data but each time my data factory pipeline is giving me this error:

{ "errorCode": "2200", "message": "ErrorCode=UserErrorColumnNameNotAllowNull,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Empty or Null string found in Column Name 2. Please make sure column name not null and try again.,Source=Microsoft.DataTransfer.Common,'", "failureType": "UserError", "target": "xxx", "details": [] }

My Copy data source code is something like this: { "name": "xxx", "description": "uuu", "type": "Copy", "dependsOn": [], "policy": { "timeout": "7.00:00:00", "retry": 0, "retryIntervalInSeconds": 30, "secureOutput": false, "secureInput": false }, "userProperties": [], "typeProperties": { "source": { "type": "DelimitedTextSource", "storeSettings": { "type": "AzureBlobStorageReadSettings", "recursive": true, "wildcardFileName": "*" }, "formatSettings": { "type": "DelimitedTextReadSettings" } }, "sink": { "type": "AzureSqlSink" }, "enableStaging": false, "translator": { "type": "TabularTranslator", "mappings": [ { "source": { "name": "populationId", "type": "Guid" }, "sink": { "name": "PopulationID", "type": "String" } }, { "source": { "name": "inputTime", "type": "DateTime" }, "sink": { "name": "inputTime", "type": "DateTime" } }, { "source": { "name": "inputCount", "type": "Decimal" }, "sink": { "name": "inputCount", "type": "Decimal" } }, { "source": { "name": "inputBiomass", "type": "Decimal" }, "sink": { "name": "inputBiomass", "type": "Decimal" } }, { "source": { "name": "inputNumber", "type": "Decimal" }, "sink": { "name": "inputNumber", "type": "Decimal" } }, { "source": { "name": "utcOffset", "type": "String" }, "sink": { "name": "utcOffset", "type": "Int32" } }, { "source": { "name": "fishGroupName", "type": "String" }, "sink": { "name": "fishgroupname", "type": "String" } }, { "source": { "name": "yearClass", "type": "String" }, "sink": { "name": "yearclass", "type": "String" } } ] } }, "inputs": [ { "referenceName": "DelimitedTextFTDimensions", "type": "DatasetReference" } ], "outputs": [ { "referenceName": "AzureSqlTable1", "type": "DatasetReference" } ] } { "name": "xxx", "description": "uuu", "type": "Copy", "dependsOn": [], "policy": { "timeout": "7.00:00:00", "retry": 0, "retryIntervalInSeconds": 30, "secureOutput": false, "secureInput": false }, "userProperties": [], "typeProperties": { "source": { "type": "DelimitedTextSource", "storeSettings": { "type": "AzureBlobStorageReadSettings", "recursive": true, "wildcardFileName": "*" }, "formatSettings": { "type": "DelimitedTextReadSettings" } }, "sink": { "type": "AzureSqlSink" }, "enableStaging": false, "translator": { "type": "TabularTranslator", "mappings": [ { "source": { "name": "populationId", "type": "Guid" }, "sink": { "name": "PopulationID", "type": "String" } }, { "source": { "name": "inputTime", "type": "DateTime" }, "sink": { "name": "inputTime", "type": "DateTime" } }, { "source": { "name": "inputCount", "type": "Decimal" }, "sink": { "name": "inputCount", "type": "Decimal" } }, { "source": { "name": "inputBiomass", "type": "Decimal" }, "sink": { "name": "inputBiomass", "type": "Decimal" } }, { "source": { "name": "inputNumber", "type": "Decimal" }, "sink": { "name": "inputNumber", "type": "Decimal" } }, { "source": { "name": "utcOffset", "type": "String" }, "sink": { "name": "utcOffset", "type": "Int32" } }, { "source": { "name": "fishGroupName", "type": "String" }, "sink": { "name": "fishgroupname", "type": "String" } }, { "source": { "name": "yearClass", "type": "String" }, "sink": { "name": "yearclass", "type": "String" } } ] } }, "inputs": [ { "referenceName": "DelimitedTextFTDimensions", "type": "DatasetReference" } ], "outputs": [ { "referenceName": "AzureSqlTable1", "type": "DatasetReference" } ] } Can anyone please help me understand the issue. I see in some blogs they ask me use treatnullasempty but I am not allowed to modify the JSON. is there a way to do that??

Answer 1

I suggest to using Data Flow DerivedColumn , DerivedColumn can help you build expression to replace the null column.

For example:

Derived Column, if Column_2 is null = true , return 'dd' :

iifNull(Column_2,'dd')

Mapping the column

Reference: Data transformation expressions in mapping data flow

Hope this helps.

Answer 2

修复了它。这是一个简单的修复，因为我在目的地中的一列被标记为非空，我将其更改为空并且它起作用了。