Azure SQL Bulk Insert from Azure Storage

Question

I am trying to test the bulk insert using OPENRECORDSET command in Azure SQL with no success.

Destination Table...eventually

CREATE TABLE [dbo].[BCPTestTable](
  [Id] [int] IDENTITY(1,1) NOT NULL,
  [TextField] [varchar](500) NULL,
  [IntFIeld] [int] NULL,
  [DateField] [date] NULL,
  [DateTimeField] [datetime] NULL,
  [i18nTextField] [nvarchar](500) NULL,
  [BitField] [bit] NULL,
  [TInyIntField] [tinyint] NULL,
CONSTRAINT [PK_IX_BCPTestTable] PRIMARY KEY CLUSTERED 
(
  [Id] ASC
) WITH (STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF) ON [PRIMARY]
) ON [PRIMARY]

Data File

1   Jason   5   3/1/2018    3/1/2018 09:30:00   Test    1   50
2   Cindy   10  3/2/2018    3/2/2018 10:30:00   Testing 0   50

Format File

<?xml version="1.0"?>
<BCPFORMAT xmlns="http://schemas.microsoft.com/sqlserver/2004/bulkload/format" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <RECORD>
    <FIELD ID="1" xsi:type="CharFixed" LENGTH="12"/>
    <FIELD ID="2" xsi:type="CharFixed" LENGTH="500" COLLATION="SQL_Latin1_General_CP1_CI_AS"/>
    <FIELD ID="3" xsi:type="CharFixed" LENGTH="12"/>
    <FIELD ID="4" xsi:type="CharFixed" LENGTH="11"/>
    <FIELD ID="5" xsi:type="CharFixed" LENGTH="24"/>
    <FIELD ID="6" xsi:type="CharFixed" LENGTH="1000" COLLATION="SQL_Latin1_General_CP1_CI_AS"/>
    <FIELD ID="7" xsi:type="CharFixed" LENGTH="1"/>
    <FIELD ID="8" xsi:type="CharTerm" TERMINATOR="\r\n" MAX_LENGTH="5"/
  </RECORD>
  <ROW>
    <COLUMN SOURCE="1" NAME="Id" xsi:type="SQLINT"/>
    <COLUMN SOURCE="2" NAME="TextField" xsi:type="SQLVARYCHAR"/>
    <COLUMN SOURCE="3" NAME="IntFIeld" xsi:type="SQLINT"/>
    <COLUMN SOURCE="4" NAME="DateField" xsi:type="SQLDATE"/>
    <COLUMN SOURCE="5" NAME="DateTimeField" xsi:type="SQLDATETIME"/>
    <COLUMN SOURCE="6" NAME="i18nTextField" xsi:type="SQLNVARCHAR"/>
    <COLUMN SOURCE="7" NAME="BitField" xsi:type="SQLBIT"/>
    <COLUMN SOURCE="8" NAME="TInyIntField" xsi:type="SQLTINYINT"/>
  </ROW>
</BCPFORMAT>

For now, I'm just trying to select the data using the following query:

SELECT  ID,
        TextField,
        IntField,
        DateField,
        DateTimeField,
        i18nTextField,
        BitField,
        TInyIntField
FROM    OPENROWSET(
            BULK 'test\TestData.txt', 
            DATA_SOURCE = 'xyzstorage',
            FORMATFILE = 'test\BCPTestTableFormat.txt', 
            FORMATFILE_DATA_SOURCE = 'xyzstorage'
        ) as Data

Previous to this, I did create a database scoped credential as well as a external data source which appears to be working.

When I run the above SQL, I get the following error:

Msg 4862, Level 16, State 1, Line 1 Cannot bulk load because the file "test\\BCPTestTableFormat.txt" is incomplete or could not be read. Operating system error code 38(Reached the end of the file.).

I have tried format files using both the XML version and original version both of which where generated using the latest (v14) BCP command. In both cases, I get the same error.

Random stuff I've tried:

1. Change file name extensions (dumb but worth a shot)
2. Made sure both versions of the format file had a blank line at the end.
3. Changed the direction of the slashed (doesn't find file if the other way)

Please help!

Answer 1

Most likely there were characters added to the xml file such as spaces, or lines etc... Could use notepad ++ or a code editor, edit the sql /xml files then save and retry ? this might resolve your issue. This is a similar error message and similar resolution : https://www.sqlservercentral.com/Forums/Topic1490738-3077-1.aspx

Answer 2

Based upon the information made available in your post, there is a missing closing angle bracket on line 11:

 <?xml version="1.0"?> <BCPFORMAT xmlns="http://schemas.microsoft.com/sqlserver/2004/bulkload/format" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <RECORD> <FIELD ID="1" xsi:type="CharFixed" LENGTH="12"/> <FIELD ID="2" xsi:type="CharFixed" LENGTH="500" COLLATION="SQL_Latin1_General_CP1_CI_AS"/> <FIELD ID="3" xsi:type="CharFixed" LENGTH="12"/> <FIELD ID="4" xsi:type="CharFixed" LENGTH="11"/> <FIELD ID="5" xsi:type="CharFixed" LENGTH="24"/> <FIELD ID="6" xsi:type="CharFixed" LENGTH="1000" COLLATION="SQL_Latin1_General_CP1_CI_AS"/> <FIELD ID="7" xsi:type="CharFixed" LENGTH="1"/> <FIELD ID="8" xsi:type="CharTerm" TERMINATOR="\\r\\n" MAX_LENGTH="5"/ </RECORD>