导入bcp输出，在日期时间进行倒推

Question

I've been beating my head against this for a while, so I'm finally asking for some suggestions. We have a very unfortunately designed table that doesn't perform well, and it's been out there long enough that was have many customers who've put around 50M rows each into it.

I've been experimenting with ways of fixing the table design, but obviously, the biggest constraint is how to migrate the data efficiently. Dropping primary keys and having the table reshuffle blows out the disk space on the transaction log. Creating a new table and doing INSERT new SELECT * OLD likewise.

I started experimenting with bcp , and trying to follow recommendations I found here (this was linked from an Oflow article - http://www.sommarskog.se/bulkload.html ) but so far I'm stymied by a problem on the import. It always fails on the first row, trying to process the datetime column. Nothing I've found in any Oflow article seems to address the problem.

I did the bcp export with -N to get all native mode, created a format file with -N. The new table I tried to import into was just different in the primary keys and indexes, not the column layout; that didn't work. I tried creating an empty table of the same structure; that didn't work. I tried all 3 import methods described in the above article; none of them got past the datetime import issue.

I'm hoping someone can direct me how to get past the datetime import issue.

The table schema:

CREATE TABLE [Notes](
[NoteID] [int] IDENTITY(1,1) NOT NULL,
[ObjectTypeID] [tinyint] NOT NULL,
[ObjectID] [int] NOT NULL DEFAULT ((0)),
[CreationTime] [datetime] NOT NULL,
[NoteTypeID] [tinyint] NOT NULL,
[AuthorUserID] [int] NOT NULL,
[AuthorName] [nvarchar](200) NULL,
[AuthorEmail] [nvarchar](100) NULL,
[Message] [nvarchar](max) NOT NULL,
[ObjectSubTypeID] [tinyint] NOT NULL,
[IpAddr] [nvarchar](48) NULL,
[Source] [tinyint] NULL,
CONSTRAINT [PK_RC_Note] PRIMARY KEY CLUSTERED 
(
[ObjectTypeID] ASC,
[ObjectID] ASC,
[CreationTime] DESC,
[NoteID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]

The export method:

bcp NotesTest.Notes out .\Notes2.bcp -c -N -U foo -P bar -S dbserver
bcp NotesTest.Notes format nul -T  -f Notes.fmt -N -U foo -P bar -S dbserver

The import methods tried:

bcp NotesTest.Notes in .\Notes2.bcp -U foo -P bar -S dbserver -h TABLOCK -b 50000 -f Notes.fmt

BULK INSERT Notes
FROM '\\myserver\myshare\notes2.bcp'
WITH (
     BATCHSIZE      = 50000
    ,CODEPAGE       = 'RAW'
    ,DATAFILETYPE   = 'native'
    ,FIRSTROW       = 1
    ,FORMATFILE     = '\\myserver\myshare\Notes.fmt'
    ,MAXERRORS      = 20
    ,TABLOCK
    );

SET IDENTITY_INSERT RC.RC_Note ON
INSERT INTO RC.RC_Note (NoteID, ObjectTypeID, ObjectID, CreationTime, NOteTypeID,AuthorUserID,AUthorName, AuthorEmail,Message,ObjectSubTypeID,IpAddr,Source) 
    SELECT NoteID, ObjectTypeID, ObjectID, CreationTime, NOteTypeID,AuthorUserID,AUthorName, AuthorEmail,Message,ObjectSubTypeID,IpAddr,Source
    FROM OPENROWSET(BULK '\\myserver\myshare\notes2.bcp',
    FORMATFILE='\\myserver\myshare\Notes.fmt'
) AS t1;

The error message received on all the imports (with a little different dressing around it, depending on the method):

Bulk load data conversion error (truncation) for row 1, column 4 (CreationTime).

Thanks

Answer 1

Please check out this description of native format import options.

Datafiletype for a native BCP import should be 'Native', and no format file should be used If the tables are identical. Here are two specific command line options:

Command      Option                  Description
bcp            -n                    Causes the bcp utility to use the native data types of the data.
BULK INSERT  DATAFILETYPE ='native'  Uses the native or wide native data types of the data. Note that DATAFILETYPE is not needed if a format file specifies the data types.

Try the default native file imports, and see if they work for you.

Answer 2

So it appears to be a silent but deadly conflict of export options. I noticed when going back to review the link in Vergil's answer that I'd accidentally left -c in the export line when I switched to native mode. Bcp didn't complain, but the output file it produced doesn't appear to be valid.

bcp NotesTest.Notes out .\Notes2.bcp -c -N -U foo -P bar -S dbserver