从关系表插入到面向对象表的快速方法

Question

I have a relational table with lots of columns. (import_table) I'm trying to insert all of this data into an object-oriented database.

The object oriented database has tables:

#table (tableId, name)
#row (rowId, table_fk)
#column(colId, table_fk, col_name)
#value(valueId, col_fk, row_fk)

So far I have created a procedure that will read the import_table information_schema and insert the table and the columns correctly into the object-orientated structure. I then copy the import_data into a temp table with an extra identity-column just to get row-ids. Then iterate through all rows, with an inner loop to iterate through each column and do an insert pr. column. Like this:

SELECT ROWID=IDENTITY(INT, 1, 1), * INTO #TEST
FROM import_table

DECLARE @COUNTER INT = 1
WHILE @COUNTER <= (SELECT COUNT(*) FROM #TEST)
BEGIN
    INSERT INTO #ROW (ROWID, TABLE_FK) VALUES(@COUNTER, 1)
    DECLARE @COLUMNCOUNTER INT = 1
    WHILE @COLUMNCOUNTER <= (SELECT COUNT(*) FROM #COLUMN WHERE TABLE_FK = 1)
    BEGIN
        DECLARE @COLNAME NVARCHAR(254) = select col_name from #column where table_fk = 1 and rowid = @columnCounter
        DECLARE @INSERTSQL NVARCHAR(1000) = 'insert into #value (column_fk, row_fk, value) select '+cast(@columnCounter as nvarchar(20))', '+cast(@counter as nvarchar(20))+', ' + @colName+' from #test where rowId = '+cast(@counter as nvarchar20))
        exec (@insertSQL)
        set @columncounter = @columncounter +1
    end
    set @counter = @counter +1
end

This works, but it is extremely slow. Any suggestions on how to speed things up?

Answer 1

Redesign the database.

What you have there is a property bad type of table. These are known trade off setups, and guess what - speed is the item they are bad in. Ouch.

You likely could do things faster in C# outside and then stream the inserts back in.

Answer 2

One way to speed up your code is to wrap your inserts in transactions (maybe one transaction per iteration of the outer loop?). Having each one in a separate transaction, as the code is now, will be very slow if there are lots of inserts.