MonetDB.R批量插入

Question

Is there a way to do a bulk insert using MonetDB.R (not via a for loop and dbSendUpdate)?

Does dbWriteTable allow for updates (append=TRUE)?

About "INSERT INTO" the MonetDB documentation states: "The benefit is clear: this is very straightforward. However, this is a seriously inefficient way of doing things in MonetDB."

Thanks.

Answer 1

Hannes might have a smarter solution, but for the time being, this might help :)

# start with an example data set
nrow( mtcars )

# and a MonetDB.R connection
db

# here's how many records you'd have if you stack your example data three times
nrow( mtcars ) * 3

# write to three separate tables
dbWriteTable( db , 'mtcars1' , mtcars )
dbWriteTable( db , 'mtcars2' , mtcars )
dbWriteTable( db , 'mtcars3' , mtcars )

# stack them all
dbSendUpdate( db , "CREATE TABLE mtcars AS SELECT * FROM mtcars1 UNION ALL SELECT * FROM mtcars2 UNION ALL SELECT * FROM mtcars3 WITH DATA" )

# correct number of records
nrow( dbReadTable( db , 'mtcars' ) )

Answer 2

I'll consider it. monetdb.read.csv does use COPY INTO, so you might get away with creating a temp. CSV file.

Answer 3

I see what you mean although this doesn't change anything to the fact that dbWriteTable uses a for loop and "INSERT INTO" which can be rather slow. I may not have been very clear in my initial post.

As a workaround I guess "START TRANSACTION" and "COMMIT" with dbSendUpdate might work.

Ideally something like this would be great:

"COPY INTO table FROM data.frame"

Answer 4

We just published version 0.9.4 of MonetDB.R on CRAN. The main change in this release are major improvements to the dbWriteTable method. By default, INSERTs are now chunked into 1000 rows per statement. Also, if the database is running on the same machine as R, you may use the csvdump=T parameter. This writes the data.frame to a local temporary CSV file, and uses an automatically generated COPY INTO statement to import. Both these methods obviously are designed to improve the speed with which dbWriteTable imports data. Also, he append/overwrite parameter handling has been fixed.