在MonetDBLite中使用“ INSERT INTO test VALUES”加载大表

Question

In my use case I'm generating data at runtime, and the cost of serializing such data into a CSV file and then loading to MonetDBLite using COPY will be potentially greater than using something like INSERT INTO tablename VALUES . Given that MonetDBLite does not use DBConnections as in ODBC/iODBC, does that mean that I do not need to clear the memory using explicit COMMIT statements, or do I still need to run COMMIT after loading some GB of data into a table in order not to occupy all the available primary memory? Thanks in advance.

Answer 1

Which MonetDBLite are you using? You can use from-memory appends for very fast data loading in this case I think. For R/Python, we can directly append from data frames, for C the process is a little less convenient.