[英]RAM consumption inside a C++ program using Sqlite3 blob
I'm using sqlite3 dbms inside a C++ program that I use mainly to store files as blob objects (I know that's not the best option). 我在C ++程序中使用sqlite3 dbms,主要用于将文件存储为blob对象(我知道这不是最佳选择)。
Obviously I write them incrementally since they can be sometimes large (40-80MB) in order to do so I have to create first a placeholder of the blob using the binding function sqlite3_bind_zeroblob(...)
and after that I open the blob writing and reading incrementally from and to it. 显然我会逐步写它们,因为它们有时可能很大(40-80MB),所以我必须首先使用绑定函数
sqlite3_bind_zeroblob(...)
创建blob的占位符,之后我打开blob写入和从中逐渐读取。
The problem I'm facing is that when i create the blob placeholder (during sqlite3_step
) the RAM consumption of my app reaches 80-160MB for 2-3 seconds, once it has been created the RAM consumption goes back to 2-3MB at most. 我面临的问题是,当我创建blob占位符时(在
sqlite3_step
期间)我的应用程序的RAM消耗达到80-160MB,持续2-3秒,一旦创建,RAM消耗最多回到2-3MB 。
I do not get why! 我不明白为什么! If they created a way to write to a blob incrementally there is for sure a way to create that stupid placeholder without wasting 160MB of RAM, but I didn't find it.
如果他们创建了一种逐步写入blob的方法,那么肯定有一种方法可以创建那个愚蠢的占位符而不会浪费160MB的RAM,但我没有找到它。 Have you got any suggestion?
你有什么建议吗?
sqlite3_stmt* stm = NULL;
sqlite3_blob *BLOB = NULL;
rc = sqlite3_prepare_v2(db, sql.c_str(), -1, &stm, NULL);
rc = sqlite3_bind_blob(stm, 1, wpath.c_str(), wpath.size()*sizeof(wchar_t), SQLITE_STATIC);
rc = sqlite3_bind_text(stm, 2, hash.c_str(), hash.size(), SQLITE_STATIC);
rc = sqlite3_bind_zeroblob(stm, 3, size);
rc = sqlite3_bind_int(stm, 4, versione);
rc = sqlite3_bind_blob(stm, 5, last.c_str(), last.size()*sizeof(wchar_t), SQLITE_STATIC);
rc = sqlite3_step(stm);
if (rc != SQLITE_DONE) {
fprintf(stderr, " This file was already present in the database!\n", rc);
return;
}
else {
fprintf(stdout, "Record FILE created successfully\n");
}
It is an issue reported HERE . 这是一个在这里报道的问题。
And the oficial answer is: 而官方的答案是:
In order for zeroblobs to work as above (using a fixed amount of memory no matter how big they are) all zeroblobs must be at the end of the row.
为了使zeroblobs如上所述工作(使用固定数量的内存,无论它们有多大),所有zeroblobs必须位于行的末尾。 In other words, the columns of the table that are receiving the zeroblobs must be the last columns in the table.
换句话说,接收zeroblobs的表的列必须是表中的最后一列。 If any non-zero content follows the zeroblob, then the zeroblob is expanded into a literal sequence of zero bytes, meaning memory must be allocated for the entire zeroblob.
如果zeroblob后面有任何非零内容,那么zeroblob将扩展为零字节的文字序列,这意味着必须为整个zeroblob分配内存。
So you need to change the order to fix it: 所以你需要更改顺序来修复它:
sqlite3_stmt* stm = NULL;
sqlite3_blob *BLOB = NULL;
rc = sqlite3_prepare_v2(db, sql.c_str(), -1, &stm, NULL);
rc = sqlite3_bind_blob(stm, 1, wpath.c_str(), wpath.size()*sizeof(wchar_t), SQLITE_STATIC);
rc = sqlite3_bind_text(stm, 2, hash.c_str(), hash.size(), SQLITE_STATIC);
rc = sqlite3_bind_int(stm, 3, versione);
rc = sqlite3_bind_blob(stm, 4, last.c_str(), last.size()*sizeof(wchar_t), SQLITE_STATIC);
rc = sqlite3_bind_zeroblob(stm, 5, size);
rc = sqlite3_step(stm);
if (rc != SQLITE_DONE) {
fprintf(stderr, " This file was already present in the database!\n", rc);
return;
}
else {
fprintf(stdout, "Record FILE created successfully\n");
}
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.