简体繁体 English

内存中 sqlite 与 python 一起生产

[英]in-memory sqlite in production with python

原文 2020-05-14 07:14:53 0 2 python/ sqlite

I am creating a python system that needs to handle many files.我正在创建一个需要处理许多文件的 python 系统。 Each of the file has more than 10 thousand lines of text data.每个文件都有超过一万行的文本数据。

Because DB (like mysql) can not be used in that environment, when file is uploaded by a user, I think I will save all the data of the uploaded file in in-memory-SQLite so that I can use SQL to fetch specific data from there.因为DB（如mysql）不能在那种环境下使用，所以当用户上传文件时，我想我会将上传文件的所有数据保存在内存中-SQLite中，这样我就可以使用SQL来获取特定数据从那里。

Then, when all operations by program are finished, save the processed data in a file.然后，当程序的所有操作完成后，将处理后的数据保存在一个文件中。 This is the file users will receive from the system.这是用户将从系统收到的文件。

But some websites say SQLite shouldn't be used in production.但是有些网站说 SQLite 不应该在生产中使用。 But in my case, I just save them temporarily in memory to use SQL for the data.但就我而言，我只是将它们暂时保存在 memory 中，以使用 SQL 获取数据。 Is there any problem for using SQLite in production even in this scenario?即使在这种情况下，在生产中使用 SQLite 是否有任何问题？

Edit: The data in in-memory-DB doesn't need to be shared between processes.编辑：内存数据库中的数据不需要在进程之间共享。 It just creates tables, process data, then discard all data and tables after saving the processed data in file.它只是创建表格，处理数据，然后在将处理后的数据保存在文件中后丢弃所有数据和表格。 I just think saving everything in list makes search difficult and slow.我只是认为将所有内容保存在列表中会使搜索变得困难且缓慢。 So using SQLite is still a problem?那么使用SQLite还有问题吗？

2 个解决方案

SQLite shouldn't be used in production is not a one-for-all rule, it's more of a rule of thumb. SQLite shouldn't be used in production这不是一个万能的规则，它更像是一个经验法则。 Of course there are appliances where one could think of reasonable use of SQLite even in production environments.当然，即使在生产环境中，也有一些设备可以考虑合理使用 SQLite。

However your case doesn't seem to be one of them.但是，您的情况似乎不是其中之一。 While SQLite supports multi-threaded and multi-process environments, it will lock all tables when it opens a write transaction.虽然 SQLite 支持多线程和多进程环境，但它会在打开写入事务时锁定所有表。 You need to ask yourself whether this is a problem for your particular case, but if you're uncertain go for "yes, it's a problem for me".您需要问自己这是否是您的特定情况的问题，但如果您不确定 go 是否为“是的，这对我来说是个问题”。

You'd be probably okay with in-memory structures alone, unless there are some details you haven't uncovered.除非有一些你没有发现的细节，否则你可能只使用内存结构就可以了。

I'm not familiar with the specific context of your system, but if what you're looking for is a SQL database that is我不熟悉您系统的具体上下文，但如果您要查找的是 SQL 数据库

light光
Access is from a single process and a single thread .访问来自单个进程和单个线程。
If the system crashes in the middle, you have a good way to recover from it (either backing up the last stable version of the database or just create it from scratch).如果系统在中间崩溃，您有一个很好的方法来从中恢复（备份数据库的最后一个稳定版本或从头开始创建它）。

If you meet all these criteria, using SQLite is production is fine.如果您满足所有这些标准，使用 SQLite 就可以生产。 OSX, for example, uses sqlite for a few purposes (eg . /var/db/auth.db ).例如，OSX 将 sqlite 用于一些目的（例如 . /var/db/auth.db ）。