通过命令行或Python将CSV导入SQLite数据库

Question

应伯尼的要求，我试图将其简化为一个更简单的示例：

我有一个CSV文件，其中包含一个月，其中星期几是列标题：

Monday,Tuesday,Wednesday,Thursday,Friday,Saturday,Sunday
1,2,3,4,5,6,7
8,9,10,11,12,13,14
15,16,17,18,19,20,21
22,23,24,25,26,27,28

在命令行中，我创建了一个SQLite表，天：

sqlite> CREATE TABLE days(
   ...> Monday int,
   ...> Tuesday int,
   ...> Wednesday int,
   ...> Thursday int,
   ...> Friday int,
   ...> Saturday int,
   ...> Sunday int
   ...> );

当尝试从csv导入数据时，这是我得到的：

sqlite> .import example.csv days
Error: example.csv line 1: expected 7 columns of data but found 1

如何将这个csv文件导入数据库，以便它可以识别每一个新行？ 谢谢！

Answer 1

在执行.import命令之前，您需要包括以下行：

.separator ,

这告诉import命令查找分隔符（在这种情况下为逗号）。

您可以在此处找到有关sqlite命令行命令的更多信息： http : //www.sqlite.org/sqlite.html

Answer 2

SQLite外壳相当精细。 当前版本不提供行标题，并且其行为在标准（RFC 4180）和常见实践之间有所不同。 即将发布的3.8版本将提供行标题。

由于您使用的是Python，因此您可能会发现APSW Shell很有用（披露：我是作者）。 您可以像SQLite shell一样从命令行使用它，也可以以编程方式使用它，包括添加自己的命令。

值得注意的是，它具有一个自动导入命令，该命令可以弄清楚所有内容，包括标题，分隔符，数据类型等。

sqlite> .help autoimport

.autoimport FILENAME ?TABLE?  Imports filename creating a table and
                              automatically working out separators and data
                              types (alternative to .import command)

The import command requires that you precisely pre-setup the table and schema,
and set the data separators (eg commas or tabs).  In many cases this information
can be automatically deduced from the file contents which is what this command
does.  There must be at least two columns and two rows.

If the table is not specified then the basename of the file will be used.

Additionally the type of the contents of each column is also deduced - for
example if it is a number or date.  Empty values are turned into nulls.  Dates
are normalized into YYYY-MM-DD format and DateTime are normalized into ISO8601
format to allow easy sorting and searching.  4 digit years must be used to
detect dates.  US (swapped day and month) versus rest of the world is also
detected providing there is at least one value that resolves the ambiguity.

Care is taken to ensure that columns looking like numbers are only treated as
numbers if they do not have unnecessary leading zeroes or plus signs.  This is
to avoid treating phone numbers and similar number like strings as integers.

This command can take quite some time on large files as they are effectively
imported twice.  The first time is to determine the format and the types for
each column while the second pass actually imports the data.

Answer 3

看看termsql，它是为实现此目的而设计的工具。 您的任务将非常容易。

手册： http : //tobimensch.github.io/termsql/

检查页面底部的示例。 那里有一个CSV导入工具，还可以检查不同的选项。

项目： https ： //github.com/tobimensch/termsql