简体   繁体   中英

Efficient Way to SQL Csv w/ > 255 Columns in C#?

I have the following OleDb method I really like because I can eventually pass SQL queries against a csv.

The trouble is that OleDb can only read < 256 columns and I have more than this.

What is an alternative to efficiently querying a csv table in C# with such a column size?

I'd like to avoid bringing the entire csv into a datatable to later LINQ.

            DataTable GetDataTableFromCsv(string path, bool isFirstRowHeader)
            {
                string header = isFirstRowHeader ? "Yes" : "No";

                string pathOnly = Path.GetDirectoryName(path);
                string fileName = Path.GetFileName(path);

                string sql = @"SELECT * FROM [" + fileName + "]";

                using (OleDbConnection connection = new OleDbConnection(
                          @"Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + pathOnly +
                          ";Extended Properties=\"Text;HDR=" + header + "\""))
                using (OleDbCommand command = new OleDbCommand(sql, connection))
                using (OleDbDataAdapter adapter = new OleDbDataAdapter(command))
                {
                    DataTable dataTable = new DataTable
                    {
                        Locale = CultureInfo.CurrentCulture
                    };
                    adapter.Fill(dataTable);
                    return dataTable;
                }
            }

Update: Tried a comment in the suggestion of the following script to use the ACE driver but the following OleDb connection string still only gets 255 columns. Still looking for a way to accomplish this.

using (OleDbConnection connection = new OleDbConnection(
                      "Provider = Microsoft.ACE.OLEDB.12.0; Data Source = " + pathOnly +"; Extended Properties =\"Text; HDR = Yes; FORMAT = Delimited\""))

How about something like this?:

        static DataTable GetDataTableFromCsv(string path, bool isFirstRowHeader)
        {
            DataTable dataTable = null;

            using (var inStream = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
            using (var sr = new StreamReader(inStream))
            {
                string line;
                while ((line = sr.ReadLine()) != null)
                {
                    var fields = line.Split(',');

                    if (dataTable == null) // create datatable based on header row
                    {
                        dataTable = new DataTable();
                        foreach (var field in fields)
                            dataTable.Columns.Add(new DataColumn(field));
                        continue; // don't add header row to rows collection
                    }

                    dataTable.Rows.Add(fields);
                }
            }

            return dataTable;
        }

If your first row is not a header, you'd have to come up with a different approach to naming the columns.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM