I have the following OleDb method I really like because I can eventually pass SQL queries against a csv.
The trouble is that OleDb can only read < 256 columns and I have more than this.
What is an alternative to efficiently querying a csv table in C# with such a column size?
I'd like to avoid bringing the entire csv into a datatable to later LINQ.
DataTable GetDataTableFromCsv(string path, bool isFirstRowHeader)
{
string header = isFirstRowHeader ? "Yes" : "No";
string pathOnly = Path.GetDirectoryName(path);
string fileName = Path.GetFileName(path);
string sql = @"SELECT * FROM [" + fileName + "]";
using (OleDbConnection connection = new OleDbConnection(
@"Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + pathOnly +
";Extended Properties=\"Text;HDR=" + header + "\""))
using (OleDbCommand command = new OleDbCommand(sql, connection))
using (OleDbDataAdapter adapter = new OleDbDataAdapter(command))
{
DataTable dataTable = new DataTable
{
Locale = CultureInfo.CurrentCulture
};
adapter.Fill(dataTable);
return dataTable;
}
}
Update: Tried a comment in the suggestion of the following script to use the ACE driver but the following OleDb connection string still only gets 255 columns. Still looking for a way to accomplish this.
using (OleDbConnection connection = new OleDbConnection(
"Provider = Microsoft.ACE.OLEDB.12.0; Data Source = " + pathOnly +"; Extended Properties =\"Text; HDR = Yes; FORMAT = Delimited\""))
How about something like this?:
static DataTable GetDataTableFromCsv(string path, bool isFirstRowHeader)
{
DataTable dataTable = null;
using (var inStream = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
using (var sr = new StreamReader(inStream))
{
string line;
while ((line = sr.ReadLine()) != null)
{
var fields = line.Split(',');
if (dataTable == null) // create datatable based on header row
{
dataTable = new DataTable();
foreach (var field in fields)
dataTable.Columns.Add(new DataColumn(field));
continue; // don't add header row to rows collection
}
dataTable.Rows.Add(fields);
}
}
return dataTable;
}
If your first row is not a header, you'd have to come up with a different approach to naming the columns.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.