向数据表中添加行

Question

如何通过从 csv 读取记录在没有 foreach 循环的情况下将行添加到数据表中？

var records = File.ReadLines(FD.FileName, Encoding.UTF8).Skip(1);
//not working because datatable is not thread safe
Parallel.ForEach(records, record => dataTable.Rows.Add(record.Split(',')));
//using for each is taking aroung 13 sec for 45000 records, need a better solution than this.
foreach (var record in records)
 {
 dataTable.Rows.Add(record.Split(','));
 }

Answer 1

您是否尝试过使用像 LINQ 到 CSV 这样的库？

LINQ转CSV图书馆

我最近用过它，它非常好。

基本上你在你的 class 中设置结构如下

using LINQtoCSV;
using System;
class Product
{
    [CsvColumn(Name = "ProductName", FieldIndex = 1)]
    public string Name { get; set; }
    [CsvColumn(FieldIndex = 2, OutputFormat = "dd MMM HH:mm:ss")]
    public DateTime LaunchDate { get; set; }
    [CsvColumn(FieldIndex = 3, CanBeNull = false, OutputFormat = "C")]
    public decimal Price { get; set; }
    [CsvColumn(FieldIndex = 4)]
    public string Country { get; set; }
    [CsvColumn(FieldIndex = 5)]
    public string Description { get; set; }
}

之后使用下面的代码读取文件

CsvFileDescription inputFileDescription = new CsvFileDescription
{
    SeparatorChar = ',', 
    FirstLineHasColumnNames = true
};
CsvContext cc = new CsvContext();
IEnumerable<Product> products =
    cc.Read<Product>("products.csv", inputFileDescription);
// Data is now available via variable products.
var productsByName =
    from p in products
    orderby p.Name
    select new { p.Name, p.LaunchDate, p.Price, p.Description };
// or ...
foreach (Product item in products) { .... }

让我知道事情的后续

Answer 2

我敢肯定，您不需要一次显示 50000 行。 仅仅因为这不是人类可读的。

考虑分页：加载前 N 行，然后，当用户单击“下一页”按钮时，加载下 N 行，依此类推。

将这个 class 作为数据行：

public class MyDataRow
{
    public int Id { get; set; }

    public string Name { get; set; }
}

分页演示将是这样的：

public partial class Form1 : Form
{
    private const int PageSize = 10;
    private readonly BindingList<MyDataRow> dataRows;
    private readonly IEnumerator<string> csvLinesEnumerator;

    public Form1()
    {
        InitializeComponent();

        // start enumeration
        csvLinesEnumerator = ReadLines().Skip(1).GetEnumerator();
        // assign data source to DGV
        dataRows = new BindingList<MyDataRow>();
        dataGridView1.DataSource = dataRows;
        // fetch first page
        FetchNextPage();
    }

    private void button1_Click(object sender, EventArgs e) => FetchNextPage();

    private void FetchNextPage()
    {
        // disable notifications
        dataRows.RaiseListChangedEvents = false;
        try
        {
            dataRows.Clear();

            while (dataRows.Count < PageSize)
            {
                if (csvLinesEnumerator.MoveNext())
                {
                    // parse next line and make row object from line's data
                    var lineItems = csvLinesEnumerator.Current.Split(',');
                    var row = new MyDataRow
                    {
                        Id = int.Parse(lineItems[0]),
                        Name = lineItems[1]
                    };

                    // add next row to the page
                    dataRows.Add(row);
                }
                else
                {
                    break;
                }
            }
        }
        finally
        {
            // enable notifications and reload data source into DGV
            dataRows.RaiseListChangedEvents = true;
            dataRows.ResetBindings();
        }
    }

    /// <summary>
    /// This is just File.ReadLines replacement for testing purposes
    /// </summary>
    private IEnumerable<string> ReadLines()
    {
        for (var i = 0; i < 50001; i++)
        {
            yield return $"{i},{Guid.NewGuid()}";
        }
    }
}

用户界面：

另一种选择是使用DataGridView.VirtualMode ，但它主要针对某些情况进行了优化，当您知道总行数时（例如，您可以通过 SQL 查询对它们进行计数），但对于 CSV 文件则不是这样。