DirectoryInfo GetFiles() with more than 8k files

Question

I'm working on a project and previously blog articles were a text file in a folder. Now we want to store the article content in our SQL Server 2008 server's database.

Right now I'm just trying to get the file name, not inserting any records. Right now it takes more than 15 minutes to finish. This is a Windows Form.

Here's what I have now:

private void LoadButton_Click(object sender, EventArgs e)
{
    var dirInfo = new DirectoryInfo(@"P:\op\articles");

    foreach (var currFile in dirInfo.GetFiles())
    {
        using (StreamReader sr = new StreamReader(currFile.FullName))
        {
            OutputTextBox.Text += currFile.FullName + "\r\n";
        }
    }
}

Is there anyway I can do something where it only grabs say 1k articles to process and then delete? Is there a more efficient way to make this work faster?

Answer 1

I think you'll want to look at Directory.EnumerateFiles . I believe that will enumerate the collection lazily, returning them more efficiently for large sets of files and allowing you to control iteration.

See: Directory.EnumerateFiles