how to read big text files in C#?

Question

I have a win Form and I want to read a text file and show it in list Box but there is problem when i try to load file with 1-3mb size after a few seconds its throw this exception:

Managed Debugging Assistant 'ContextSwitchDeadlock' : 'The CLR has been unable to transition from COM context 0x149cf10 to COM context 0x149ce58 for 60 seconds. The thread that owns the destination context/apartment is most likely either doing a non pumping wait or processing a very long running operation without pumping Windows messages. This situation generally has a negative performance impact and may even lead to the application becoming non responsive or memory usage accumulating continually over time. To avoid this problem, all single threaded apartment (STA) threads should use pumping wait primitives (such as CoWaitForMultipleHandles) and routinely pump messages during long running operations.'

and here is my code:

private void Button7_Click(object sender, EventArgs e)
    {
        openFileDialog = new OpenFileDialog();
        if (openFileDialog.ShowDialog() == DialogResult.OK)
        {
            StreamReader sr = new StreamReader(openFileDialog.FileName);
            var emails = sr.ReadToEnd();
            foreach (var items in emails)
            {
                aFlistBoxEmail.Items.Add(items);
            }
        }
    }

also i use this solution but didnt help: enter link description here

Answer 1

Your click handler executes on the UI thread, which also happens to run the message pump for your application. Since your handler takes so long to run, it makes your application nonresponsive to UI and system events for a period that is probably unacceptable.

You can remedy the problem by making your logic async.

private async Task Button7_Click(object sender, EventArgs e)
{
    openFileDialog = new OpenFileDialog();
    if (openFileDialog.ShowDialog() == DialogResult.OK)
    {
        StreamReader sr = new StreamReader(openFileDialog.FileName);
        var emails = await sr.ReadToEndAsync();
        foreach (var items in emails)
        {
            aFlistBoxEmail.Items.Add(items);
        }
    }
}

Answer 2

When there are long running process, then automatically the application will go into non responsive mode , because the long running process is inside the UI thread, Its better to use the asyn programming using await & async , this will help to run the long running activities into a seperate thread and it wont break the UI thread. My sample code is given below.

private async void Button7_Click(object sender, EventArgs e)
{
    openFileDialog = new OpenFileDialog();
    if (openFileDialog.ShowDialog() == DialogResult.OK)
    {
        StreamReader sr = new StreamReader(openFileDialog.FileName);
        var emails =  await sr.ReadToEndAsync();
        foreach (var items in emails)
        {
            aFlistBoxEmail.Items.Add(items);
        }
    }
}

The the file reading part can be moved to another function so that it can give more modulatiry.

Answer 3

Why are you doing this exactly? your code adds every single character in the text file as a separate entry in the list box, with a 3 MB file it will be a lot of entries, don't you need to parse the text to a specific value or a class first? if you're trying to read emails from a file split the file like this for example string.Split('\\n'); in a way you separate each email and then parse them using the MailAddress Class from System.Net.Mail.MailAddress

Answer 4

You can try reading all lines at once :

private async void Button7_Click(object sender, EventArgs e)
{
    openFileDialog = new OpenFileDialog();
    if (openFileDialog.ShowDialog() == DialogResult.OK)
        aFlistBoxEmail.DataSource = File.ReadAllLines(openFileDialog.FileName);
}

Answer 5

it doesn't necessarily mean you have an issue but it can cause issues since you're working on the main thread of the UI - so your app will be stuck until the reader finishes it's work and since it's a heavy file you're getting this error. you can switch this off in:

Debug > windows > Exceptions settings > Managed Debugging Assistants (expand) > disable "ContextSwitchDeadlock"

it's a good idea to put this task in a thread or a task and make sure you have try catch INSIDE the method as well in case it fails (or a using statement) and meanwhile (until the file reader finishes) you can display on the UI a message saying "app is busy" or something similar.

Thread thread = new Thread(Butoon_Click(sender, ev));
thread.IsBackground = true;