在C＃中读取txt文件的最快方法

Question

I'm working with a project and I'm a little bit confused. I've got from my teacher some txt files (from his site files: wt40.txt, wt50.txt, wt100.txt).

Every file structure looks similiar:

26    24    79    46    32    35    73    74    14    67    86    46    78    40    29    94    64    27    90    55
35    52    36    69    85    95    14    78    37    86    44    28    39    12    30    68    70     9    49    50
 1    10     9    10    10     4     3     2    10     3     7     3     1     3    10     4     7     7     4     7
 5     3     5     4     9     5     2     8    10     4     7     4     9     5     7     7     5    10     1     3

• Every number has 6 chars, but instead of leading zeros there are spaces
• At every line there are 20 numbers

File wt40.txt should be read as: first two lines to first List, next two lines to next List and third pair of lines to the third list. Next lines again should be put in pairs to those Lists.

In C++ I'm doing it in this simple way:

for(int ins=0; ins<125; ins++) //125 instances in file
{
    for(int i=0; i<N; i++)  file>>tasks[i].p; //N elements at two first lines
    for(int i=0; i<N; i++)  file>>tasks[i].w;
    for(int i=0; i<N; i++)  file>>tasks[i].d;
    tasks[i].putToLists();
}

But when I'm writing this in C# I have to open StreamReader, read every line, split it by regexp, cast them to int and add to lists. That's a lot of loops. I cannot read every 6 chars and add them in three loops because those text files have messed up end of lines chars - sometimes it's just '\\n' sometimes something more.

Isn't there any more simple way?

Answer 1

There is essentially a 20 by n table of 6 digit(character) numbers with leading spaces.

26    24    79    46    32    35    73    74    14    67    86    46    78    40    29    94    64    27    90    55
35    52    36    69    85    95    14    78    37    86    44    28    39    12    30    68    70     9    49    50
 1    10     9    10    10     4     3     2    10     3     7     3     1     3    10     4     7     7     4     7
 5     3     5     4     9     5     2     8    10     4     7     4     9     5     7     7     5    10     1     3

I don't understand the last sentence:

File wt40.txt should be read as: first two lines to first List, next two lines to next List and third pair of lines to the third list. Next lines again should be put in pairs to those Lists.

Say you want to get the first 6 rows and create 3 lists each with 2 rows, you do could something like:

It is eager in that it reads everything into memory and then does its work.

const int maxNumberDigitLength = 6;
const int rowLengthInChars = maxNumberDigitLength * 20;
const int totalNumberOfCharsToRead = rowLengthInChars * maxNumberDigitLength;

char[] buffer = new char[totalNumberOfCharsToRead];
using (StreamReader reader = new StreamReader("wt40.txt")
{
    int numberOfCharsRead = reader.Read(buffer, 0, totalNumberOfCharsToRead);
}

// put them in your lists
IEnumerable<char> l1 = buffer.Take(rowLengthInChars);
IEnumerable<char> l2 = buffer.Skip(rowLengthInChars).Take(rowLengthInChars);
IEnumerable<char> l3 = buffer.Skip(rowLengthInChars*2).Take(rowLengthInChars);

// Get the list of strings from the list of chars using non LINQ method.
List<string> list1 = new List<string>();
int i = 0;
StringBuilder sb = new StringBuilder();
foreach(char c in l1)
{
    if(i < maxNumberDigitLength)
    {
        sb.Append(c);
        i++;
    }
    i = 0;
    list1.Add(sb.ToString());
}

// LINQ method
string s = string.Concat(l1);
List<string> list1 = Enumerable
                   .Range(0, s.Length / maxNumberDigitLength)
                   .Select(i => s.Substring(i * maxNumberDigitLength, maxNumberDigitLength))
                   .ToList();     

// Parse to ints using LINQ projection
List<int> numbers1 = list1.Select(int.Parse);
List<int> numbers2 = list2.Select(int.Parse);
List<int> numbers3 = list3.Select(int.Parse);

Answer 2

Isn't there any more simple way?

Don't know if it's simpler but there is only one loop and a bit of LINQ :

List<List<int>> lists = new List<List<int>>();
using (StreamReader reader = new StreamReader("wt40.txt"))
{
    string line;
    int count = 0;
    while ((line = reader.ReadLine()) != null)
    {
        List<int> currentList =
            Regex.Split(line, "\\s")
            .Where(s => !string.IsNullOrWhiteSpace(s))
            .Select(int.Parse).ToList();
        if (currentList.Count > 0) // skip empty lines
        {
            if (count % 2 == 0) // append each second list to the previous one
            {
                lists.Add(currentList);
            }
            else
            {
                lists[count / 2].AddRange(currentList);
            }
        }
        count++;
    }
}

In total you end up with 375 lists each containing 40 numbers (at least for wt40.txt input).