如何将大文本文件拆分为较小的文件？

Question

I have a big file with some text, and I want to split it into smaller files.

In this example, What I do:

1. I open a text file let's say with 10 000 lines into it

2. I set a number of package=300 here, which means, that's the small file limit, once a small file has 300 lines into it, close it, open a new file for writing for example (package2).

3. Same, as step 2.

4. You already know

Here is the code from my function that should do that. The ideea (what I dont' know) is how to close, and open a new file once it has reached the 300 limit (in our case here).

Let me show you what I'm talking about:

        int nr = 1;
        package=textBox1.Text;//how many lines/file (small file)
        string packnr = nr.ToString();
        string filer=package+"Pack-"+packnr+"+_"+date2+".txt";//name of small file/s
        int packtester = 0;
        int package= 300;
        StreamReader freader = new StreamReader("bigfile.txt");
        StreamWriter pak = new StreamWriter(filer);
        while ((line = freader.ReadLine()) != null)
        {
            if (packtester < package)
            {
                pak.WriteLine(line);//writing line to small file
                packtester++;//increasing the lines of small file
            }
            else if (packtester == package)//in this example, checking if the lines 
                                           //written, got to 300 
            {
                packtester = 0;
                pak.Close();//closing the file
                nr++;//nr++ -> just for file name to be Pack-2;
                packnr = nr.ToString();   
                StreamWriter pak = new StreamWriter(package + "Pack-" + packnr + "+_" + date2 + ".txt");
            }
        }

I get this errors:

Cannot use local variable 'pak' before it is declared

A local variable named 'pak' cannot be declared in this scope because it would give a different meaning to 'pak', which is already used in a 'parent or current' scope to denote something else

Answer 1

Try this:

public void SplitFile()
{
    int nr = 1;
    int package = 300;
    DateTime date2 = DateTime.Now;
    int packtester = 0;
    using (var freader = new StreamReader("bigfile.txt"))
    {
        StreamWriter pak = null;
        try
        {
            pak = new StreamWriter(GetPackFilename(package, nr, date2), false);
            string line;

            while ((line = freader.ReadLine()) != null)
            {
                if (packtester < package)
                {
                    pak.WriteLine(line); //writing line to small file
                    packtester++; //increasing the lines of small file
                }
                else
                {
                    pak.Flush();
                    pak.Close(); //closing the file
                    packtester = 0;
                    nr++; //nr++ -> just for file name to be Pack-2;
                    pak = new StreamWriter(GetPackFilename(package, nr, date2), false);
                }
            }
        }
        finally
        {
            if(pak != null)
            {
                pak.Dispose();
            }
        }
    }
}

private string GetPackFilename(int package, int nr, DateTime date2)
{
    return string.Format("{0}Pack-{1}+_{2}.txt", package, nr, date2);
}

Answer 2

This code looks like it closes the stream and re-opens a new stream when you hit 300 lines. What exactly doesn't work in this code?

One thing you'll want to add is a final close (probably with a check so it doesn't try to close an already closed stream) in case you don't have an even multiple of 300 lines.

EDIT:

Due to your edit I see your problem. You don't need to redeclare pak in the last line of code, simply reinitialize it to another streamwriter. (I don't remember if that is disposable but if it is you probably should do that before making a new one).

StreamWriter pak = new StreamWriter(package + "Pack-" + packnr + "+_" + date2 + ".txt");

becomes

pak = new StreamWriter(package + "Pack-" + packnr + "+_" + date2 + ".txt");

Answer 3

Logrotate can do this automatically for you. Years have been put into it and it's what people trust to handle their sometimes very large webserver logs.

Answer 4

Note that the code, as written, will not compile because you define the variable pak more than once. It should otherwise function, though it has some room for improvement.

When working with files, my suggestion and the general norm is to wrap your code in a using block, which is basically syntactic sugar built on top of a finally clause:

using (var stream = File.Open("C:\hi.txt"))
{
    //write your code here. When this block is exited, stream will be disposed.
}

Is equivalent to:

try
{
    var stream = File.Open(@"C:\hi.txt");
}
finally
{
    stream.Dispose();
}

In addition, when working with files, always prefer opening file streams using very specific permissions and modes as opposed to using the more sparse constructors that assume some default options. For example:

var stream = new StreamWriter(File.Open(@"c:\hi.txt", FileMode.CreateNew, FileAccess.ReadWrite, FileShare.Read));

This will guarantee, for example, that files should not be overwritten -- instead, we assume that the file we want to open doesn't exist yet.

Oh, and instead of using the check you perform, I suggest using the EndOfStream property of the StreamReader object.