How to read a line and write to that line, of a text file?

Question

I'm use MS-Visual Studio 2015, develop a Winforms application in C#.

What I'm trying to reach is a reader&writer which opens a CSV file with UTF-8 coding , and reads line for line. My program actually reads a line, split it at the semicolons (;) and send that informations to my database. Now it should mark that line as already read, by appending a text or a special sign eg ("read" or "done" or "§$%").

Because it's possible that someone or something (ERP-Systems), appends new data to that CSV file. So, the next time my program iterates through that file, it shall only read the line without my special mark.

my program:

foreach (string sFile in Directory.GetFiles(sImportPfad, "*.*", SearchOption.TopDirectoryOnly))
{
    var oStream = new FileStream(sFile, FileMode.Append, FileAccess.Write, FileShare.Read);
    var iStream = new FileStream(sFile, FileMode.Open, FileAccess.Read, FileShare.ReadWrite);

    var sw = new System.IO.StreamWriter(oStream);
    var sr = new System.IO.StreamReader(iStream);
    int c = 0;

    //  alle Zeilen jedes Files laden
    while (!sr.EndOfStream)
    {
        String line = sr.ReadLine();
        String[] splitLine = line.Trim().Split(txtDivider.Text.Trim().ToCharArray());

        if (line.Contains("§$%"))
            break;

        DatenbankEintragAuftragsPool dbEintrag = new DatenbankEintragAuftragsPool();

             foreach (myImportFilterObject ob in arrImportFilterObjects)
             {
             .
             .
             .
             }

        String result = Program.myPaletti.AuftragInDieDatenbankSchreiben(dbEintrag);

        if (result.Equals("ok"))
        {
            sw.WriteLine(line + "   §$%"); sw.Flush();
        }
    }
}

My problem is the writer is appending the line+"special mark" to the end of my file.

Additionally I didn't figure out how to read the file with UTF-8 coding.

I appreciate your answers !!

EDIT: ##############

This code would do the trick...

string[] lines = System.IO.File.ReadAllLines("test");
lines[0] = lines[0] + " $%&";   /* replace with whatever you need */
System.IO.File.WriteAllLines("test", lines);

But for my usage it's not recommended to read all lines, 'cause it's possible that the guys never delete any data for the next 20 years.

I'll go further to find a solution line by line...

Answer 1

There are some problems in your code that I will try to solve here

using(var stream = new FileStream(sFile, FileMode.Open, FileAccess.Read)
using(var reader = new StreamReader(stream, Encoding.UTF8))
{
    long position = GetFirstNewRecordOfFile(sFile);
    stream.Seek(position, SeekOrigin.Begin);
    while(!reader.EndOfStream)
    {
        var line = reader.ReadLine();
        // Process line 
    }
    SaveFirstNewRecordOfFile(sFile, stream.Position);
}

Now you just need to figure out where and how to save the position of the file.

If you have a writer that appends data to the file the file might grow to a huge size over time, maybe it is better to truncate or delete the file when it has been read. I recommend deleting the file since you will not have to loop through a lot of empty files, that will however require that you rename/move the file before processing it to avoid that the writer process appends data to it after you close it but before you delete it. If you just move the file to a sub folder you can use that as a backup.

Answer 2

My solution now is to create a new file, write into this file, delete the original file and rename the new file.

        foreach (string sFile in Directory.GetFiles(sImportPfad, "*.*", SearchOption.TopDirectoryOnly))
        {
            FileStream iStream;
            try
            {
                using (iStream = new FileStream(sFile, FileMode.Open, FileAccess.Read, FileShare.None))
                {
                    var sr = new System.IO.StreamReader(iStream, Encoding.UTF8);

                    if (rbCSVfilesMarkieren.Checked)
                    {
                        using (var oStream = new FileStream(sFile + "_new", FileMode.Create, FileAccess.Write, FileShare.None))
                        {
                            var sw = new System.IO.StreamWriter(oStream, Encoding.UTF8);
                            int c = 0;

                            while (!sr.EndOfStream)
                            {
                                String line = sr.ReadLine();
                                String[] splitLine = line.Trim().Split(txtDivider.Text.Trim().ToCharArray());

                                if (line.Contains("$$$"))
                                {
                                    sw.WriteLine(line);
                                    sw.Flush();
                                    continue;
                                }

                                String result = Program.myPaletti.Irgendwasneues(splitLine, arrImportFilterObjects);

                                if (result.Equals("ok"))
                                {
                                    sw.WriteLine(line + "$$$");
                                    sw.Flush();
                                    anzNeueDatensätze++;
                                }
                            }
                        }
                    }

                    System.IO.File.Delete(sFile);
                    System.IO.File.Move(sFile + "_new", sFile);

                }
            }
        }

I also included the UTF-8 coding. Furthermore I've found a way to block the file I'm reading/writing, by using FileShare.None.

Thank you guys for your help !! I appreciate it !