File open memory c#

Question

While opening a file in C# using stream reader is the file going to remain in memory till it closed. For eg if a file of size 6MB is opened by a program using streamreader to append a single line at the end of the file. Will the program hold the entire 6 MB in it's memory till file is closed. OR is a file pointer returned internally by .Net code and the line is appended at the end. So the 6MB memory will not be taken up by the program

Answer 1

The whole point of a stream is so that you don't have to hold an entire object in memory. You read from it piece by piece as needed.

If you want to append to a file, you should use File.AppendText which will create a StreamWriter that appends to the end of a file.

Here is an example:

 string path = @"c:\temp\MyTest.txt";

 // This text is always added, making the file longer over time
 // if it is not deleted.
 using (StreamWriter sw = File.AppendText(path)) 
 {
     sw.WriteLine("This");
     sw.WriteLine("is Extra");
     sw.WriteLine("Text");
 } 

Again, the whole file will not be stored in memory.

Documentation: http://msdn.microsoft.com/en-us/library/system.io.file.appendtext.aspx

Answer 2

The .NET FileStream will buffer a small amount of data (you can set this amount with some of the constructors).

The Windows OS will do more significant caching of the file, if you have plenty of RAM this might be the whole file.

Answer 3

StreamReader will not read the 6 MB file into memory. Also, you can't append a line to the end of the file using StreamReader. You might want to use StreamWriter.

update: not counting buffering and OS caching as someone else mentioned

Answer 4

A StreamReader uses FileStream to open the file. FileStream stores a Windows handle, returned by the CreateFile() API function. It is 4 bytes on a 32-bit operating system. FileStream also has a byte[] buffer, it is 4096 bytes by default. This buffer avoids having to call the ReadFile() API function for every single read call. StreamReader itself has a small buffer to make decoding the text in the file more efficient, it is 128 bytes by default. And it has some private variables to keep track of the buffer index and whether or not a BOM has been detected.

This all adds up to a few kilobytes. The data you read with StreamReader will of course take space in your program's heap. That could add up to 12 megabytes if you store every string in, say, a List. You usually want to avoid that.