C＃确定平面文件中的EOL字符

Question

I am attempting to identify what the EOL character is from a given .txt or .csv flat file. Based on what the EOL character is from the first row of data in the flat file, I want to process the data from this file accordingly (I am using Bulk Load to create tables on SQL Server and need to pass in the EOL to the bulk load command). From what I understand, Readline() handles the EOL automatically, so I can't parse the Readline() string for the EOL character. The code below gives an example of what I am trying to do:

int EOLChar_CRLF = 0;
int EOLChar_LF = 0;
int EOLChar_CR = 0;
int EOLChar_Hex = 0;

string eol_line = file2.ReadLine();
MessageBox.Show(eol_line);
EOLChar_CRLF = eol_line.IndexOf("\\r\\\n");
EOLChar_LF = eol_line.IndexOf("\\n");
EOLChar_CR = eol_line.IndexOf("\\r");
EOLChar_Hex = eol_line.IndexOf("\\0x0a");

MessageBox.Show("CRLF is line feed if " + EOLChar_CRLF.ToString() + " <> -1");
MessageBox.Show("LF is line feed if " + EOLChar_LF.ToString() + " <> -1");
MessageBox.Show("CR is line feed if " + EOLChar_CR.ToString() + " <> -1");
MessageBox.Show("0x0a is line feed if " + EOLChar_Hex.ToString() + " <> -1");

Does anybody know of a way to determine the EOL using the StreamReader.ReadLine() method or any other way of accomplishing this? I only want to read in the first row of data and parse that for the EOL since some of these files are 20+ million rows.

Answer 1

The usual way to determine the end-of-line convention for a text file is to slurp in a buffer of sufficient size from the start of the file and examine it. The size of the buffer, of course, is somewhat dependent on the line length expected. You want to slurp in enough data to get a reasonable number of lines.

You are unlikely to encounter end-of-line conventions other than Windows (CR+LF), Unix/Linux/OS X (LF) or old-school MacOS (CR). For speed, it would be hard to beat something like this.

public enum EndOfLineStyle
{
  Unknown = 0     ,
  CR      = 1     ,
  LF      = 2     ,
  CRLF    = CR|LF ,
  Unix    = LF    ,
  MacOs   = CR    ,
  Windows = CRLF  ,
}

const int BUFFER_SIZE = 8192 ;
public EndOfLineStyle DetermineEndOfLineStyle( string pathToFile )
{
  int    bufl  = 0 ;
  char[] buf   = new char[BUFFER_SIZE] ;

  using ( StreamReader reader = File.OpenText( pathToFile ) )
  {
    bufl = reader.ReadBlock( buf , 0 , buf.Length ) ;
  }

  int crlfs = 0 ;
  int crs   = 0 ;
  int lfs   = 0 ;

  for ( int i = 0 ; i < bufl ; )
  {
    if      ( buf[i] == '\r' && i < bufl-1 && buf[i+1] == '\n' ) { ++crlfs ; i+=2 ; }
    else if ( buf[i] == '\r'                                   ) { ++crs   ; i+=1 ; }
    else if ( buf[i] == '\n' )                                   { ++lfs   ; i+=1 ; }
  }

  EndOfLineStyle style ;
  if      ( crlfs > crs   && crlfs > lfs ) style = EndOfLineStyle.Windows ;
  else if ( lfs   > crlfs && lfs   > crs ) style = EndOfLineStyle.Unix    ;
  else if ( crs   > crlfs && crs   > lfs ) style = EndOfLineStyle.MacOs   ;
  else                                     style = EndOfLineStyle.Unknown ;

  return style ;
}