在Win32中解析巨大的Char *缓冲区

Question

I have some huge csv files and try read files, parsing and write in one DB. csv format is same below:

EUR/USD 20100103 21:27:59.694   1.43067 1.43097
EUR/USD 20100103 21:27:59.732   1.43075 1.43095
EUR/USD 20100103 21:28:08.152   1.43078 1.43099
EUR/USD 20100103 21:28:16.897   1.43076 1.43102
EUR/USD 20100103 21:28:28.757   1.43071 1.43101
EUR/USD 20100103 21:29:07.659   1.43071 1.43106

And my C++ code is:

void Read(char* fileName){
  FILE * pFile;
        long lSize;
        char * buffer;
        size_t result;

        pFile = fopen (fileName , "rb" );
        if (pFile==NULL) {fputs ("File error",stderr); exit (1);}

        fseek (pFile , 0 , SEEK_END);
        lSize = ftell (pFile);
        cout<<lSize<<"\n";
        rewind (pFile);

        // allocate memory to contain the whole file:
        buffer = (char*) malloc (sizeof(char)*lSize);
        if (buffer == NULL) {fputs ("Memory error",stderr); exit (2);}

        result = fread (buffer,1,lSize,pFile);
        if (result != lSize) {fputs ("Reading error",stderr); exit (3);}

                char * pch;
        pch = strtok (buffer,"\n");

        while (pch != NULL) {

            Parsing(pch);
                pch = strtok (NULL, "\n");
        }

        free (buffer);
        fclose (pFile);
}

void Parsing(char* tmpp){
    char * pch;
    pch = strtok (tmpp,",");

    char temp[256];
        strcpy(temp, "INSERT INTO dukas  VALUES('" );
    while (pch != NULL) {
        cout<<pch<<"\n";
        pch = strtok (NULL, ",");

    }


}

Problem is that this file, only read first line and cant move to second line. Any bugs in this script? Regards,

Answer 1

From the MSDN on strtok: within a single thread, interleaving calls to one of these functions is highly likely to produce data corruption and inaccurate results. When parsing different strings, finish parsing one string before starting to parse the next .

Which is exactly what you're doing: you have one strtok reading lines, another parsing fields inside the Parsing function.