CSV-Parser 似乎无法正确解析换行符数据

Question

Yes, I know I should have better data, and if nothing works I will go about fixing my data, but I was wondering if there was any way I could get csv-parser parser to parse

"United States of
America",140640,17987,2398,286,Local transmission,0

Into

{
Country: United States of America
... blah blah
... blah blah
... blah blah
... blah blah
}

fs.createReadStream("./csv/03312020.csv")
    .pipe(
        csv([
            "Country",
            "Total",
            "TotalNew"
        ])
    )
    .on("data", row => {
        console.log(row.Country);
        let result = contains(row.Country);
        if (result !== undefined) {
            row.Date = today;
            row.id = result + "-" + today;
            if (db.dates.get(row.id) === undefined) db.dates.create(row);
        }
    })
    .on("end", () => {
        console.log("CSV file successfully processed for", today);
    });

I thought the csv-parser would see that there is a quotation mark and wrap that as one "column" but apparently it doesn't. Is there a better way to parse this data other than reparsing my CSV file itself?

Answer 1

What you can do is split that file into lines, then join the lines that have an odd number of " characters.

My script also handles the case where the \\n character appears multiple times in a single row of data.
This is based on the fact that only the first and the last line of a multi-line row will have an odd number of " characters.

You can reformat your file using my script and then feed it into your csv parser.

 const example1 = `"United States of America",140640,17987,2398,286,Local transmission,0`; console.log(reformatCsv(example1)); const example2 = `"United States of America",140640,17987,2398,286,"Local transmission",0`; console.log(reformatCsv(example2)); // @param file: string function reformatCsv(file) { const lines = file.split('\\n'); let reformattedRows = []; const parts = []; for (const line of lines) { const quoteMatches = line.match(/"/g); const isEvenNumberOfQuotes = !quoteMatches || quoteMatches.length % 2 == 0; const noPartialRowsYet = !parts.length; if (noPartialRowsYet) { if (isEvenNumberOfQuotes) // normal row { reformattedRows.push(line); } else // this is a partial row { parts.push(line); } } else // continuation of a partial row { parts.push(line); if (!isEvenNumberOfQuotes) // we got all of the parts { // join the parts // I replace \\n with a space character, but you don't have to reformattedRows.push(parts.join(' ')); } } } return reformattedRows.join('\\n'); }