无法通过 nodejs 中的 fs.readFile() 读取整个文件

Question

I am trying to read and process following file

• Name: 'TechCrunchcontinentalUSA.csv'.
• Size: 92 kb.
• The said file has total 1461 lines of data in it.
• The file is in the current working folder, downloaded from following link.
• Link: https://support.spatialkey.com/spatialkey-sample-csv-data/

My code is as under:

const fs = require('fs')

function test(fileName = 'TechCrunchcontinentalUSA.csv'){
    return new Promise((resolve, reject)=>{
        fs.readFile(fileName, 'utf-8', (err, data)=>{
            if (err) reject(err.message)
            resolve(data)
        })
    })
}

async function temp(){
    let data = await test()
    console.log(data)
}

temp()

The out put is just two lines (and that too with junk data) as under:

international-liars-poker-association,International Liars Poker Association,24,other,St. Louis,MO,1-Nov-07,1250000,USD,s
grid-networks,Grid Networks,,web,Seattle,WA,20-May-08,10500000,USD,bd00,USD,a,bD,b0,USD,seedaD,a

I can not understand what is wrong. Is the above code the right way to read a file like this? Or is there other better way to read such file fully? Please guide me.

Answer 1

Nothing is wrong. You're just reading a file as a buffer and expecting it to act as an array of lines.

If you run hexdump on the file you'll see the line breaks are all carriage returns ( '\r' or 0x0d). There are no linefeeds. When you dump the file, each carriage return causes the output to restart at the first column, without creating a new line.

0000000 6570 6d72 6c61 6e69 2c6b 6f63 706d 6e61
0000010 2c79 756e 456d 706d 2c73 6163 6574 6f67
0000020 7972 632c 7469 2c79 7473 7461 2c65 7566
0000030 646e 6465 6144 6574 722c 6961 6573 4164
0000040 746d 722c 6961 6573 4364 7275 6572 636e
0000050 2c79 6f72 6e75 0d64 696c 6566 6f6c 6b63
                       ^^ carriage return
0000060 4c2c 6669 4c65 636f 2c6b 772c 6265 542c

When I cat the file on Linux, it all prints on one line. You're probably seeing two lines because one of the lines was long enough to trigger an auto-newline in your terminal. (Additional evidence: The "International Liars Poker Association" line is in fact the longest line in the file.)

If you split your buffer into lines using the '\r' as the delimiter and then print all the lines separately, you should get the output you expect.

async function temp(){
    let data = await test()
    let lines = data.split('\r')
    lines.forEach(line => console.log(line))
}

Answer 2

nothing wrong with your code at first, it also didn't work in my local machine:

my first assumption: something was wrong with the file,

I created a git repository and added the CSV to it. I opened and saved the file with my editor, which changes Windows-based text to Unix,

and after that, it worked: