Node.js - FileSystem.writeFile returning badly formatted JSON

Question

So the premise here is a JavaScript script that, among other things, is supposed to maintain a little "dictionary" of sorts as a local JSON file, where some user-defined strings can be mapped to a key (a string) and serialized. The script happens to already depend on Node.js and I'm not trying to run this in a browser anyway, which is why I'm using Node.js's FileSystem library. The problem is when I try to save a new definition to this dictionary, using the function below.

(Also note that, just because of how data is juggled in the wrapper function in which this function is called, arguments are passed as an array of strings, rather than just a simple "key" and "value" string. This is intentional.)

var sDictFile = "dict.json";

async function com_setdef(args){
    if (args.length > 1) {
        let sJSONKey = args[0];
        let sDef = args.slice(1).join(" ");

        fs.readFile(sDictFile, "utf8", (err, data) => {
            if (err) throw err;

            let pDict = JSON.parse(JSON.stringify(data));

            if (pDict.hasOwnProperty(sJSONKey)) { // this entry already exists in the dictionary, so just edit it
                let pRow = pDict.sJSONKey;

                if (pRow.locked == true) {
                    return "This dictionary entry is locked. Its definition cannot be changed unless it is unlocked, which can only be done by an admin.";
                } else {
                    pRow.def = sDef;
                    fs.writeFile(sDictFile, JSON.stringify(pDict, null, 2), err => {
                        if (err) throw err ;
                    });
                }   
            } else { // this entry doesn't exist yet, so create it from scratch
                let sJSONValue = {"def":sDef,"locked":false,"viewlocked":false};
                pDict[sJSONKey] = sJSONValue;
                fs.writeFile(sDictFile, JSON.stringify(pDict, null, 2), err => {
                    if (err) throw err ;
                });
                return "completed";
            }
        });
    }   
}

And just to give it some data to load to begin with, I put this in the othwerwise empty dict.json:

{
    "A": {
        "def": "the first letter in the alphabet",
        "locked": false,
        "viewlocked": false
    }
}

When this function is first called with the arguments com_setdef( ["cedar", "a", "species", "of", "tree"] ) , the resulting dict.json that had previously been empty now looks like this:

"{\r\n\t\"A\": {\r\n\t\t\"def\": \"the first letter in the alphabet\",\r\n\t\t\"locked\": false,\r\n\t\t\"viewlocked\": false\r\n\t}\r\n}"

So not only did it not save any of the new data, I also have no idea where all this other junk was coming from. The Node.js documentation for FileSystem.readFile says that if the encoding isn't specified, that the raw buffer is returned, so I suspected that's what this is, but even when UTF-8 is specified, it's still filling the JSON with a bunch of garbage I didn't explicitly pass to it.

There's some additional bad news - it never ends up returning the string "completed". This whole function is called within an async wrapper function that awaits the string returned from this function so it can then compare the length of it and make sure it's > 0 with str.length > 0 . Instead, I get: (node:89944) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'length' of undefined . All this makes it seem like fs.readFile, an async process according to the documentation, never actually terminates. I think.

So I clearly don't know what I'm doing. Why is the script failing so spectacularly to add a new line to the JSON file and save it?

Answer 1

Your problem is that data which is returned from fs.readfile() is already a string. So when you do JSON.parse(JSON.stringify(data)) - you're actually first stringifying a string, and then parsing a stringified string will give you a string (hence all these annoying \\r\\n ).
A way to see it is to add in your code console.log(typeof pDict) which will print string .

The solution: make sure your dict.json is a well-formatted JSON, and then change to JSON.parse(data) (ie without JSON.stringify() ).