[英]Write objects into file with Node.js
I've searched all over stackoverflow / google for this, but can't seem to figure it out.我已经在整个stackoverflow/google上搜索过这个,但似乎无法弄清楚。
I'm scraping social media links of a given URL page, and the function returns an object with a list of URLs.我正在抓取给定 URL 页面的社交媒体链接,该函数返回一个带有 URL 列表的对象。
When I try to write this data into a different file, it outputs to the file as [object Object]
instead of the expected: [ ' https://twitter.com/#!/101Cookbooks ', ' http://www.facebook.com/101cookbooks '] as it does when I console.log()
the results.当我尝试将此数据写入不同的文件时,它会以[object Object]
而不是预期的形式输出到文件中:[ ' https://twitter.com/#!/101Cookbooks ', ' http://www. facebook.com/101cookbooks '] 就像我console.log()
结果一样。
This is my sad attempt to read and write a file in Node, trying to read each line(the url) and input through a function call request(line, gotHTML)
:这是我在 Node 中读取和写入文件的可悲尝试,尝试读取每一行(url)并通过函数调用request(line, gotHTML)
:
fs.readFileSync('./urls.txt').toString().split('\n').forEach(function (line){
console.log(line);
var obj = request(line, gotHTML);
console.log(obj);
fs.writeFileSync('./data.json', obj , 'utf-8');
});
for reference -- the gotHTML
function:供参考—— gotHTML
函数:
function gotHTML(err, resp, html){
var social_ids = [];
if(err){
return console.log(err);
} else if (resp.statusCode === 200){
var parsedHTML = $.load(html);
parsedHTML('a').map(function(i, link){
var href = $(link).attr('href');
for(var i=0; i<socialurls.length; i++){
if(socialurls[i].test(href) && social_ids.indexOf(href) < 0 ) {
social_ids.push(href);
};
};
})
};
return social_ids;
};
Building on what deb2fast said I would also pass in a couple of extra parameters to JSON.stringify() to get it to pretty format:在 deb2fast 所说的基础上,我还将向 JSON.stringify() 传递几个额外的参数以使其具有漂亮的格式:
fs.writeFileSync('./data.json', JSON.stringify(obj, null, 2) , 'utf-8');
The second param is an optional replacer function which you don't need in this case so null
works.第二个参数是一个可选的替换函数,在这种情况下你不需要它,所以null
有效。
The third param is the number of spaces to use for indentation.第三个参数是用于缩进的空格数。 2 and 4 seem to be popular choices. 2 和 4 似乎是流行的选择。
obj
is an array in your example. obj
在您的示例中是一个数组。
fs.writeFileSync(filename, data, [options]) requires either String
or Buffer
in the data parameter. fs.writeFileSync(filename, data, [options])在 data 参数中需要String
或Buffer
。 see docs . 请参阅文档。
Try to write the array in a string format:尝试以字符串格式编写数组:
// writes 'https://twitter.com/#!/101Cookbooks', 'http://www.facebook.com/101cookbooks'
fs.writeFileSync('./data.json', obj.join(',') , 'utf-8');
Or:或者:
// writes ['https://twitter.com/#!/101Cookbooks', 'http://www.facebook.com/101cookbooks']
var util = require('util');
fs.writeFileSync('./data.json', util.inspect(obj) , 'utf-8');
edit: The reason you see the array in your example is because node's implementation of console.log
doesn't just call toString
, it calls util.format
see console.js source编辑:您在示例中看到数组的原因是因为 node 的console.log
实现不只是调用toString
,它调用util.format
参见 console.js 源
If you're geting [object object]
then use JSON.stringify
如果你得到[object object]
然后使用JSON.stringify
fs.writeFile('./data.json', JSON.stringify(obj) , 'utf-8');
It worked for me.它对我有用。
In my experience JSON.stringify is slightly faster than util.inspect.根据我的经验,JSON.stringify 比 util.inspect 稍快。 I had to save the result object of a DB2 query as a json file, The query returned an object of 92k rows, the conversion took very long to complete with util.inspect, so I did the following test by writing the same 1000 record object to a file with both methods.我不得不将 DB2 查询的结果对象保存为 json 文件,该查询返回了一个 92k 行的对象,使用 util.inspect 转换需要很长时间才能完成,因此我通过编写相同的 1000 条记录对象进行了以下测试到具有这两种方法的文件。
JSON.Stringify JSON.Stringify
fs.writeFile('./data.json', JSON.stringify(obj, null, 2));
Time: 3:57 (3 min 57 sec)时间:3:57(3分57秒)
Result's format:结果格式:
[
{
"PROB": "00001",
"BO": "AXZ",
"CNTRY": "649"
},
...
]
util.inspect实用程序检查
var util = require('util'); fs.writeFile('./data.json', util.inspect(obj, false, 2, false));
Time: 4:12 (4 min 12 sec)时间:4:12(4分12秒)
Result's format:结果格式:
[ { PROB: '00001',
BO: 'AXZ',
CNTRY: '649' },
...
]
could you try doing JSON.stringify(obj);你可以尝试做 JSON.stringify(obj);
Like this像这样
var stringify = JSON.stringify(obj);
fs.writeFileSync('./data.json', stringify , 'utf-8');
Just incase anyone else stumbles across this, I use the fs-extra library in node and write javascript objects to a file like this:以防万一其他人偶然发现这一点,我在节点中使用 fs-extra 库并将 javascript 对象写入这样的文件:
const fse = require('fs-extra');
fse.outputJsonSync('path/to/output/file.json', objectToWriteToFile);
Further to @Jim Schubert
's and @deb2fast
's answers:进一步@Jim Schubert
和@deb2fast
的回答:
To be able to write out large objects
of order which are than ~100 MB
, you'll need to use for...of as shown below and match to your requirements.为了能够写出大于~100 MB
大objects
,您需要使用 for...of ,如下所示并符合您的要求。
const fsPromises = require('fs').promises; const sampleData = {firstName:"John", lastName:"Doe", age:50, eyeColor:"blue"}; const writeToFile = async () => { for (const dataObject of Object.keys(sampleData)) { console.log(sampleData[dataObject]); await fsPromises.appendFile( "out.json" , dataObject +": "+ JSON.stringify(sampleData[dataObject])); } } writeToFile();
Refer https://stackoverflow.com/a/67699911/3152654 for full reference for node.js
limits有关node.js
限制的完整参考,请参阅https://stackoverflow.com/a/67699911/3152654
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.