如何使用 angular 和 nodejs 将 csv 数据转储到 mongoDB

Question

I've a csv file and want to dump the data in it into mongodb using angular and nodejs.我有一个 csv 文件，想使用 angular 和 nodejs 将其中的数据转储到 mongodb 中。

Need help to read csv file's data using angular and parse it and store in mongodb需要帮助使用 angular 读取 csv 文件的数据并解析它并存储在 mongodb 中

 import { Injectable } from '@angular/core'; import { Http } from '@angular/http'; import 'rxjs/add/operator/map'; @Injectable() export class TemplateService { domain = 'http://localhost:3000'; constructor(private http: Http) { } postCsvToMongo(csvJson: any) { console.log('CSV!!!' + csvJson); return this.http.post(this.domain + '/fu/fileupload', csvJson).map(res => res.json()); } }

this is the code i'm using to send json object over post routes.这是我用来通过 post 路由发送 json 对象的代码。

 <!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <title>Error</title> </head> <body> <pre>TypeError: Converting circular structure to JSON <br> &nbsp; &nbsp;at JSON.stringify (&lt;anonymous&gt;) <br> &nbsp; &nbsp;at router.post (/home/agam/projects/AMEX/routes/fileUpload.js:10:23) <br> &nbsp; &nbsp;at Layer.handle [as handle_request] (/home/agam/projects/AMEX/node_modules/express/lib/router/layer.js:95:5) <br> &nbsp; &nbsp;at next (/home/agam/projects/AMEX/node_modules/express/lib/router/route.js:137:13) <br> &nbsp; &nbsp;at Route.dispatch (/home/agam/projects/AMEX/node_modules/express/lib/router/route.js:112:3) <br> &nbsp; &nbsp;at Layer.handle [as handle_request] (/home/agam/projects/AMEX/node_modules/express/lib/router/layer.js:95:5) <br> &nbsp; &nbsp;at /home/agam/projects/AMEX/node_modules/express/lib/router/index.js:281:22 <br> &nbsp; &nbsp;at Function.process_params (/home/agam/projects/AMEX/node_modules/express/lib/router/index.js:335:12) <br> &nbsp; &nbsp;at next (/home/agam/projects/AMEX/node_modules/express/lib/router/index.js:275:10) <br> &nbsp; &nbsp;at Function.handle (/home/agam/projects/AMEX/node_modules/express/lib/router/index.js:174:3) <br> &nbsp; &nbsp;at router (/home/agam/projects/AMEX/node_modules/express/lib/router/index.js:47:12) <br> &nbsp; &nbsp;at Layer.handle [as handle_request] (/home/agam/projects/AMEX/node_modules/express/lib/router/layer.js:95:5) <br> &nbsp; &nbsp;at trim_prefix (/home/agam/projects/AMEX/node_modules/express/lib/router/index.js:317:13) <br> &nbsp; &nbsp;at /home/agam/projects/AMEX/node_modules/express/lib/router/index.js:284:7 <br> &nbsp; &nbsp;at Function.process_params (/home/agam/projects/AMEX/node_modules/express/lib/router/index.js:335:12) <br> &nbsp; &nbsp;at next (/home/agam/projects/AMEX/node_modules/express/lib/router/index.js:275:10) </pre> </body> </html>

this is the error i'm getting when I'm trying to debug the request received over the post request on node side.这是我在尝试调试通过节点端的 post 请求收到的请求时遇到的错误。 And how to receive the file over back end??以及如何通过后端接收文件？

 var csv = require('fast-csv'); var mongoose = require('mongoose'); var Bulletin = require('../models/bulletin'); var multer = require("multer"); module.exports = (router) => { router.post('/fileupload', (req, res) => { console.log('@@@ ' + req); res.send(req.files); }); /*router.post('/fileupload', multer({dest: "./uploads/"}).array("uploads", 12), function(req, res) { res.send(req.files); });*/ return router; };

This is the nodejs code这是 nodejs 代码

 1","Fed official says weak data caused by weather, should not slow taper","http://www.latimes.com/business/money/la-fi-mo-federal-reserve-plosser-stimulus-economy-20140310,0,1312750.story\\?track=rss","Los Angeles Times","b","www.latimes.com","1394470370698 2","Fed's Charles Plosser sees high bar for change in pace of tapering","http://www.livemint.com/Politics/H2EvwJSK2VE6OF7iK1g3PP/Feds-Charles-Plosser-sees-high-bar-for-change-in-pace-of-ta.html","Livemint","b","www.livemint.com","1394470371207 3","US open: Stocks fall after Fed official hints at accelerated tapering","http://www.ifamagazine.com/news/us-open-stocks-fall-after-fed-official-hints-at-accelerated-tapering-294436","IFA Magazine","b","www.ifamagazine.com","1394470371550 4","Fed risks falling 'behind the curve', Charles Plosser says","http://www.ifamagazine.com/news/fed-risks-falling-behind-the-curve-charles-plosser-says-294430","IFA Magazine","b","www.ifamagazine.com","1394470371793 5","Fed's Plosser: Nasty Weather Has Curbed Job Growth","http://www.moneynews.com/Economy/federal-reserve-charles-plosser-weather-job-growth/2014/03/10/id/557011","Moneynews","b","www.moneynews.com","1394470372027 6","Plosser: Fed May Have to Accelerate Tapering Pace","http://www.nasdaq.com/article/plosser-fed-may-have-to-accelerate-tapering-pace-20140310-00371","NASDAQ","b","www.nasdaq.com","1394470372212

This is my sample csv file这是我的示例 csv 文件

 > amex@1.0.0 start /home/agam/projects/AMEX > node index.js (node:6478) DeprecationWarning: current URL string parser is deprecated, and will be removed in a future version. To use the new parser, pass option { useNewUrlParser: true } to MongoClient.connect. listening on port 3000 Conected to database: amex @@@ [object Object]

this is the thing it is printing这是它正在打印的东西

 // 1) Express for nodejs const express = require('express'); const app = express(); const template = require('./routes/template'); const upload = require('./routes/upload'); const path = require('path'); // const fileUpload = require('express-fileupload'); const router = express.Router(); const fu = require('./routes/fileUpload')(router); //const carts = require('./routes/carts')(router); // 5) body parser const bodyParser = require('body-parser'); const cors = require('cors'); // CORS is a node.js package for providing a Connect/Express middleware that can be used to enable CORS with various options. // 2) Mongodb connection var mongoose = require('mongoose'); const config = require('./config/database'); mongoose.connect(config.uri, (err) => { if (err) { console.log('error ' + err); } else { console.log('Conected to database: ' + config.db); } }); app.use(cors({ origin: 'http://localhost:4200' })); // 5) put bodyparser code before the routes (converts request to req.body) app.use(bodyParser.urlencoded({ extended: true })); app.use(bodyParser.json()); app.use(function(req, res, next) { res.header("Access-Control-Allow-Origin", "*"); res.header('Access-Control-Allow-Methods', 'PUT, GET, POST, DELETE, OPTIONS'); res.header("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept"); next(); }); // 3) connect node.js to angular app.use(express.static(__dirname + '/client/dist/')); // app.use(fileUpload()); app.use('/fu', fu); // Use Product routes in application //app.use('/carts', carts); // app.get('*', (req, res) => { res.sendFile(path.join(__dirname + '/client/dist/index.html')); }); app.listen(3000, () => { console.log('listening on port 3000'); });

this is my index.js file which you are expecting这是您期望的我的 index.js 文件

Answer 1

Hi I have done a few changes in your StackBiltz example, so I'm able to achieve the payload to be posted via a post call to in the form of JSON array and created a service which accepts this payload and on subscription passes the payload to your node post api嗨，我在您的StackBiltz示例中做了一些更改，因此我能够通过 post 调用以 JSON 数组的形式实现要发布的有效负载，并创建了一个接受此有效负载的服务，并在订阅时将有效负载传递给您的节点发布 api

Created a function named pass创建了一个名为 pass 的函数

test.component.ts test.component.ts

import { Component, OnInit } from '@angular/core';    
import { Router } from "@angular/router";
import { FileUtil } from './file.util';
import { Constants } from './test.constants';
import {TestService} from './test.service';

@Component({
  selector: 'app-test',
  templateUrl: './test.component.html',
  styleUrls: ['./test.component.css']
})
export class TestComponent implements OnInit {

  fileImportInput: any;
  data;
  csvRecords:any = [];

  constructor(private _router: Router,
    private _fileUtil: FileUtil,private testService:TestService
  ) { }

  ngOnInit() {
  }

  // METHOD CALLED WHEN CSV FILE IS IMPORTED
  fileChangeListener($event): void {

    var text = [];
    var target = $event.target || $event.srcElement;
    var files = target.files;

    if (Constants.validateHeaderAndRecordLengthFlag) {
      if (!this._fileUtil.isCSVFile(files[0])) {
        alert("Please import valid .csv file.");
        this.fileReset();
      }
    }

    var input = $event.target;
    var reader = new FileReader();
    reader.readAsText(input.files[0]);

    reader.onload = (data) => {
      let csvData = reader.result;
      let csvRecordsArray = csvData.split(/\r\n|\n/);

      var headerLength = -1;
      if (Constants.isHeaderPresentFlag) {
        let headersRow = this._fileUtil.getHeaderArray(csvRecordsArray, Constants.tokenDelimeter);
        headerLength = headersRow.length;
      }

      this.csvRecords = this._fileUtil.getDataRecordsArrayFromCSVFile(csvRecordsArray,
        headerLength, Constants.validateHeaderAndRecordLengthFlag, Constants.tokenDelimeter);
        this.passThisFunction(this.csvRecords);
        console.log('this.csvRecords',this.csvRecords);
      if (this.csvRecords == null) {
        //If control reached here it means csv file contains error, reset file.
        this.fileReset();
      }      
    }
    reader.onerror = function () {
      alert('Unable to read ' + input.files[0]);
    };
  };

  fileReset() {
    this.fileImportInput.nativeElement.value = "";
    this.csvRecords = [];
  }

  passThisFunction(records:any){
      console.log('inside function call');
      console.log('records',records);
      let json=JSON.stringify(records);
      console.log('jsonified',json);
      var str = json.replace(/},/g, "},\r\n");
        console.log('das' + str);
      this.testService.postCsvToMongo(str).subscribe((data) => {
                console.log(data);
      });
  }

}

test.component.html测试组件.html

<h4>Read .CSV File using Angular2, JQuery and Bootstrap</h4>

<table>
    <tr>
        <td>
            <input type="file" 
                #fileImportInput
                name="File Upload" 
                id="txtFileUpload" 
                class="btn btn-primary" 
                (change)="fileChangeListener($event)" 
                accept=".csv"
                />
        </td>

    </tr>
</table>

<div>
    <table class="table table-responsive table-hover" border="1" 
        style="width : 50%;">
        <tbody>
            <ng-container >
                <tr >                   
                        <td *ngFor="let csvRec of csvRecords;let i=index">{{csvRec}}</td>
                </tr>
            </ng-container>
        </tbody>
    </table>
</div>

file.util.ts文件.util.ts

import { Injectable }       from '@angular/core';

@Injectable()
export class FileUtil {

    constructor() {}

    isCSVFile(file) {
        return file.name.endsWith(".csv");
    }

    getHeaderArray(csvRecordsArr, tokenDelimeter) {        
        let headers = csvRecordsArr[0].split(tokenDelimeter);
        let headerArray = [];
        for (let j = 0; j < headers.length; j++) {
            headerArray.push(headers[j]);
        }
        return headerArray;
    }

    validateHeaders(origHeaders, fileHeaaders) {
        if (origHeaders.length != fileHeaaders.length) {
            return false;
        }

        var fileHeaderMatchFlag = true;
        for (let j = 0; j < origHeaders.length; j++) {
            if (origHeaders[j] != fileHeaaders[j]) {
                fileHeaderMatchFlag = false;
                break;
            }
        }
        return fileHeaderMatchFlag;
    }

    getDataRecordsArrayFromCSVFile(csvRecordsArray, headerLength, 
        validateHeaderAndRecordLengthFlag, tokenDelimeter) {
          console.log(csvRecordsArray);
          console.log(headerLength);
          console.log(validateHeaderAndRecordLengthFlag);
          console.log(tokenDelimeter);
        var dataArr:any = [];

        for (let i = 0; i < csvRecordsArray.length; i++) {
            let data = csvRecordsArray[i].split(tokenDelimeter);

            if(validateHeaderAndRecordLengthFlag && data.length != headerLength){
                if(data==""){
                    alert("Extra blank line is present at line number "+i+", please remove it.");
                    return null;
                }else{
                    alert("Record at line number "+i+" contain "+data.length+" tokens, and is not matching with header length of :"+headerLength);
                    return null;
                }
            }
            dataArr=data;
            console.log('dataArr',dataArr);
        }   
        return dataArr;
    }

}

test.service.ts测试服务.ts

import{Injectable} from '@angular/core';
import{Http} from '@angular/http';
import {map} from 'rxjs/operators';

@Injectable()
export class TestService{
  API_URL:string='your api url';
  constructor(private http:Http){}

  postCsvToMongo(csvJson:any){
    return this.http.post(this.API_URL,csvJson)
    .pipe(map((res)=>console.log(res)));
  }
}

如何使用 angular 和 nodejs 将 csv 数据转储到 mongoDB

问题描述

1 个解决方案

解决方案1
1 2018-08-17 17:10:48

如何使用 angular 和 nodejs 将 csv 数据转储到 mongoDB

问题描述

1 个解决方案

解决方案1 1 2018-08-17 17:10:48

解决方案1
1 2018-08-17 17:10:48