Import data inside database from JSON in Node.js

Question

I'm just starting to learn node.js and trying to do it with my «Hello World!» project.

So, the idea here is very simple, I'm asking for a JSON file from server via API and receive an answer which looks like this:

{
    "files": [{
        "url": "http://auction-api-eu.worldofwarcraft.com/auction-data/ef92b9868d1316c1066e9fb5e6d979a0/auctions.json",
        "lastModified": 1485460486000
    }]
}

Right after it and only if lastModified parameter has been changed I request for a auctions.json file via POST request. Structure inside auctions.json is very simple:

{
"realms": [
  {"name":"Outland","slug":"outland"}],
"auctions": [
{"auc":1270238130,"item":82800,"owner":"Tapicka","ownerRealm":"Outland","bid":79999999,"buyout":79999999,"quantity":1,"timeLeft":"VERY_LONG","rand":0,"seed":577282176,"context":0,"modifiers":[{"type":3,"value":1333},{"type":4,"value":50331667},{"type":5,"value":1}],"petSpeciesId":1333,"petBreedId":19,"petLevel":1,"petQualityId":3}

But I have no need for a full data inside of it. I'm only required in "auc", "item" "owner", "owner Realm" "bid" "buyout" "quantity" "timeleft" and only if "item" parameter is equal to X. So here is my question. What is the best way to use data inside actions.json? To be honest I don't have enough experience with node.js and have no idea what's the best way to use it in my case: So my next step should be:

1. parse JSON in web via url and import this data in DB?
2. download JSON somewhere then parse it locally and only right after it import data in DB?
3. download JSON file and import all of it (w/o filtering) in DB and only right after it show the exact data that I needed
4. another way

and by the way. what DB should I use? I'm not sure that MongoDB is the right one here. Someone give me an advice that I should take a close look at Redis . Is it true?

I hope that someone me an answer that will help me to achieve my goal.

Answer 1

First of all Redis is not a database it's a in-memory-cache so if you need to persist your data you have to use a database and if you are not sure which one to choose MongoDB is a good start. For the json data it seems you need to download it using "request" package and then modify it as you like and finally persist it to database. If you choose MongoDB you can use "mongoose" package to deal with database.