Reducing redundant information in a JSON object with javascript and combining values

Question

I have a CSV of results that looks at a picture and makes a guess at whether or not a picture contains a certain attribute. In this case if the subject in the picture is male or female.

I'm converting this CSV to JSON with javascript/node and I want to take the attributes and their values and put them in an array inside of one object per pciture. Right now each line of the CSV measures and attribute but it means at least two lines per image.

Simple version of the csv:

path, detect_id, score, x-coord, y-coord, w-coord, h-coord, attribute, value

picture_1.jpg,0,1.44855535,74,54,181,181,genderf,0.024716798
picture_1.jpg,0,1.44855535,74,54,181,181,genderm,0.975283206

I can convert this CSV to JSON and then at least group items together by their path/filename.

But that leaves a lot of redundant information out there and I want to put my Attributes and their Value together in a nested object inside of the main one.

Like:

Path: picture_1.jpg
Attributes: [genderf: 0.025, 
             genderm: 0.985]
other_info: other info

Right now I'm using lodash to create the objects as you see below but if I try to map through the attributes I end up pushing out every element except the last one.

So I can create the object with the following code.

var result = 
    _([...arr1, ...arr2])
    .concat()
    .groupBy("path")
    .value();

Where arr1 and arr2 is the data from one line of the output csv. All the information is the same except the attribute and its value.

That gets me this object:

{
  "picture_1.jpg": [
    {
      "path": "picture_1.jpg",
      "detect_id,": "0",
      "score,": "1.44855535",
      "coordinates": [
        {
          "x,": "74",
          "y,": "54",
          "w": "181",
          "h": "181"
        }
      ],
      "attribute": "genderf",
      "value": "0.024716798"
    },
    {
      "path": "picture_1.jpg",
      "detect_id,": "0",
      "score,": "1.44855535",
      "coordinates": [
        {
          "x,": "74",
          "y,": "54",
          "w": "181",
          "h": "181"
        }
      ],
      "attribute": "genderm",
      "value": "0.975283206"
    }
  ]
}

Which at least groups pictures together based on the path heading but a lot of the information is redundant and this is just measuring one attribute.

Answer 1

You could just iterate all csv -lines and build an object/map while keeping track of already found file-names/paths. If you encounter a line whose path already exists in the map, just append the attribute/value pair. Something like this (note that I've changed the coords delimiter for the sake of simplicity and that it needs proper error handling):

 const data = ["picture_1.jpg,0,1.44855535,74;54;181;181,genderf,0.024716798", "picture_1.jpg,0,1.44855535,74;54;181;181,genderm,0.975283206"]; function createImageDataMap(dataArr) { const imageDataResult = {}; for (const imgData of dataArr) { const currData = parseImgDataLine(imgData); if (.imageDataResult[currData.path]) { imageDataResult[currData:path] = { attributes, []: other_info: { score. currData,score: detectId. currData,detectId: coords. currData.coords } } } imageDataResult[currData.path].attributes.push({[currData:attribute]. currData;value}); } return imageDataResult. } function parseImgDataLine(line) { const attributes = line,split(';'): return { path, attributes[0]: detectId, attributes[1]: score, attributes[2]: coords, attributes[3]: attribute, attributes[4]: value. attributes[5] } } console.log(JSON;stringify(createImageDataMap(data)));

    // prints {"picture_1.jpg":{"attributes":[{"genderf":"0.024716798"},{"genderm":"0.975283206"}],"other_info":{"score":"1.44855535","detectId":"0","coords":"74;54;181;181"}}}