Convert CSV to nested JSON

Question

I am trying to convert following CSV to JSON below. Any help will be appreciated.

Sample of CSV file (File would contain lot of network groups with network,host attributes)

Type,Value ,Name
Network,10.0.0.0/8,network_group_3
Host,10.0.0.27,network_group_3
Host,10.0.0.28,network_group_3
Network,10.10.10.0/24,network_group_4
Network,10.10.20.0/24,network_group_4
Host,10.10.10.6,network_group_4

Output in JSON Needed

netgroup =
 "literals": [
    {
      "type": "Network",
      "value": "10.0.0.0/8"
    },
    {
      "type": "Host",
      "value": "10.0.0.27"
    },
    {
      "type": "Host",
      "value": "10.0.0.28"
    }
  ],
  "name": "network_group_3"
},
{ 
 "literals": [
    {
      "type": "Network",
      "value": "10.10.10.0/24"
    },
    {
      "type": "Network",
      "value": "10.10.20.0/24"
    },
    {
      "type": "Host",
      "value": "10.0.0.6
    }
  ],
  "name": "network_group_4"

Answer 1

Here is a good explanation of Python for conerting CSV to JSON:

http://www.idiotinside.com/2015/09/18/csv-json-pretty-print-python/

Answer 2

Here is a solution using jq

If the file filter.jq contains

[
  split("\n")                                                  # split string into lines
| (.[0]    | split(",")) as $headers                           # split header
| (.[1:][] | split(","))                                       # split data rows
| select(length>0)                                             # get rid of empty lines
]
| [
      group_by(.[2])[]
    | {
         name:     .[0][2],
         literals: map({type:.[0], value:.[1]})
      }
  ]

and your data is in a file called data then

jq -M -R -s -r -f filter.jq data

will generate

[
  {
    "name": "network_group_3",
    "literals": [
      {
        "type": "Network",
        "value": "10.0.0.0/8"
      },
      {
        "type": "Host",
        "value": "10.0.0.27"
      },
      {
        "type": "Host",
        "value": "10.0.0.28"
      }
    ]
  },
  {
    "name": "network_group_4",
    "literals": [
      {
        "type": "Network",
        "value": "10.10.10.0/24"
      },
      {
        "type": "Network",
        "value": "10.10.20.0/24"
      },
      {
        "type": "Host",
        "value": "10.10.10.6"
      }
    ]
  }
]

Answer 3

Late is better than never, so using convtools library:

from convtools import conversion as c
from convtools.contrib.tables import Table


# store converter somewhere if it needs to be reused
converter = (
    c.group_by(c.item("Name"))
    .aggregate(
        {
            "literals": c.ReduceFuncs.Array(
                {
                    "type": c.item("Type"),
                    "value": c.item("Value"),
                }
            ),
            "name": c.item("Name"),
        }
    )
    .gen_converter()
)

# iterable of rows and it can only be consumed once
rows = Table.from_csv("tmp2.csv", header=True).into_iter_rows(dict)

assert converter(rows) == [
 {'literals': [{'type': 'Network', 'value': '10.0.0.0/8'},
   {'type': 'Host', 'value': '10.0.0.27'},
   {'type': 'Host', 'value': '10.0.0.28'}],
  'name': 'network_group_3'},
 {'literals': [{'type': 'Network', 'value': '10.10.10.0/24'},
   {'type': 'Network', 'value': '10.10.20.0/24'},
   {'type': 'Host', 'value': '10.10.10.6'}],
  'name': 'network_group_4'}]