Find and edit a Json file using bash

Question

I have multiple files in the following format with different categories like:

{
    "id": 1,
    "flags": ["a", "b", "c"],
    "name": "test",
    "category": "video",
    "notes": ""
}

Now I want to append all the files flags whose category is video with string d . So my final file should look like the file below:

{
    "id": 1,
    "flags": ["a", "b", "c", "d"],
    "name": "test",
    "category": "video",
    "notes": ""
}

Now using the following command I am able to find files of my interest, but now I want to work with editing part which I an unable to find as there are 100's of file to edit manually, eg

find . - name * | xargs grep "\"category\": \"video\"" | awk '{print $1}' | sed 's/://g'

Answer 1

You can do this

find . -type f | xargs grep -l '"category": "video"' | xargs  sed -i -e '/flags/ s/]/, "d"]/'

This will find all the filnames which contain line with "category": "video" , and then add the "d" flag.

Details:

find . -type f 

=> Will get all the filenames in your directory

xargs grep -l '"category": "video"'

=> Will get those filenames which contain the line "category": "video"

xargs  sed -i -e '/flags/ s/]/, "d"]/'

=> Will add the "d" letter to the flags: line.

Answer 2

If the files are in the same format, this command may help (version for a single file):

ex +':/category.*video/norm kkf]i, "d"' -scwq file1.json

or:

ex +':/flags/,/category/s/"c"/"c", "d"/' -scwq file1.json

which is basically using Ex editor (now part of Vim).

Explanation:

• + - executes Vim command ( man ex )
• :/pattern_or_range/cmd - find pattern, if successful execute another Vim commands ( :h :/ )

• norm kkf]i - executes keystrokes in normal mode

  • kk - move cursor up twice
  • f] - find ]
  • i, "d" - insert , "d"
• -s - silent mode
• -cwq - executes wq (write & quit)

For multiple files, use find and -execdir or extend above ex command to:

ex +'bufdo!:/category.*video/norm kkf]i, "d"' -scxa *.json

Where bufdo! executes command for every file, and -cxa saves every file. Add -V1 for extra verbose messages.

If flags line is not 2 lines above, then you may perform backward search instead. Or using similar approach to @sps by replacing ] with d .

See also: How to change previous line when the pattern is found? at Vim.SE .

Answer 3

"TWEET!!" ... (yellow flag thown to the ground) ... Time Out!

What you have, here, is "a JSON file." You also have, at your #!shebang command, your choice of(!) full-featured programming languages ... with intimate and thoroughly-knowledgeale support for JSON ... with which you can very-speedily write your command-file.

Even if it is "theoretically possible" to do this using "bash scripts," this is roughly equivalent to "putting a beautiful stone archway over the front-entrance to a supermarket." Therefore, "waste ye no time" in such an utterly-profitless pursuit. Write a script, using a language that "honest-to-goodness knows about(!) JSON," to decode the contents of the file, then manipulate it (as a data-structure), then re-encode it again.

Answer 4

Here is a more appropriate approach using PHP in shell:

FILE=foo2.json php -r '$file = $_SERVER["FILE"]; $arr = json_decode(file_get_contents($file)); if ($arr->category == "video") { $arr->flags[] = "d"; file_put_contents($file,json_encode($arr)); }'

Which will load the file, decode into array, add "d" into flags property only when category is video, then write back to the file in JSON format.

To run this for every json file, you can use find command, eg

find . -name "*.json" -print0 | while IFS= read -r -d '' file; do
    FILE=$file
    # run above PHP command in here
done

Answer 5

Using jq :

find . -type f | xargs cat | jq 'select(.category=="video") | .flags |= . + ["d"]'

Explanation:

jq 'select(.category=="video") | .flags |= . + ["d"]'
   # select(.category=="video") =>  filters by category field
   # .flags |= . + ["d"] => Updates the flags array