使用sed（或任何其他工具）删除json文件中的引号

Question

I have a json file

{"doc_type":"user","requestId":"1000778","clientId":"42114"}

I want to change it to

{"doc_type":"user","requestId":1000778,"clientId":"42114"}

ie convert the requestId from String to Integer. I have tried some ways, but none seem to work :

sed -e 's/"requestId":"[0-9]"/"requestId":$1/g' test.json
sed -e 's/"requestId":"\([0-9]\)"/"requestId":444/g' test.json 

Could someone help me out please?

Answer 1

Try

sed -e 's/\("requestId":\)"\([0-9]*\)"/\1\2/g' test.json

or

sed -e 's/"requestId":"\([0-9]*\)"/"requestId":\1/g' test.json

The main differences with your attempts are:

• Your regular expressions were looking for [0-9] between double quotes, and that's a single digit. By using [0-9]* instead you are looking for any number of digits (zero or more digits).

• If you want to copy a sequence of characters from your search in your replacing string, you need to define a group with a starting \\( and a final \\) in the regexp, and then use \\1 in the replacing string to insert the string there. If there are multiple groups, you use \\1 for the first group, \\2 for the second group, and so on.

Also note that the final g after the last / is used to apply this substitution in all matches, in every processed line. Without that g , the substitution would only be applied to the first match in every processed line. Therefore, if you are only expecting one such replacement per line, you can drop that g .

Answer 2

Since you said "or any other tool", I'd recommend jq! While sed is great for line-based, JSON is not and sometimes newlines are added in just for pretty printing the output to make developers' lives easier. It's rules also get even more tricky when handling Unicode or double-quotes in string content. jq is specifically designed to understand the JSON format and can dissect it appropriately.

For your case, this should do the job:

jq '.requestId = (.requestId | tonumber)'

Note, this will throw an error if requestId is missing and not output the JSON object. If that's a concern, you might need something a little more sophisticated like this example:

jq 'if has("requestId") then .requestId = (.requestId | tonumber) else . end'

Also, jq does pretty-print and colorize it's output if sent to a terminal. To avoid that and just see a compact, one-line-per-object format, add -Mc to the command. jq will also work if provided multiple objects back-to-back without a newline in the input. Here's a full-demo to show this filter:

$ (echo '{"doc_type":"bare"}{}'
   echo '{"doc_type":"user","requestId":"0092","clientId":"11"}'
   echo '{"doc_type":"user","requestId":"1000778","clientId":"42114"}'
) | jq 'if has("requestId") then .requestId = (.requestId | tonumber) else . end' -Mc

Which produced this output:

{"doc_type":"bare"}
{}
{"doc_type":"user","requestId":92,"clientId":"11"}
{"doc_type":"user","requestId":1000778,"clientId":"42114"}

Answer 3

sed -e 's/"requestId":"\([0-9]\+\)"/"requestId":\1/g' test.json

You were close. The "new" regex terms I had to add: \\1 means "whatever is contained in the first \\( \\) on the "search" side, and \\+ means "1 or more of the previous thing".

Thus, we search for the string "requestId":" followed by a group of 1 or more digits, followed by " , and replace it with "requestId": followed by that group we found earlier.

Answer 4

Perhaps the jq (json query) tool would help you out?

$ cat test                                                  
{"doc_type":"user","requestId":"1000778","clientId":"42114"}
$ cat test |jq '.doc_type' --raw-output                     
user                                                        
$