Get all unique JSON key names with JQ

Question

Is there a way to get all unique key names, without invoking a unique sort outside jq ?

Example file:

{"a": 1, "b": 2, "c": 3}
{"a": 4, "b": 5, "d": 6}

And jq and sort command as I use it now, but I think it's not so efficient:

jq -r keys[] example | sort -u                                                                               
a                                                                                                                                       
b                                                                                                                                       
c                                                                                                                                       
d     

     

Answer 1

Of course.

$ jq -n '[inputs | keys[]] | unique | sort' input.json
[
  "a",
  "b",
  "c",
  "d"
]

Here's another option that may perform better as it doesn't require collecting the keys into an array.

$ jq -n 'reduce (inputs | keys[]) as $k ({}; .[$k] = null) | keys' input.json

Or perhaps, even better:

$ jq -n 'foreach (inputs | keys[]) as $k ({}; .[$k]+=1; if .[$k]==1 then $k else empty end)' input.json

And for larger files, you will want to stream them in anyway so use this variation:

$ jq --stream -n 'foreach inputs[0][-1] as $k ({}; .[$k]+=1; if .[$k]==1 then $k else empty end)' input.json

Answer 2

Two points:

1. The original solution invoking jq and then sort is efficient, especially with respect to memory usage. (The solution involving the -s option effectively forces the entire file to be read into memory).

2. jq's unique implies sort . That is, unique|sort should be simplified to unique to avoid sorting twice.