提取嵌套JSON对象的模式

Question

Let's assume this is the source json file:

{    
    "name": "tom",
    "age": 12,
    "visits": {
        "2017-01-25": 3,
        "2016-07-26": 4,
        "2016-01-24": 1
    }
}

I want to get:

[
  "age",
  "name",
  "visits.2017-01-25",
  "visits.2016-07-26",
  "visits.2016-01-24"
]

I am able to extract the keys using: jq '. | keys' file.json jq '. | keys' file.json jq '. | keys' file.json , but this skips nested fields. How to include those?

Answer 1

With your input, the invocation:

jq 'leaf_paths | join(".")'

produces:

"name"
"age"
"visits.2017-01-25"
"visits.2016-07-26"
"visits.2016-01-24"

If you want to include "visits", use paths . If you want the result as a JSON array, enclose the filter with square brackets: [ ... ]

If your input might include arrays, then unless you are using jq 1.6 or later, you will need to convert the integer indices to strings explicitly; also, since leaf_paths is now deprecated, you might want to use its def. The result:

jq 'paths(scalars) | map(tostring) | join(".")'

allpaths

To include paths to null, you could use allpaths defined as follows:

def allpaths:
  def conditional_recurse(f):  def r: ., (select(.!=null) | f | r); r;
  path(conditional_recurse(.[]?)) | select(length > 0);

Example:

{"a": null, "b": false} | allpaths | join(".")

produces:

"a"
"b"

all_leaf_paths

Assuming jq version 1.5 or higher, we can get to all_leaf_paths by following the strategy used in builtins.jq, that is, by adding these definitions:

def allpaths(f):
  . as $in | allpaths | select(. as $p|$in|getpath($p)|f);

def isscalar:
  . == null or . == true or . == false or type == "number" or type == "string";

def all_leaf_paths: allpaths(isscalar);

Example:

{"a": null, "b": false, "object":{"x":0} } | all_leaf_paths | join(".")

produces:

"a"
"b"
"object.x"

Answer 2

This does what you want but it doesn't return the data in an array, but it should be an easy modification:

https://github.com/ilyash/show-struct

you can also check out this page: https://ilya-sher.org/2016/05/11/most-jq-you-will-ever-need/

Answer 3

Some time ago, I wrote a structural-schema inference engine that produces simple structural schemas that mirror the JSON documents under consideration, eg for the sample JSON given here, the inferred schema is:

{
  "name": "string",
  "age": "number",
  "visits": {
    "2017-01-25": "number",
    "2016-07-26": "number",
    "2016-01-24": "number"
  }
}

This is not exactly the format requested in the original posting, but for large collections of objects, it does provide a useful overview.

More importantly, there is now a complementary validator for checking whether a collection of JSON documents matches a structural schema. The validator checks against schemas written in JESS (JSON Extended Structural Schemas), a superset of the simple structural schemas (SSS) produced by the schema inference engine .

(The idea is that one can use the SSS as a starting point to add more elaborate constraints, including recursive constraints, within-document referential integrity constraints, etc.)

For reference, here is how one the SSS for your sample.json would be produced using the "schema" module :

jq 'include "schema"; schema' source.json > source.schema.json

And to validate source.json against a SSS or ESS:

JESS --schema  source.schema.json  source.json