How to define and store data structure properties in a database

Question

I'm working on a PHP web app with a Postgres backend. The app uses a variety of APIs and want to be able to add/edit the API endpoints used by the system dynamically.

I'm planning to handle variations in the API request URLs with replacement codes, for example: http://api.com/?key=%%api_key%%&user_id=%%user_id%%

The part I don't have a plan for is how to define and store the "shape" of the returned API data. For example, let's say I want to get a user's comments from different APIs. The structure of the data will likely differ from one to another. Even if they are all json data (vs. XML), the property(s) I care about will be located in different places. Is there an established way to do this?

I'm considering a text field with a json "map" to the location of the properties:

{
    "user": {
        "comments" : %%HERE%%
    }
}

Presumably my app would parse this, and loop through it to find the indicated location and then use it to find the data in the corresponding location in the response data. But I'm not exactly how to do it or if this is even the best way. Any suggestions are welcome.

Thinking this through a bit more, I realize that an alternative approach would be to store some kind of algorithm to finding the data. Is there a precedent for this? I briefly considered the idea of storing raw PHP code that could be executed to parse the data, but this feels very wrong and potentially dangerous/insecure.

Answer 1

JOLT may be helpful. It's for transforming JSON to JSON, much like XSLT for XML. You could write a spec for each new api, which would transform the data into a uniform format for your app to read.