How do I import this JSON into 2 separate tables?

Question

I'm working with the set of data at this endpoint: https://db.ygoprodeck.com/api/v7/cardinfo.php

I need to take this JSON and import it into 2 tables: Cards and Sets . Below is the schema for each table.

Cards

id
name
type
desc
race
archetype
atk
def
ban_tcg
ban_ocg
ban_goat
level
attribute
linkval
image_url
image_url_small
pendulum_scale
linkmarker_topleft --deeper than single level
linkmarker_top --deeper than single level
linkmarker_topright --deeper than single level
linkmarker_left --deeper than single level
linkmarker_right --deeper than single level
linkmarker_bottomleft --deeper than single level
linkmarker_bottom --deeper than single level
linkmarker_bottomright --deeper than single level

For the Cards table, I can insert any single level data into a table using the code below, but how do I go one level deeper into the linkmarkers node and return the value found into the correct column? For example, for row 8614 in the dataset, the linkmarkers node has only 1 key and its value is "Top", so I would need the script to insert "Top" into the linkmarker_top column and null for all the other linkmarker columns.

Declare @JSON varchar(max)
Declare @JSON2 varchar(max)
SELECT @JSON=BulkColumn
FROM OPENROWSET (BULK 'C:\Users\User\Desktop\YGOCards.json', SINGLE_CLOB) import
SELECT @JSON2 = value 
FROM OPENJSON (@JSON)
select * into #cards
from openjson(@JSON2)
with (
    [id] nvarchar(4000)
    , [name] nvarchar(1000)
    , [type] nvarchar(1000)
    , [desc] nvarchar(1000)
    , [race] nvarchar(1000)
    , [archetype] nvarchar(1000)
    , [atk] nvarchar(1000)
    , [def] nvarchar(1000)
    , [ban_tcg] nvarchar(1000)
    , [ban_ocg] nvarchar(1000)
    , [ban_goat] nvarchar(1000)
    , [level] nvarchar(1000)
    , [attribute] nvarchar(1000)
    , [linkval] nvarchar(1000)
)

Example output row:

id ...   linkmarker_topleft  linkmarker_top  linkmarker_topright  linkmarker_left...
94259633 null                Top             null                 null 

Sets

id
set_name
set_code
set_rarity
set_rarity_code

For the Sets table, I want to basically create a table where I relate each id to the card_set where each card came from, but I'm not sure how to do this. So using the first row of the dataset as an example:

Example output row:

id        set_name              set_code    set_rarity...
34541863  Force of the Breaker  FOTB-EN043  Common

Answer 1

For linkmarkers , you can pull it out as a JSON array, then inside a CROSS APPLY you can shred it using OPENJSON and pivot it.

For the card_sets array, you can CROSS APPLY it from the original cards array, and thereby pull out the cards.id at the same time. This can become the foreign key of your table.

SELECT
  c.*,
  lm.*
INTO #cards
FROM OPENJSON(@json, '$.data')
  WITH (
    [id] int
    , [name] nvarchar(1000)
    , [type] nvarchar(1000)
    , [desc] nvarchar(1000)
    , [race] nvarchar(1000)
    , [archetype] nvarchar(1000)
    , [atk] nvarchar(1000)
    , [def] nvarchar(1000)
    , [ban_tcg] nvarchar(1000)
    , [ban_ocg] nvarchar(1000)
    , [ban_goat] nvarchar(1000)
    , [level] nvarchar(1000)
    , [attribute] nvarchar(1000)
    , [linkval] nvarchar(1000)
    , linkmarkers nvarchar(max) AS JSON
  ) c
CROSS APPLY (
    SELECT
       MIN(CASE WHEN mark.value = 'Top'          THEN 1 END) [Top]
      ,MIN(CASE WHEN mark.value = 'Top-Left'     THEN 1 END) [Top-Left]
      ,MIN(CASE WHEN mark.value = 'Top-Right'    THEN 1 END) [Top-Right]
      ,MIN(CASE WHEN mark.value = 'Bottom'       THEN 1 END) [Bottom]
      ,MIN(CASE WHEN mark.value = 'Bottom-Left'  THEN 1 END) [Bottom-Left]
      ,MIN(CASE WHEN mark.value = 'Bottom-Right' THEN 1 END) [Bottom-Right]
    FROM OPENJSON(c.linkmarkers) mark
) lm;

  
SELECT
  c.id card_id,
  s.*
INTO #sets
FROM OPENJSON(@json, '$.data')
  WITH (
    [id] int,
    card_sets nvarchar(max) AS JSON
  ) c
CROSS APPLY OPENJSON(c.card_sets)
  WITH (
    set_name nvarchar(100),
    set_code nvarchar(10),
    set_rarity nvarchar(100),
    set_rarity_code nvarchar(10)
  ) s;

Note the following:

• You can do the same thing for the other inner arrays.
• Use of JSON path to jump straight to $.data .
• If you need to get the array position you can use OPENJSON without a WITH schema, which gives you the array index of each object.
• Better choice of data types.
• Ideally you would create fixed tables with primary and foreign keys, rather than inserting into # temp tables.

db<>fiddle

Answer 2

Here is my logic (with javascript) for insert record into card table:

1. Grep all record
2. If linkmarkers found, add new field name save the record
3. Insert record

Sample Code:

const jp = require('jsonpath')
const source = require("./source.js") // reference from cardinfo.php

async function main() {
  // Cards
  let query = jp.query(source, "$.data[*]")
  query = query.map((x) => {
    if(x.linkmarkers){
      (x.linkmarkers).forEach((y) => {
        console.log(`linkmarker_${y.toLowerCase()}`)
        x[`linkmarker_${y.toLowerCase()}`] = y
      })      
    }
    return x
  })

  // Set Table
  let table = [["id", "nams", "linkmarket_topleft", "linkmarket_top", "linkmarker_bottom"]]
  for(let i=0; i<query.length; i++){
    let { id, name, linkmarket_topleft, linkmarket_top, linkmarker_bottom } = query[i]
    
    // Your insert SQL query
    // ...
  }
}
main()

Result in csv after point 2: