如何从存储为 json 数据的字符串中提取字符并将它们放在 SQL 服务器中的动态列数中

Question

I have a column of string in SQL Server that stores JSON data with all the braces and colons included.

My problem is to extract all the key and value pairs and store them in separate columns with the key as the column header. What makes this challenging is that every record has different number of key/value pairs.

For example in the image below showing 3 records, the first record has 5 key/value pairs- EndUseCommunityMarket of 2 , EndUseProvincial Market of 0 , and so on. The second record has 1 key/value pair, and the third record has two key/value pairs.

If I have to show how I want this in excel it would be like:

I have seen some SQL code examples that does something similar but for a fixed number of columns, unlike this one it varies for every record.

Please I need a SQL statement that can achieve this as I am working on thousands of records.

Below is this data copied from sql server :

catch_ext
{"NfdsFadMonitoring":{"EndUseEaten":1}}
{"NfdsFadMonitoring":{"EndUseCommunityMarket":3}}
{"NfdsFadMonitoring":{"SpeciesComment":"","EndUseCommunityMarket":2}}
{"NfdsFadMonitoring":{"SpeciesComment":"mix reef fis","EndUseEaten":31}}
{"NfdsFadMonitoring":{"SpeciesComment":"10 fish with a total of 18kg","EndUseCommunityMarket":0,"EndUseProvincialMarket":0,"EndUseUrbanMarket":8,"EndUseEaten":1,"EndUseGivenAway":1}}
{"NfdsFadMonitoring":{"SpeciesComment":"mix reef fis","EndUseEaten":18}}

Answer 1

I expect you don't want to dynamically create a table, instead you probably want to create a property mapping table. Here is a quick overview of the design.

Object table -- this stores the base information about your object
============
ID            -- unique id field for every object.
Name

Property types table -- this stores all the property types
====================
Property_Type_ID -- unique type id
Description      -- describes property

Object2Property -- stores the values for each property
===============
ObjectID -- the object
Property_Type_ID -- the property type
Value -- the value.

Using a model like this lets your properties be as dynamic as you wish but you don't have to create columns dynamically -- something that is hard and error prone.

using your specific example the tables would look like this

OBJECT 
ID     NAME
1      WHAOO
2      RED SNAMPPER
3      KAWAKAWA

Property Types
ID     DESC
1      EndUseCommunityMarket
2      EndUseProvincialMarket 
3      EndUseUrbanMarket
4      EndUseEaten
5      EndUseGivenAway
6      Comment

Map
ObjID   TypeID   Value
1        1       2  
1        2       0
1        3       0
1        4       0
1        5       0
2        2       50
3        3       8
3        5       1

Answer 2

A. ROWS

Dynamic columns are a lot like rows.

You could use OPENJSON (Transact-SQL)

DECLARE @json2 NVARCHAR(4000) = N'{"NfdsFadMonitoring":{"SpeciesComment":"10 fish with a total of 18kg","EndUseCommunityMarket":0,"EndUseProvincialMarket":0,"EndUseUrbanMarket":8,"EndUseEaten":1,"EndUseGivenAway":1}}';

SELECT [key], value
FROM OPENJSON(@json2,'lax $.NfdsFadMonitoring')

Output

key value
SpeciesComment  10 fish with a total of 18kg
EndUseCommunityMarket   0
EndUseProvincialMarket  0
EndUseUrbanMarket   8
EndUseEaten 1
EndUseGivenAway 1

Your inputs

CREATE TABLE ForEloga (Id int,Json nvarchar(max));

Insert into ForEloga Values 
(1,'{"NfdsFadMonitoring":{"EndUseEaten":1}}'),
(2,'{"NfdsFadMonitoring":{"EndUseCommunityMarket":3}}'),
(3,'{"NfdsFadMonitoring":{"SpeciesComment":"","EndUseCommunityMarket":2}}'),
(4,'{"NfdsFadMonitoring":{"SpeciesComment":"mix reef fis","EndUseEaten":31}}'),
(5,'{"NfdsFadMonitoring":{"SpeciesComment":"10 fish with a total of 18kg","EndUseCommunityMarket":0,"EndUseProvincialMarket":0,"EndUseUrbanMarket":8,"EndUseEaten":1,"EndUseGivenAway":1}}'),
(6,'{"NfdsFadMonitoring":{"SpeciesComment":"mix reef fis","EndUseEaten":18}}');

SELECT Id, [key], value
FROM ForEloga CROSS APPLY OPENJSON(Json,'lax $.NfdsFadMonitoring')

Output

Id  key value
1   EndUseEaten 1
2   EndUseCommunityMarket   3
3   SpeciesComment  
3   EndUseCommunityMarket   2
4   SpeciesComment  mix reef fis
4   EndUseEaten 31
5   SpeciesComment  10 fish with a total of 18kg
5   EndUseCommunityMarket   0
5   EndUseProvincialMarket  0
5   EndUseUrbanMarket   8
5   EndUseEaten 1
5   EndUseGivenAway 1
6   SpeciesComment  mix reef fis
6   EndUseEaten 18

---

B. COLUMNS: CROSS APPLY WITH WITH

If you know all possible properties then I recommend CROSS APPLY with WITH as shown in Example 3 - Join rows with JSON data stored in table cells using CROSS APPLY in OPENJSON (Transact-SQL) .

 SELECT store.title, location.street, location.lat, location.long FROM store CROSS APPLY OPENJSON(store.jsonCol, 'lax $.location') WITH ( street varchar(500), postcode varchar(500) '$.postcode', lon int '$.geo.longitude', lat int '$.geo.latitude' ) AS location

Answer 3

Try this:

Table Schema:

CREATE TABLE #JsonValue(sp_name VARCHAR(100),catch_ext VARCHAR(1000))

INSERT INTO #JsonValue VALUES ('WAHOO','{"NfdsFadMonitoring":{"EndUseEaten":1}}')
INSERT INTO #JsonValue VALUES ('RUBY SNAPPER','{"NfdsFadMonitoring":{"EndUseCommunityMarket":3}}')
INSERT INTO #JsonValue VALUES ('KAWAKAWA','{"NfdsFadMonitoring":{"SpeciesComment":"","EndUseCommunityMarket":2}}')
INSERT INTO #JsonValue VALUES ('XXXXXXXX','{"NfdsFadMonitoring":{"SpeciesComment":"mix reef fis","EndUseEaten":31}}')
INSERT INTO #JsonValue VALUES ('YYYYYYYY','{"NfdsFadMonitoring":{"SpeciesComment":"10 fish with a total of 18kg","EndUseCommunityMarket":0,"EndUseProvincialMarket":0,"EndUseUrbanMarket":8,"EndUseEaten":1,"EndUseGivenAway":1}}')
INSERT INTO #JsonValue VALUES ('ZZZZZZZZZZ','{"NfdsFadMonitoring":{"SpeciesComment":"mix reef fis","EndUseEaten":18}}')

Query:

SELECT sp_name
    ,ISNULL(MAX(CASE WHEN [Key]='EndUseCommunityMarket' THEN Value END),'')EndUseCommunityMarket
    ,ISNULL(MAX(CASE WHEN [Key]='EndUseProvincialMarket' THEN Value END),'')EndUseProvincialMarket
    ,ISNULL(MAX(CASE WHEN [Key]='EndUseUrbanMarket' THEN Value END),'')EndUseUrbanMarket
    ,ISNULL(MAX(CASE WHEN [Key]='EndUseEaten' THEN Value END),'')EndUseEaten
    ,ISNULL(MAX(CASE WHEN [Key]='EndUseGivenAway' THEN Value END),'')EndUseGivenAway
FROM(
    SELECT sp_name, [key], value
    FROM #JsonValue CROSS APPLY OPENJSON(catch_ext,'$.NfdsFadMonitoring')
    )D
GROUP BY sp_name

Output:

sp_name       EndUseCommunityMarket EndUseProvincialMarket EndUseUrbanMarket EndUseEaten EndUseGivenAway
------------- --------------------- ---------------------- ----------------- ----------- ---------------
KAWAKAWA      2                                                                          
RUBY SNAPPER  3                                                                          
WAHOO                                                                        1           
XXXXXXXX                                                                     31          
YYYYYYYY      0                     0                      8                 1           1
ZZZZZZZZZZ                                                                   18  

Hope this will help you.