如何使用简单的查询将 JSON 文件导入 MySQL 数据库，而不实际将其转换为任何其他文件格式，如 CSV 等？

Question

I tried to import a JSON file which looks like:

[ 
{ 
"executionDateTime":"2017-07-07 15:21:15",
"A":1,
"B":1
},
{ 
"executionDateTime":"2017-07-07 15:21:15",
"A":2,
"B":2
},
{ 
"executionDateTime":"2017-07-07 15:21:15",
"A":3,
"B":3
},
{ 
"executionDateTime":"2017-07-07 15:21:15",
"A":4,
"B":4
}]

I want to import the above file into a mySQL DB, and i want my table to look somewhat like this:

 executionDateTime           A               B

2017-07-07 15:21:15          1               1
2017-07-07 15:21:15          2               2
2017-07-07 15:21:15          3               3
2017-07-07 15:21:15          4               4

I have tried the below query to do so (which would work out for me if the file format is CSV), but it didn't work.

LOAD DATA local INFILE '<path>/my_file.json' 
INTO TABLE database_name.my_table FIELDS TERMINATED BY ',' 
ENCLOSED BY '"'LINES TERMINATED BY '\n' IGNORE 1 ROWS;

The above query gave me a table which looked like this:

(I KNOW, mySQL HAS AN INBUILT FUNCTION TO IMPORT A FILE, BUT I DO NOT WANT TO USE THAT FUNCTION, A QUERY IS WHAT I AM LOOKING FOR... :) )

Please suggest, if anyone had to face such a problem and got a solution. Please don't mark this question as a duplicate without giving a SOLUTION THAT ACTUALLY WORKS.

Many thanks.

Answer 1

Reference: https://dev.mysql.com/doc/refman/5.7/en/json-search-functions.html

I know this is an older thread, but MySQL 5.7 now has a JSON type where you can import JSON into a field, then you can use calculated fields to split the json into separate fields. Here's rough code (not tested):

Create a JSON Test Table:

CREATE TABLE IF NOT EXISTS jsontest(
     rowid INT AUTO_INCREMENT NOT NULL UNIQUE,
     jsondata json,
     `executionDateTime` TIMESTAMP,
     `A` BIGINT UNSIGNED,
     `B` BIGINT UNSIGNED,
     );

Import your JSON into JSON field:

LOAD DATA LOCAL INFILE '/path/to/testfile.json' into table jsontest(jsondata);

Split your Data Up (this could be combined into a single command)

UPDATE jsontest set executionDateTime=(jsondata->>'$.executionDateTime');
UPDATE jsontest set A=(jsondata->>'$.A');
UPDATE jsontest set B=(jsondata->>'$.B');

If you don't want to have extra fields, you can query the jsondata field like this:

SELECT jsondata->>"$.executionDateTime" AS executionDateTime,
       jsondata->>"$.A" AS A,
       jsondata->>"$.B" AS B;

Answer 2

I hope that in the near future there is a native functionality from MySQL.

An option (not simple query) is something like the following script (adapt as needed). Depending on the number of items may have performance problems.

File: /path/to/file/loadsetProfile.json :

[
  {
    "executionDateTime":"2017-07-07 15:21:15",
    "A":1,
    "B":1
  },
  {
    "executionDateTime":"2017-07-07 15:21:15",
    "A":2,
    "B":2
  },
  {
    "executionDateTime":"2017-07-07 15:21:15",
    "A":3,
    "B":3
  },
  {
    "executionDateTime":"2017-07-07 15:21:15",
    "A":4,
    "B":4
  }
]

MySQL Command-Line :

mysql> SELECT VERSION();
+-----------+
| VERSION() |
+-----------+
| 5.7.18    |
+-----------+
1 row in set (0.00 sec)

mysql> DROP PROCEDURE IF EXISTS `import_from_json`;
Query OK, 0 rows affected (0.00 sec)

mysql> DROP FUNCTION IF EXISTS `uuid_to_bin`;
Query OK, 0 rows affected (0.00 sec)

mysql> DROP TABLE IF EXISTS `temp_my_table`, `my_table`;
Query OK, 0 rows affected (0.00 sec)

mysql> CREATE TABLE IF NOT EXISTS `temp_my_table` (
    ->   `id` BINARY(16) NOT NULL PRIMARY KEY,
    ->   `content` JSON NOT NULL
    -> );
Query OK, 0 rows affected (0.00 sec)

mysql> CREATE TABLE IF NOT EXISTS `my_table` (
    ->   `executionDateTime` TIMESTAMP,
    ->   `A` BIGINT UNSIGNED,
    ->   `B` BIGINT UNSIGNED
    -> );
Query OK, 0 rows affected (0.00 sec)

mysql> CREATE FUNCTION `uuid_to_bin` (`id` VARCHAR(36))
    -> RETURNS BINARY(16)
    -> DETERMINISTIC
    ->   RETURN UNHEX(REPLACE(`id`, '-', ''));
Query OK, 0 rows affected (0.00 sec)

mysql> DELIMITER //

mysql> CREATE PROCEDURE `import_from_json`(`_id` VARCHAR(36))
    -> BEGIN
    ->   DECLARE `_id_current_json` BINARY(16) DEFAULT `uuid_to_bin`(`_id`);
    ->   DECLARE `_items_length`,
    ->           `_current_item` BIGINT UNSIGNED DEFAULT 0;
    ->   DECLARE `_content` JSON DEFAULT (SELECT `content`
    ->                                    FROM `temp_my_table`
    ->                                    WHERE `id` = `_id_current_json`);
    -> 
    ->   IF JSON_VALID(`_content`) THEN
    ->     SET `_items_length` := JSON_LENGTH(`_content`),
    ->         @`insert_import_from_json` := NULL;
    ->     WHILE `_current_item` < `_items_length` DO
    ->       SET @`insert_import_from_json` := CONCAT('
    '>         INSERT INTO `my_table` (
    '>            `executionDateTime`,
    '>            `A`,
    '>            `B`
    '>         )
    '>         SELECT
    '>           `content` ->> \'$[', `_current_item`, '].executionDateTime\',
    '>           `content` ->> \'$[', `_current_item`, '].A\',
    '>           `content` ->> \'$[', `_current_item`, '].B\'
    '>         FROM `temp_my_table`
    '>         WHERE `id` = \'', `_id_current_json`, '\'
    '>       ');
    ->       PREPARE `stmt` FROM @`insert_import_from_json`;
    ->       EXECUTE `stmt`;
    ->       SET `_current_item` := `_current_item` + 1;
    ->     END WHILE;
    ->
    ->     IF `_current_item` > 0 THEN
    ->       SET @`insert_import_from_json` := NULL;
    ->       DEALLOCATE PREPARE `stmt`;
    ->     END IF;
    ->   END IF;
    -> END//
Query OK, 0 rows affected (0.00 sec)

mysql> DELIMITER ;

mysql> SET @`UUID` := UUID();
Query OK, 0 rows affected (0.00 sec)

mysql> LOAD DATA LOCAL INFILE '/path/to/file/loadsetProfile.json' 
    -> INTO TABLE `temp_my_table`
    -> LINES TERMINATED BY '\r'
    -> (`content`)
    -> SET `id` = `uuid_to_bin`(@`UUID`);
Query OK, 1 row affected (0.00 sec)
Records: 1  Deleted: 0  Skipped: 0  Warnings: 0

mysql> CALL `import_from_json`(@`UUID`);
Query OK, 0 rows affected (0.00 sec)

mysql> SELECT
    ->   `executionDateTime`,
    ->   `A`,
    ->   `B`
    -> FROM
    ->   `my_table`;
+---------------------+------+------+
| executionDateTime   | A    | B    |
+---------------------+------+------+
| 2017-07-07 15:21:15 |    1 |    1 |
| 2017-07-07 15:21:15 |    2 |    2 |
| 2017-07-07 15:21:15 |    3 |    3 |
| 2017-07-07 15:21:15 |    4 |    4 |
+---------------------+------+------+
4 rows in set (0.01 sec)

Answer 3

I am using this web page which transform json file in sql file, after that, I execute script in toad mysql and works.

URI of web I am using is converter web