MySQL能源系统建模数据库EAV设计

Question

We are designing a database schema for an open source energy system optimization project.

We are using an EAV model because we want it to be flexible and allow problem definition through data. Ie we want to be able to create new object classes and data parameters associated with them on the fly without having to change the database structure.

The model will be implemented in Julia JuMP and we will query the data via sql. There will be an interface with Julia to access the data via SQL and a GUI in PyQT to view, edit, filter the data.

I have given the SQL for the current schema below.

The idea is pretty simple... and object_class table to define types of object. An object table to define instances of objects and a parameter_definition table to define parameters that can be associated with objects of each type and a parameter table to define actual values for parameters associated with object instances. Lets forget about relationships for the moment to keep the discussion simple. Also I have omitted id fields which we will include.

The parameter table is a little nuanced because our optimization models are chronological models that run over time - so it is very common to have time varying data. To handle this, we can give a "time_pattern" to a data value. Also our models will have scenarios. In the parameters table we can set this value to indicate the parameter value applies to a particular scenario. A NULL in the scenario field indicates a "base case" data item.

Now to the crunch... a member of the team has had the idea to replace the parameters table with a JSON field in the objects table. He is convinced this is better because all the data for an object is contained in this and it avoids the nulls that we will have in the parameters table when there is no time pattern or scenario specified. However, I have a number of problems with this: 1. we lose the referential integrity between the parameter table and the parameter definition table. I know you can link on JSON fields, but isn't it inefficient? 2. There may be some merit to the idea if we were to assume that each object could have a list of parameters which are static (ie no time_pattern or scenario defined. However, when these are defined, we need to have nested JSONs in the JSON field of the objects table. Can we join on this? 3. From what I've read, it is not good practice to bury any fields which you might want to query on inside a JSON field. Is this correct? 4. Is it even possible to join on fields inside a nested JSONs and if you can is it inefficient? 5. Even if you can query and join on fields buried inside a nested JSON, won't the sql be complicated for no real gain?

To me, on the surface, the only advantage is you get rid of one table which may simplify some calls. However you are just burying the data of that table inside a JSON field in another table

Am I missing something? Are there benefits to this proposal that I'm not getting?

Since the data will be queried en masse to Julia, this element probably doesnt care, JSON or no... but the PyQT Gui where there will be filtering of data etc... the code for this could be greatly complicated?

CREATE TABLE IF NOT EXISTS `db1303628_spine`.`class_category` (
  `name` VARCHAR(255) NOT NULL,
  `description` VARCHAR(255) NULL DEFAULT NULL,
  PRIMARY KEY (`name`))
ENGINE = InnoDB
DEFAULT CHARACTER SET = latin1;


-- -----------------------------------------------------
-- Table `db1303628_spine`.`object`
-- -----------------------------------------------------
DROP TABLE IF EXISTS `db1303628_spine`.`object` ;

CREATE TABLE IF NOT EXISTS `db1303628_spine`.`object` (
  `class_name` VARCHAR(155) NOT NULL,
  `name` VARCHAR(255) NOT NULL,
  `description` VARCHAR(255) NULL DEFAULT NULL,
  `category_name` VARCHAR(255) NULL DEFAULT NULL,
  PRIMARY KEY (`class_name`, `name`))
ENGINE = InnoDB
DEFAULT CHARACTER SET = latin1;


-- -----------------------------------------------------
-- Table `db1303628_spine`.`object_category`
-- -----------------------------------------------------
DROP TABLE IF EXISTS `db1303628_spine`.`object_category` ;

CREATE TABLE IF NOT EXISTS `db1303628_spine`.`object_category` (
  `object_class_name` VARCHAR(255) NOT NULL,
  `name` VARCHAR(255) NOT NULL,
  `description` VARCHAR(255) NULL DEFAULT NULL,
  PRIMARY KEY (`object_class_name`, `name`))
ENGINE = InnoDB
DEFAULT CHARACTER SET = latin1;


-- -----------------------------------------------------
-- Table `db1303628_spine`.`object_class`
-- -----------------------------------------------------
DROP TABLE IF EXISTS `db1303628_spine`.`object_class` ;

CREATE TABLE IF NOT EXISTS `db1303628_spine`.`object_class` (
  `name` VARCHAR(255) NOT NULL,
  `description` VARCHAR(255) NULL DEFAULT NULL,
  `category_name` VARCHAR(155) NULL DEFAULT NULL,
  `display_order` INT(11) NULL DEFAULT '99',
  `display_icon` INT(11) NULL DEFAULT NULL,
  `hidden` TINYINT(1) NULL DEFAULT '0',
  PRIMARY KEY (`name`))
ENGINE = InnoDB
DEFAULT CHARACTER SET = latin1;


-- -----------------------------------------------------
-- Table `db1303628_spine`.`parameter`
-- -----------------------------------------------------
DROP TABLE IF EXISTS `db1303628_spine`.`parameter` ;

CREATE TABLE IF NOT EXISTS `db1303628_spine`.`parameter` (
  `object_name` VARCHAR(155) NOT NULL,
  `name` VARCHAR(155) NOT NULL,
  `index` TINYINT(2) NULL DEFAULT '1',
  `value` VARCHAR(155) NULL DEFAULT NULL,
  `expression` VARCHAR(255) NULL DEFAULT NULL,
  `time_pattern` VARCHAR(155) NULL DEFAULT NULL,
  `deterministic_timeseries_id` VARCHAR(155) NULL DEFAULT NULL,
  `stochastic_timeseries_id` VARCHAR(155) NULL DEFAULT NULL,
  `stochasticmodel_id` VARCHAR(155) NULL DEFAULT NULL,
  `comment` VARCHAR(255) NULL DEFAULT NULL,
  PRIMARY KEY (`object_name`, `name`))
ENGINE = InnoDB
DEFAULT CHARACTER SET = latin1;


-- -----------------------------------------------------
-- Table `db1303628_spine`.`parameter_definition`
-- -----------------------------------------------------
DROP TABLE IF EXISTS `db1303628_spine`.`parameter_definition` ;

CREATE TABLE IF NOT EXISTS `db1303628_spine`.`parameter_definition` (
  `name` VARCHAR(155) NOT NULL,
  `description` VARCHAR(155) NULL DEFAULT NULL,
  `data_type` VARCHAR(155) NULL DEFAULT 'NUMERIC',
  `object_type` VARCHAR(155) NOT NULL,
  `object_class_name` VARCHAR(155) NOT NULL,
  `CanHaveTimeSeries` TINYINT(1) NULL DEFAULT '0',
  `CanHaveTimePattern` TINYINT(1) NULL DEFAULT '1',
  `CanBeStochastic` TINYINT(1) NULL DEFAULT '0',
  `dafault_value` VARCHAR(155) NULL DEFAULT '0',
  `is_madatory` TINYINT(1) NULL DEFAULT '0',
  `precision` TINYINT(2) NULL DEFAULT '2',
  `minimum_value` DOUBLE(10,4) NULL DEFAULT NULL,
  `maximum_value` DOUBLE(10,4) NULL DEFAULT NULL,
  PRIMARY KEY (`object_class_name`, `name`))
ENGINE = InnoDB
DEFAULT CHARACTER SET = latin1;


-- -----------------------------------------------------
-- Table `db1303628_spine`.`relationship`
-- -----------------------------------------------------
DROP TABLE IF EXISTS `db1303628_spine`.`relationship` ;

CREATE TABLE IF NOT EXISTS `db1303628_spine`.`relationship` (
  `class_name` VARCHAR(155) NOT NULL,
  `name` VARCHAR(155) NOT NULL,
  `parent_object_name` VARCHAR(155) NOT NULL,
  `child_object_name` VARCHAR(155) NOT NULL,
  PRIMARY KEY (`class_name`, `name`))
ENGINE = InnoDB
DEFAULT CHARACTER SET = latin1;


-- -----------------------------------------------------
-- Table `db1303628_spine`.`relationship_class`
-- -----------------------------------------------------
DROP TABLE IF EXISTS `db1303628_spine`.`relationship_class` ;

CREATE TABLE IF NOT EXISTS `db1303628_spine`.`relationship_class` (
  `name` VARCHAR(155) NOT NULL,
  `parent_class_name` VARCHAR(155) NOT NULL,
  `child_class_name` VARCHAR(155) NOT NULL,
  `inheritance` VARCHAR(155) NULL DEFAULT NULL,
  `hidden` TINYINT(1) NULL DEFAULT '0',
  `type` INT(11) NULL DEFAULT NULL,
  PRIMARY KEY (`name`))
ENGINE = InnoDB
DEFAULT CHARACTER SET = latin1;

Answer 1

You can use the function

common_schema.extract_json_value(json_data,'title')

To query JSON in columns, but as I said above it's Inefficient, and precludes the use of indexes / all the things that make use SQL powerful. Sounds like he's just wants to be lazy regarding his queries. There's no reason at all to store information that should be laid out as tables / columns in JSON format.