Inserting Values Into Table with Identity Column via Databricks

Question

I've created a table in Databricks that is mapped to a table hosted in an Azure SQL DB. I'm trying to do a very simple insert statement on a small table, but an identity column is giving me issues. This table has the aforementioned identity column and three additional columns.

I first tried something similar to below:

%sql

INSERT INTO tableName (col2, col3, col4)
VALUES (1, 'Test Value', '2018-11-16')

That was giving me a syntax error, so I did some searching and learned that Hive SQL doesn't allow you to specify columns for an INSERT statement. So then I tried something like below as a test:

%sql

INSERT INTO tableName
VALUES (100, 1, 'Test Value', '2018-11-16')

That gives me an error message that I can't insert explicit values into an identity column, but that's what I expected to happen.

If I can't specify the columns for my INSERT statement, how do I avoid issues when I have an identity column? I just want to insert values for the non-identity columns, and I want the ID column to continue incrementing like normal. The above example is extremely watered-down. I will need to do much larger insertions based on SELECT statements eventually, so any solution involving toggling on IDENTITY_INSERT probably isn't feasible.

Answer 1

Below is how we can create a table with an identity column -
CREATE TABLE table_name (column_name1 data_type GENERATED ALWAYS AS IDENTITY, column_name2......)

Below are the two ways how we can insert the data into the table with the Identity column -

• First way -
  INSERT INTO T2 (CHARCOL2) SELECT CHARCOL1 FROM T1;
• Second way -
  INSERT INTO T2 (CHARCOL2,IDENTCOL2) OVERRIDING USER VALUE SELECT * FROM T1;

Links for reference-
Create table - https://docs.databricks.com/sql/language-manual/sql-ref-syntax-ddl-create-table-using.html
Insert into table - https://www.ibm.com/docs/en/db2-for-zos/11?topic=statement-rules-inserting-data-into-identity-column