在Teradata中为具有一对一关系的表插入策略

Question

In our data model, which is derived from the Teradata industry models, we observe a common pattern, where the superclass and subclass relationships in the logical data model are transformed into one-to-one relationships between the parent and the child table.

I know you can roll-up or roll-down the attributes to end up with a single table but we are not using this option overall. At the end what we have is a model like this:

Where City Id references a Geographical Area Id.

I am struggling with a good strategy to load the records in these tables.

Option 1: I could select the max(Geographical Area Id) and calculate the next Ids for a batch insert and reuse them for the City Table.

Option 2: I could use an Identity column in the Geographical Area Table and retrieve it after I insert every record in order to use it for the City table.

Any other options?

I need to assess the solution in terms of performance, reliability and maintenance.

Any comment will be appreciated.

Kind regards,

Paul

Answer 1

When you say "load the records into these tables", are you talking about a one-time data migration or a function that creates records for new Geographical Area/City?

If you are looking for a surrogate key and are OK with gaps in your ID values, then use an IDENTITY column and specify the NO CYCLE clause, so it doesn't repeat any numbers. Then just pass NULL for the value and let TD handle it.

If you do need sequential IDs, then you can just maintain a separate "NextId" table and use that to generate ID values. This is the most flexible way and would make it easier for you to manage your BATCH operations. It requires more code/maintenance on your part, but is more efficient than doing a MAX() + 1 on your data table to get your next ID value. Here's the basic idea:

BEGIN TRANSACTION

• Get the "next" ID from a lookup table
• Use that value to generate new ID values for your next record(s)
• Create your new records
• Update the "next" ID value in the lookup table and increment it by the # rows newly inserted (you can capture this by storing the value in the ACTIVITY_COUNT value variable directly after executing your INSERT/MERGE statement)
• Make sure to LOCK the lookup table at the beginning of your transaction so it can't be modified until your transaction completes

END TRANSACTION

Here is an example from Postgres, that you can adapt to TD:

CREATE TABLE NextId (
    IDType VARCHAR(50) NOT NULL,
    NextValue INTEGER NOT NULL,
    PRIMARY KEY (IDType)
);

INSERT INTO Users(UserId, UserType)
SELECT 
    COALESCE(
        src.UserId, -- Use UserId if provided (i.e. update existing user)
        ROW_NUMBER() OVER(ORDER BY CASE WHEN src.UserId IS NULL THEN 0 ELSE 1 END ASC) + 
        (id.NextValue - 1) -- Use newly generated UserId (i.e. create new user)
    )
    AS UserIdFinal,
    src.UserType
FROM (
    -- Bulk Upsert (get source rows from JSON parameter)
    SELECT src.FirstName, src.UserId, src.UserType
    FROM JSONB_TO_RECORDSET(pUserDataJSON->'users') AS src(FirstName VARCHAR(100), UserId INTEGER, UserType CHAR(1))
) src
CROSS JOIN ( 
    -- Get next ID value to use
    SELECT NextValue
    FROM NextId 
    WHERE IdType = 'User'
    FOR UPDATE -- Use "Update" row-lock so it is not read by any other queries also using "Update" row-lock
) id
ON CONFLICT(UserId) DO UPDATE SET
UserType = EXCLUDED.UserType;

-- Increment UserId value
UPDATE NextId
SET NextValue = NextValue + COALESCE(NewUserCount,0)
WHERE IdType = 'User'
;   

Just change the locking statement to Teradata syntax (LOCK TABLE NextId FOR WRITE) and add an ACTIVITY_COUNT variable after your INSERT/MERGE to capture the # rows affected. This assumes you're doing all this inside a stored procedure.

Let me know how it goes...