How can I create a table if it doesn't exist, else alter the table schema to match in HiveQL

Question

I have an automated workflow that essentially builds out a table from a bunch of other existing tables. We get asks to add new fields to that and other tables relatively frequently. Currently I have to manually go in and alter the existing tables to add the columns, but I want to be able to have this automated as it's very cumbersome and error prone, especially when we need to do this against production tables.

All of our workflows are meant to be idempotent so they typically start with some task that runs a CREATE IF NOT EXISTS statement. This also generates downstream documentation for our table schema. Generally people add the new columns to that statement and the subsequent statements used to fill the table with data but there is no real way to alter the table with the new columns. Ideally I would like essentially just want the columns mentioned in the create to be used in a subsequent alter clause but I'm not sure if that's even possible.

An example ideal workflow:

1. Initial table creation CREATE TABLE IF NOT EXISTS foo (id BIGINT, bar STRING)
2. New field added to create CREATE TABLE IF NOT EXISTS foo (id BIGINT, bar STRING, baz STRING)
3. When the workflow gets scheduled again (automated cron-based cadence with manual backfills), the CREATE statement instead implicitly alters the table to match the schema defined in the latest CREATE statement.

Is there a way to do this natively in HiveQL?

Answer 1

For modifying the initial/PARENT table, I'm afraid there's no other way but to specify the new columns manually.

If other tables depend on the parent table, you can utilize select * to include the new columns in the parent table automatically.

create other_table as
SELECT * from parent_table;