如何使用 JDBC 接收器连接器将表名转换为大写

Question

I am implementing CDC based replication between Postgres and Oracle databases. For the source connector I am using Debezium's Postgres Connector and for the sink connector Confluent's JDBC Sink Connector.

In the Postgres database, all tables are created in lowercase, but in the Oracle database tables are in uppercase. How can I dynamically set config in the JDBC sink connector, so that I don't need to set a hardcoded uppercase table name in transforms.route.replacement ? Currently if I don't set the destination table name in uppercase manually in the config, double quotes are being added by the connector. Then, I receive the following error:

Caused by: org.apache.kafka.connect.errors.ConnectException: Table "TEST"."dummy_table" is missing and auto-creation is disabled

I have disabled auto-creation, because I don't need it.

Below is an example of my JDBC sink connector config. I have omitted some fields like connection url etc. If I leave the config like this, every time I need to add a new table, I would need to create a new JDBC sink connector.

{
    "connector.class": "io.confluent.connect.jdbc.JdbcSinkConnector",
    "tasks.max": "1",
    "topics": "test.public.dummy_table",
    "transforms": "route,unwrap",
    "transforms.route.type": "org.apache.kafka.connect.transforms.RegexRouter",
    "transforms.route.regex": "([^.]+)\\.([^.]+)\\.([^.]+)",
    "transforms.route.replacement": "TEST.DUMMY_TABLE", // I want to use TEST.$3 here, but apply uppercase. Is it possible?
    "transforms.unwrap.drop.tombstones": "false",
    "transforms.unwrap.type": "io.debezium.transforms.ExtractNewRecordState",
    "value.converter": "io.confluent.connect.avro.AvroConverter",
    "pk.mode": "record_key",
    "key.converter": "io.confluent.connect.avro.AvroConverter",
    "pk.fields": "id",
    "quote.sql.identifiers": "never"
}

Is it possible to achieve table name uppercase transformation using transforms.route.replacement or to tell the sink connector not to add double quotes? If not, could you please suggest how to implement it? Thank you in advance.

Answer 1

I know almost nothing about what you described.

However : as far as Oracle is concerned, table names are stored in uppercase in data dictionary, but Oracle lets you access them using any letter case you want (upper, lower, mixed):

SQL> select count(*) from emp union all       --> lower
  2  select count(*) from EMP union all       --> upper
  3  select count(*) from EmP;                --> mixed

  COUNT(*)
----------
        14
        14
        14

SQL>

That stands as long as you didn't use double quotes while creating the table; in that case, you must use the same letter case and enclose table name into double quotes every time you access it:

SQL> create table "teST" (id number);

Table created.

SQL> select * from test;
select * from test
              *
ERROR at line 1:
ORA-00942: table or view does not exist


SQL> select * from TEST;
select * from TEST
              *
ERROR at line 1:
ORA-00942: table or view does not exist


SQL> select * from teST;
select * from teST
              *
ERROR at line 1:
ORA-00942: table or view does not exist


SQL> select * from "teST";

no rows selected

SQL>

So: if you weren't that unfortunate to use double quotes, perhaps you don't have to bother (as my first example shows). Just don't use double quotes while referencing the table and you're fine.