Gogle Cloud Data Fusion Post GresQL read from follower database

Question

So, like the title I am trying to read from a PostGres follower database using Google Cloud Data Fusion so I can pipeline to big query. I keep running into the issue.

01/04/2022 0:28:52
INFO
Pipeline '2bf2d105-6d1f-11ec-9f9d-7ee8fe64661a' is started by user 'root' with arguments {isolationLevel=READ_COMMITTED, logical.start.time=1641274129122, system.profile.name=SYSTEM:dataproc, default_transaction_isolation=repeatable read, isolationlevel=READ_COMMITTED}
01/04/2022 0:28:52
INFO
Pipeline '2bf2d105-6d1f-11ec-9f9d-7ee8fe64661a' running
01/04/2022 0:29:15
ERROR
Spark program 'phase-1' failed with error: ERROR: cannot use serializable mode in a hot standby Detail: "default_transaction_isolation" is set to "serializable". Hint: You can use "SET default_transaction_isolation = 'repeatable read'" to change the default.. Please check the system logs for more details.
01/04/2022 0:29:15
ERROR
Pipeline '2bf2d105-6d1f-11ec-9f9d-7ee8fe64661a' failed.

I understand what it is telling me and why, but I can't for the life of me get Cloud Data Fusion to run postgresql-9.4.1211.jar with the argument for transaction isolation, which seems to be --isolationLevel=READ_COMMITTED according to https://spark.apache.org/docs/2.4.1/sql-data-sources-jdbc.html

Can someone help me understand what I am misunderstanding?

Answer 1

I had this same issue and I could solve it as stated by "Prajna Rai T" but using "Database pluging" instead of "BigQuery plugin", which doesn't have the "Transaction Isolation Level" property.