Error BINLOG_FORMAT = STATEMENT when writing from spark to MySql

Question

I'm running simple code to write dataframe to the MySql db

  val mydf = //.. create some dataframe
   mydf
  .write
  .format("jdbc")
  .option("driver", "com.mysql.jdbc.Driver")
  .option("url",url)
  .option("dbtable",table)
  .mode(SaveMode.Append)
  .save();

And as a result getting following error

Caused by: java.sql.SQLException: Cannot execute statement: impossible to write to binary log since BINLOG_FORMAT = STATEMENT and at least one table uses a storage engine limited to row-based logging. InnoDB is limited to row-logging when transaction isolation level is READ COMMITTED or READ UNCOMMITTED.
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:129)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)

Tried to add additional property, but it seems spark is ignoring that.option(BINLOG_FORMAT,"MIXED")

Is there an option to set some property on spark level and not change the database definition?

Answer 1

There is no option specified in spark documentation to change BINLOG_FORMAT . You can check the possible optionshere .

From the exception message:

InnoDB is limited to row-logging when transaction isolation level is READ COMMITTED or READ UNCOMMITTED.

READ_UNCOMMITTED is the default isolation in spark. You can try changing this option.