Google Dataflow: set a DefaultUncaughtExceptionHandler

Question

Is it possible to set a DefaultUncaughtExceptionHandler for every worker and thread in Dataflow ?

Something like Thread.setDefaultUncaughtExceptionHandler .

Answer 1

No, I don't believe so. The Dataflow model for expressing user code operates at a higher level of abstraction than individual workers and threads. Can you expand a bit more on why you'd like to do this globally?

The Dataflow Service already retries all uncaught exceptions in your user code. Elements are processed in groups called bundles -- if any element in a bundle causes an exception to be thrown, the entire bundle is retried. In batch mode, failing bundles are retried until a single bundle has failed 4 times, at which point the job is failed. In streaming mode, failing bundles are retried indefinitely, although you can use the update feature to update your code to better handle the issue.

All of these exceptions will appear in both the job logs in Dataflow UI and in your Cloud Logging worker logs. See more info here .