airflow task INFO - Task exited with return code -9

Question

i am using big pandas dataframe and while task run for 30-40 min. in the middle of heavy computation i got this message:-

-bash-4.2$ tail -f  /home/app/airflow/logs/report_dag_for_new_metrics/first_pass_yield_metrics/2020-05-21T15\:41\:46.689070+00\:00/2.log 
[2020-05-26 05:05:17,427] {logging_mixin.py:95} INFO - sort columns in data frame for merge ['product_id', 'type', 'created_date']
[2020-05-26 05:05:24,153] {logging_mixin.py:95} INFO -        product_id type              created_date
0  00000346333824    I                2020-04-17
1  00000346333824    U  2020-04-25T21:53:58.551Z
2  00000346333824    U  2020-04-25T22:07:03.581Z
3  00000346333824    U  2020-05-20T04:32:16.581Z
4  00000346357646    I                2020-04-17
[2020-05-26 05:05:24,173] {logging_mixin.py:95} INFO - (58582, 1206)
[2020-05-26 05:05:24,194] {logging_mixin.py:95} INFO - 23618
[2020-05-26 05:17:17,260] {logging_mixin.py:95} INFO - [2020-05-26 05:17:17,259] {local_task_job.py:105} INFO - Task exited with return code -9

any help, airflow community also doesn't say much about code 9. saw issues related to code 6.

Answer 1

-9 indicates the system killed the process, most likely as the DataFrame consumed too much memory. Best advice is to rewrite your script to use less memory or to scale up the machine running Airflow (if possible).