How to execute multiple spark-submit commands using python script

Question

How to execute multiple spark-submit commands using a python script.

Ex:

spark-submit --class com.demo.DemoApplication1 --master local[2] Demo1.jar

spark-submit --class com.demo.DemoApplication3 --master local[2] Demo2.jar

spark-submit --class com.demo.DemoApplication3 --master local[2] Demo3.jar

Answer 1

you can use os.system() to run your commands using python like this:

import os
os.system('command')

so you can probably just use it like this:

import os
os.system('spark-submit --class com.demo.DemoApplication1 --master local[2] Demo1.jar')
os.system('spark-submit --class com.demo.DemoApplication1 --master local[2] Demo1.jar')