Passing an array to a slave python script

Question

I am quite new to python.
I learned how to pass arguments as string or floats to a slave script.
As an instance, here it is the main script:

#main script (mainscript.py)
import subprocess, sys
import numpy as np

x = np.linspace(0.5,3.2,10)

for i in range(x.size) :
    subprocess.call([sys.executable,'slavescript.py',
                     '%s' %sys.argv[1], '%s' %sys.argv[2], '%s' %xpnt[i]])

And here the slave script:

#slave script (slavescript.py)

import sys

sys.argv[1] = str(sys.argv[1])
sys.argv[2] = int(sys.argv[2])
sys.argv[3] = float(sys.argv[3])

...
...

Now, if in python I run the following command:

run mainscript.py N 5

Then slavescript.py starts using N as a string, 5 as an integer and the third argument is converted to a float. slavescript.py is run m times, where m is the size of the array x.

I would like to pass the whole content of the array x at once, ie without the for loop in the main script. I think that the subprocess.call can have only strings among its arguments... I hope someone may have time to help me or give me some hints.

Thanks for the attention. Noctu

Answer 1

The only reason to use a separate process is if you need parallel processing. If you do need that, then if you're managing lots of workers, use something like celery.

If you do find it appropriate to roll your own, you need to reduce what you want to send to a textual representation. I suggest using the json module.

If you don't need a separate process, just import the other python module, and access its functionality directly in code (it should already by wrapped up in functions).