What's the proper way to run Ruby scripts in parallel?

Question

Current I have the following code to run some ruby scripts, within a ruby script:

def run(base_directory, run_count)
  working_directory = base_directory.gsub("\n","")
  for i in 1..run_count
    system("ruby " + working_directory + i.to_s + "\\" + "main.rb " +   working_directory + i.to_s + "\\")
  end
end

However this runs the scripts in a sequence, but I need them to run in parallel. Where I have 10 scripts to run, and I want to run 5 at a time until I reach the number of scripts that needs run. Is there a simple way to accomplish this?

Answer 1

Just discovered this gem parallel . You probably will have to run it like this:

results = Parallel.map(run_count.downto(1).to_a, :in_processes=>run_count){|i| system("ruby " + working_directory + i.to_s + "\\" + "main.rb " +   working_directory + i.to_s + "\\") }

Answer 2

Not sure what is the proper way, but I usually just run a one-liner that executes something from the command line:

Say you have a file called create_file_with_input.rb that does whatever, like one line with: File.open("file_#{ARGV[0]}.txt", 'a') .

you can do:

run_count = 5

%x(for i in `seq 0 #{run_count-1}`; do (ruby create_file_with_input.rb $i &); done)

That would create 5 files like (file_0.txt, file_1.txt, ..., file_4.txt etc.). Tweak it to fit your code from above and BOOM. You have multiple scripts running.

You could also potentially use fork , although I don't have much personal experience with it. See this question (or use The Google) for forking child processes. This way also provides the benefit of getting the PID of child processes.