Execution of several processes of consumption of web services at the same time on Ruby on Rails

Question

does anyone know how to run several processes at the same time? I explain the context:

I am currently connecting to 3 web services, the results will be stored in the database, and then I show them on the screen.

It would be something like this:

Task1 = Get to WebService method, store results, display on screen

Task2 = Method Obtain a web service, store results, display on screen

Task3 = Method Obtain a web service, store results, display on screen

The problem is that each process takes 2 to 3 seconds, and multiplied by the 3 web services, it takes about 9 seconds.

I would like to find a way to run all 3 processes at the same time, to minimize the execution time and the result is displayed faster on the screen. Also, in the future I will need to connect to more web services.

I use the web services with the RestClient gem

Answer 1

For something that's not CPU intensive like this, you can just use the Thread class in ruby for this. Using that approach the "display on screen" will all happen at once, when the response comes back.

Also, very feasible to push these tasks off into an ActiveJob queue, and update the client when they finish using ActionCable.

Answer 2

You can use the Typhoeus gem which is a HTTP client that runs requests in parallel:

require 'typhoeus'
hydra = Typhoeus::Hydra.new
hydra.queue(Typhoeus::Request.new("www.example.com/task_1", followlocation: true))
hydra.queue(Typhoeus::Request.new("www.example.com/task_2", followlocation: true))
hydra.queue(Typhoeus::Request.new("www.example.com/task_3", followlocation: true))
hydra.run

But ultimately doing this in one big honking syncronous action is probably not going to be a very sustainable solution. hydra.run is still blocking and will only complete once the slowest request finishes.

You'll want to look into using AJAX to do this in multiple requests (from the client to your rails app) so that you can give quick user feedback and then update the page.

Or you can use ActionCable which lets you notify clients via websocks of updates from the server.