shell: rebooting bunch of servers all at the same time

Question

I want to be able to reboot a bunch of servers all at the same time (in a bash script).
Currently, what I do is something like that:

function reboot_servers() {
    echo "Rebooting servers..."
    for server in "${servers[@]}"
    do
        sshpass -p 'password' ssh -o StrictHostKeyChecking=no root@$server 'reboot'
    done
}

( servers is an array of 4 servers, sometimes 8, and in the future probably more)

Now, I am aware that in theory I cannot really have them rebooted all at the exact same time, but I'd like it to be as simultaneously as possible, and the above solution is far from optimal for me.
In my current script, if every iteration takes (say) few hundreds milliseconds in average (the ssh login sometimes lags and is unpredictable), the time passed from when the first server launches the reboot command until the last one does could amount to seconds, which is completely ineffective.

I should also mention that the clocks in all the servers are synced, and also to give you some context, the above function is being run over and over again in something similar to this;

function main() {
    iteration=0

    while true
    do
        echo "------> Iteration $((++iteration)) <------"
        wait_random_time
        reboot_servers
        wait_for_servers

        if bug_reproduced
        then
            echo "Bug was reproduced."
            exit 0
        else
            echo "No reproduction, trying again..."
        fi
    done
}

I read a little bit about the at command, but I'm not sure how to use it for my benefit here.

Answer 1

我建议使用parallel-ssh 。

Answer 2

I end up using pdsh , which gave quite impressive results...

$> pdsh -l root -w server0[0-3] date "+%T.%3N"
server00: 12:29:45.845
server01: 12:29:45.830
server02: 12:29:45.870
server03: 12:29:45.893