Use of expect to run scripts on remote machine

Question

I am working on a project that requires some assistance.

I have automated most of the information required for the completion of this project but the only thing that is lagging is the running of local shell scripts on the remote machine. As we are aware that no Linux command is recognized by the script that uses the 'expect' library. Herein we have two use cases that I have tried:

1) Running the desired list of commands on the remote server using only one expect script which has both the script execution as well as pushing of output using scp to the local machine, here is a snippet of this code:

`chmod 777 localscript.sh   
cat > script1.sh <<- "ALL"`  
`#!/usr/bin/expect
set password [lindex $argv 0];  
set ipaddress [lindex $argv 1];  
set timevalue [lindex $argv 2];  
set timeout $timevalue  
spawn /usr/bin/ssh username@$ipaddress /bin/bash < ./localscript.sh  
expect "assword:"  
send "$password\r"  
set timeout $timevalue  
spawn /usr/bin/scp username@$2:"/path/from/source/*" /path/to/destination/folder/  
expect "assword:"  
send "$password\r"  
interact  
ALL  
chmod 777 script1.sh  
./script1.sh $password $2 $timevalue`  

2) Running the desired list of commands on the remote server in a separate expect script and using scp to get files in a different script:

`cat > script1.sh <<- "ALL" ` 
`#!/usr/bin/expect
set password [lindex $argv 0];  
set ipaddress [lindex $argv 1];  
set timevalue [lindex $argv 2];  
set timeout $timevalue   
spawn /usr/bin/ssh username@$ipaddress /bin/bash < ./localscript.sh  
expect "assword:"  
send "$password\r"  
interact  
ALL  
cat > script2.sh <<- "ALL2"`  
`#!/usr/bin/expect
set password [lindex $argv 0];  
set ipaddress [lindex $argv 1];  
set timevalue [lindex $argv 2];  
set timeout $timevalue   
spawn /usr/bin/scp username@ipaddress:"/path/from/source/*" /path/to/destination/folder/  
expect "assword:"  
send "$password\r"  
interact  
ALL2  
chmod 777 localscript.sh script1.sh script2.sh   
./script1.sh  $password $2 $timevalue  
sleep 5  
./script2.sh  $password $2 $timevalue`

I believe the above codes should both be valid in their own respect however, the output for the same seem to be quite unexpected:

1) Both the commands ssh and scp are being executed almost simultaneously after password is entered hence, it is not giving localscript enough time to do its job, here's the output I see:

    spawn /usr/bin/ssh username@1.2.3.4 /bin/bash < ./localscript.sh  
    Warning private system unauthorized users will be prosecuted.  
    username@1.2.3.4's password: spawn /usr/bin/scp 
    username@1.2.3.4:"/home/some/file/*" /another/file/  
    Warning private system unauthorized users will be prosecuted.  
    username@1.2.3.4's password:  
    scp: /home/some/file/*: No such file or directory  

Please note: This functionality is working fine without the involvement of expect

2) Here we are executing ssh and scp separately, however, it seems like it is unable to recognize that the file localscript.sh exists:

    spawn /usr/bin/ssh username@1.2.3.4 /bin/bash < ./localscript.sh  
    Warning private system unauthorized users will be prosecuted.  
    username@1.2.3.4's password:  
    bash: localscript.sh: No such file or directory  
    Warning private system unauthorized users will be prosecuted.  
    username@1.2.3.4's password:  
    scp: /home/some/file/*: No such file or directory  

Any feedback on the same would be appreciated, I think the first approach might be a feasible solution, except the fact that spawn is too fast and none of the 'sleep' or 'after' commands are helping/working. I think the second approach is also valid however it seems like there is a different way of running a local script on a remote server than the usual way we do on Linux when using 'expect'.

Sorry for so much elaboration, I am hoping to be out of my misery soon :)

Answer 1

Indeed the timeout you are setting is not working as you expect it to. Both scripts are spawned, and the expect "assword:" after each spawn is actually catching and reacting to the same password prompt.

expect is actually more sophisticated than a cursory glance would lead you to believe. Each spawn should return a PID, which you can use with your expect to look for output from a specific process.

expect can also be broken down into multiple parts, and have the ability to define subroutines. Here are some more advanced use examples https://wiki.tcl-lang.org/10045

In this specific case I would suggest waiting for the scp to complete before spawning the next process.

expect {
    "assword:" {
        send "$password\r"
        exp_continue # Keep expecting
    }
    eof {
        puts -nonewline "$expect_out(buffer)"
        # eof so the process should be done
        # It is safe to execute the next spawn
        # without exp_continue this expect will
        # break and continue to the next line
    }
}