Shell script run by Jenkins creates non-terminating process

Question

I'm trying to use Jenkins to create a special git repository. I created a free-style project that just executes a shell script. When I execute this script by hand, without Jenkins, it works just fine. From Jenkins, however, it behaves quite differently.

# this will remove all subtrees
git log | grep git-subtree-dir | tr -d ' ' | cut -d ":" -f2 | sort | uniq | xargs -I {} bash -c 'if [ -d $(git rev-parse --show-toplevel)/{} ] ; then rm -rf {}; fi'

rm -rf .git

If this part is executed by Jenkins, in console output I see this kind of errors:

rm: cannot remove '.git/objects/pack/pack-022eb85d38a41e66ad3f43a5f28809a5a3ee4a0f.pack': Device or resource busy
rm: cannot remove '.git/objects/pack/pack-05630eb059838f149ad30483bd48d37f9a629c70.pack': Device or resource busy
rm: cannot remove '.git/objects/pack/pack-26f510b5a2d15ba9372cf0a89628d743811e3bb2.pack': Device or resource busy
rm: cannot remove '.git/objects/pack/pack-33d276d82226c201eedd419e5fd24b6b906d4c03.pack': Device or resource busy

I modified this part of the script like this:

while true
do
    if rm -rf .git ; then
        break
    else
        continue
    fi
done

But this doesn't help. In the task manager I see a git process that just doesn't terminate. I conjured said script by a lot of googling and I do not understand very good what's going on.

Jenkins runs on Windows Server 2012 behind IIS; shell scripts are executed by bash shipped with git for Windows.

Answer 1

1/ Ensure your path is correct, and no quote/double quote escaping occurs in the process of jenkins job starting.

2/ Your command line is a bit too handy to be correctly and safely evaluated. Put your commands in a regular script, starting with #!/bin/bash instead of thru the command line.

xargs -I {} bash -c 'if [ -d $(git rev-parse --show-toplevel)/{} ] ; then rm -rf {}; fi'

becames

xargs -I {} /path/myscript.sh {}

with

#!/bin/bash

rev-parse="$(git rev-parse --show-toplevel)"
wait

if [ -d ${rev-parse}/${1} ] ; then 
   rm -rf ${1}
fi

Please note that your script is really unsafe, as you rm -rf a parameter without even evaluate it before… !

3/ You can add a wait between the git and the rm to wait for the end of the git process

4/ log your git command into a log file, with a redirection >> /tmp/git-jenkins-log

5/ put all of those commands in a script (see #2)

Answer 2

Following is an infinite loop in case rm -rf fail

while true
do
    if rm -rf .git ; then
        break
    else
        continue
    fi
done

indeed continue can be used in for or while loop to get the next entry but in this while loop it will run the same rm command forever.

Answer 3

Well, aparrently I was able to fix my issue by running the script from different user. By default on Windows Jenkins executes all jobs from the user SYSTEM. I have no idea why it affects the behaviour of my script but running it with psexec from specially created user account worked.

In case anyoune is interested, I did something like this:

psexec -accepteula -h -user Jenkins -p _password_ "full/path/to/bash.exe" full/path/to/script.sh