Git submodule fails on TFS Build

Question

When I try to build using TFS with Git, there's a limitation whereby TFS's git provider doesn't yet support sub modules. Bit of a pain, but what the heck, I'm able to tell TFS to run a Batch file prior to compilation. I've used this to call a manual git script to update all my submodules in my "super" project.

The command this batch file runs is simply: git submodule update --init --recursive

This worked fine and dandy before I migrated the submodule source to TFS, however now the TFS build is failing, because the above git module script no longer works.

So, what TFS does before a build is Pulls the current sources from Git into a folder on the build server, which I have access to.

If I open Git Bash to this folder and run the following command(s): git submodule init git submodule update

I get the following error, and I can't work out for the life of me what it is. I've tried searching this specific error which generally points to a submodule commit being pushed after the "super project" repo is pushed. But I can verify that all submodule commits & pushes are performed BEFORE the "super project" commit & push is done. Here's the output from the TFS git commands:

james@TFS /C/Builds/1/Technique Webs/MIS Console 5.2 Development/src (5.2development)
$ git submodule init

james@TFS /C/Builds/1/Technique Webs/MIS Console 5.2 Development/src (5.2development)
$ git submodule update
Username for 'http://tfs:8080': james
Password for 'http://james@TFS:8080': <password>
From http://TFS:8080/TFS/Technique/_git/Technique%20Library
 * branch            HEAD       -> FETCH_HEAD
fatal: reference is not a tree: 33106ea146d470159e327c1b2d623d14f522cdd4
Unable to checkout '33106ea146d470159e327c1b2d623d14f522cdd4' in submodule path 'calc-engine'

james@TFS /C/Builds/1/Technique Webs/MIS Console 5.2 Development/src (5.2development)
$

Answer 1

I've fixed a similar issue after much trial and error: it turned out that this was a problem with the TFS pre-build PowerShell script running with no user profile, and therefore GIT couldn't see the credential cache setting for the build user (eg as set in C:\\Users\\theuser\\.gitconfig ).

Batch scripts would suffer from the same issue.

The distinctive behaviour was that the git submodule calls would hang completely, and would put the GIT repository in a very weird state (as noted by the not a tree errors above).

The hang is due to GIT prompting for the username and password to use, however this is a non-interactive session, so everything just grinds to a halt.

Using wincred as credential cache, I was able to fix this by:

1. Logging onto the build machine as the build user.
2. Accessing the GIT server via the command line and entered the required credentials.
3. Opened Windows Credential Manager and checked that the cached credential is visible in the Generic Credentials section.
4. In elevated command prompt, forced GIT to use wincred by default, even if it has no user config file: git config --system credential.helper wincred
5. Re-ran the build from TFS: which worked.

Answer 2

The reason is that 'Git Bash' does not know the credentials of the build account, and it should prompt every time interactively for them. Indeed, if the build account have access to all the submodules remote repo, it just need give empty user name and password for the credential prompt. This is not easy in Windows because the credential manager do not accept empty user name.

One way to workaround it to change the submodule's URL to something like below. The '@' symbol is same as feeding empty user name and password.

[submodule "..."]
    path = ...
    url = http://@tfs:8080/...