git submodule branch clone

Question

I wnat to do a branch of the project including same branch in all submodules Then after comit & push for all subrepos & main repo I want to be able to clone the new branch for everithing

After I commit and everithing looks fine - I try to clone.

For some resone the submodel do not point to the new branch Forcing me to shitch/checkout in the submodule

Is this the Normal behaiore? Am I doing somthing wrong / could do somthing better? Is there a way to "clone" all submodules with branch switched?

Thank you. David.

Answer 1

TL;DR

This is normal. You'll want to write or find a little wrapper to deal with it (or hooks: see see philb's comment ). Read on for the long explanation.

Submodules use commits , not branches

This is a slight overstatement, but the main thing here is that it's only slight . If you start with that idea in mind, the rest of it all makes sense.

Remember what a submodule is:

• First, it is a Git repository, so it consists of commits. It has branch and tag names that help Git find those commits, but it's really all about the commits.

• Second—and this is the part that makes it a submodule, rather than just a plain repository—it's controlled by some other Git repository, which Git calls the superproject for this submodule.

The key to understanding everything about submodules is that the superproject lists a raw commit hash . The superproject Git will run:

git -C path/to/submodule checkout $hash

or equivalently:

git -C path/to/submodule switch --detach $hash

whenever you have the superproject in charge check out some commit in the superproject. The $hash value for this command is determined by the index in the superproject Git repository, and the index hash value is initially that from the current commit. So it's the current commit that controls the hash ID that the superproject forces upon the submodule.

Now, this too is too strong a claim: the superproject doesn't always go into the submodule and run a detached-HEAD switch / checkout command. It does so at some particular set of times. Precisely when it goes into the submodule and does this gets us into all the gory details. But it's the way submodules are intended to work. Declaring a branch for a submodule, with the git submodule add -b option for instance, does not change this behavior.

What you'd like to see, apparently, is:

git -C path/to/submodule checkout $branch

or:

git -C path/to/submodule switch $branch

where $branch is set to a branch name, that comes from somewhere. There are no built in cases where exactly this occurs, but there are a few ways to make it happen.

So when do you get branch-name behavior in submodules?

What actually runs the git checkout inside each submodule is a git submodule update command. (See sidebar below.) If you consult the documentation , you will see that git submodule update has a huge number of options. There is no combination of options that makes git submodule update run the command you want.

Git will do an automatic submodule update (see sidebar again) on git checkout in the superproject if you have set the recursion option for that checkout. That is, git checkout --recurse-submodules will do it, as will plain git checkout if you've set submodule.recurse to true in your configuration. An explicit git checkout --no-recurse-submodules will inhibit Git's automatic update.

(Note that git clone itself ends by running an internal git checkout command, unless you use git clone --no-checkout . That checkout will recurse, or not recurse, based on the --recurse-submodules flag passed to git clone , or again the configuration you've set.)

If you didn't have git checkout do an automatic update, you would normally manually run your own git submodule update command now. You can choose not to run git submodule update , or to run it but follow it up with a:

git -C path/to/submodule switch $branch

command. None of this is automatic, of course.

Sidebar: git checkout does not use git submodule update

The user-facing command for updating submodules is git submodule update . But git checkout itself doesn't literally run git submodule update : instead, it has hardcoded into it the behavior of git submodule update --checkout . This is, at least currently, in entry.c , which notices that the gitlink hash ID changed and calls submodule_move_head in submodule.c , which currently uses an invocation of git read-tree , passing it a raw hash ID. So it can't use the desired commands.

Using git submodule update

You can select a submodule update mode:

The "updating" can be done in several ways depending on command line options and the value of submodule.<name>.update configuration variable. The command line option takes precedence over the configuration variable. If neither is given, a checkout is performed. ...

checkout
the commit recorded in the superproject will be checked out ...

This is the default, and is what you are seeing.

rebase
the current branch of the submodule will be rebased onto the commit recorded in the superproject.

This is closer to what you want, but isn't what you want at all, plus there's a bootstrap issue: what exactly is the current branch of the submodule?

merge
the commit recorded in the superproject will be merged into the current branch in the submodule.

That's also not what you want, and has the same question to resolve.

The following update procedures are only available via the submodule.<name>.update configuration variable:

custom command
arbitrary shell command that takes a single argument (the sha1 of the commit recorded in the superproject) is executed. When submodule.<name>.update is set to !command , the remainder after the exclamation mark is the custom command.

This is the only one of the update commands that could do what you want. You can set a custom command that will do your desired git checkout or git switch . Note that the branch name is not provided to you; you will have to fish it out from somewhere. This is overly difficult (I think); read on, or jump to the conclusion section to see what I think is the right way to do it.

none the submodule is not updated

(I included this for completeness, but obviously doesn't do what you want.)

But wait: there is one more option! It's not described here, for some reason; it's left down in the OPTIONS section. It is the --remote option:

--remote
This option is only valid for the update command. Instead of using the superproject's recorded SHA-1 to update the submodule, use the status of the submodule's remote-tracking branch. The remote used is branch's remote ( branch.<name>.remote ), defaulting to origin . The remote branch used defaults to the remote HEAD , but the branch name may be overridden by setting the submodule.<name>.branch option in either .gitmodules or .git/config (with .git/config taking precedence).

This works for any of the supported update procedures ( --checkout , --rebase , etc.). The only change is the source of the target SHA-1. For example, submodule update --remote --merge will merge upstream submodule changes into the submodules, while submodule update --merge will merge superproject gitlink changes into the submodules.

In order to ensure a current tracking branch state, update --remote fetches the submodule's remote repository before calculating the SHA-1. If you don't want to fetch, you should use submodule update --remote --no-fetch .

This does almost what you want. There are two problems:

• It runs git -C path/to/submodule fetch , then git -C path/to/submodule rev-parse origin/$branch , then runs git -C path/to/submodule switch --detach $hash where $hash is the result of the rev-parse step. (It's a little silly that it uses three commands to achieve this when two, fetch and switch --detach , would suffice: the switch can do the rev-parse internally. But this is a side effect of the way it's coded.)

  The drawback here is the extra fetch and the use of origin/$branch . You can stop the fetch with --no-fetch , although it's probably harmless and maybe sometimes desirable. The use of origin/$branch is also probably harmless: if you've just cloned the submodule repository, $branch may not even exist , and the result is always going to be a detached HEAD because submodules are always detached HEADs, so origin/$branch should be fine.

• The bigger drawback is that you literally can't get --remote specified without manually running a git submodule update in the first place. And, in any case, you're still left on a detached HEAD in the submodule!

Conclusion

Is there a way to "clone" all submodules with branch switched?

No. However, if you're willing to use something other than Git—eg, to use a wrapper that runs Git—it's easy enough. For your case, a trivial wrapper that runs git clone followed by git submodule update --init followed by git submodule foreach --recursive with a shell function (that you provide) that invokes the git switch command would do the job. Using git submodule foreach is superior (I think) to trying to use a custom update command, as you'll gain easy access to the superproject and submodule parts so that you can fish out a branch name from the superproject's settings.

You can also use git clone followed by git submodule update --checkout --remote to get to the right hash ID , if that's good enough.