Git commands to automatically subtree merge a project with submodules?

Question

I found Using Git, what's the best way to subtree merge an external project that has submodules? but while related, it doesn't answer my question.

Suppose I have my Parent project and my Child project, which contains submodules. How do I subtree merge the Child project into a subdirectory of the Parent project and keep the references to the submodules working?

I've tried things like:

git remote add -f child_remote git://github.com/me/child.git
git checkout -b child_branch child_remote/master
git submodule init
git submodule update
git checkout master
git read-tree --prefix=child_directory/ -u child_branch

But read-tree loses the submodules, and .gitmodules appears only in the subdirectory. When I try to do git submodule init and git submodule update I get errors like "No submodule mapping found in .gitmodules for path 'child_directory/submodule_directory'".

I've also tried to modify the instructions at http://help.github.com/subtree-merge/ to work with submodules, but to no avail.

I understand that I could accomplish this by manually modifying the .gitmodules file to refer to the correct path, or merging in the Child project without submodules and then re-adding them to the Parent project, but I consider both of those solutions to be hacks. I'm looking for the actual commands one would run to get this to work automatically, without having to manually edit anything.

Answer 1

You could either change the paths in .gitmodules and then do git submodule init && git submodule update or you could convert the submodules to subtree's. The last option involves some kind of script because there is no automatic way to do this.

When using a shell script you could use these functions to get the submodules and their names (from the git-submodules source):

# This lists the unmerged submodules
module_list()
{
    git ls-files --error-unmatch --stage -- "$@" |
    perl -e '
    my %unmerged = ();
    my ($null_sha1) = ("0" x 40);
    while (<STDIN>) {
        chomp;
        my ($mode, $sha1, $stage, $path) =
            /^([0-7]+) ([0-9a-f]{40}) ([0-3])\t(.*)$/;
        next unless $mode eq "160000";
        if ($stage ne "0") {
            if (!$unmerged{$path}++) {
                print "$mode $null_sha1 U\t$path\n";
            }
            next;
        }
        print "$_\n";
    }
    '
}

# This gets the submodule's name from the .gitignore file in your webroot
module_name()
{
    # Do we have "submodule.<something>.path = $1" defined in .gitmodules file?
    re=$(printf '%s\n' "$1" | sed -e 's/[].[^$\\*]/\\&/g')
    name=$( git config -f $2/.gitmodules --get-regexp '^submodule\..*\.path$' |
        sed -n -e 's|^submodule\.\(.*\)\.path '"$re"'$|\1|p' )
    test -z "$name" &&
    die "$(eval_gettext "No submodule mapping found in .gitmodules for path '\$path'")"
    echo "$name"
}

Then, when you iterate the results, you can extract the name and url to use for the subtrees.

module_list | while read mode sha1 stage path
do
    #the path as your repo sees it
    full_path=$path

    #The path as .gitmodules in your child_directory sees it
    fake_path=${path[*]/[child_directory]\//}

    #Get the name of the submodule
    stupid_name=$(module_name "$fake_path" "[child_directory]")

    #Get the git URL from the .gitmodules file
    git_url=$(git config -f [child_directory]/.gitmodules submodule."$stupid_name".url)

    #Gets the clean name to use as remote name
    new_remote=$(echo $git_url | cut -d. -f2 | cut -d/ -f2- | cut -d/ -f2)

    #Remove the submodules folder
    rm -rf $path 

    #Add the subtree
    git remote add -f $new_remote $git_url
    git merge -s ours --no-commit $new_remote/master 
    git read-tree --prefix=$full_path/ -u $new_remote/master
    git commit -m "$new_remote merged in $full_path"
done 

I'm sure there is a way better way of doing this but it gives an idea of what is to be done

Answer 2

Why not use submodules in your project? You could now use 'git submodule update --recursive` across the board.

Also there is git-slave to help you automate branches across submodules.