git: can I subtree merge just a subpath of a repository?

Question

I have the remotes Foo and Bar. Foo is a web application that has lots of directories, relevant amongst them is /public which contains assorted files and other directories.

Bar is a set of libraries and whatnot used on the front end, as such, it should go in /public/bar in Foo. Foo has no files there.

That would all be piece of cake with either submodules or subtree merge. However…

Bar's tree is messy, it has all sorts of pre-production files like PSDs and FLAs, and the only really useful part of it is what is inside its /www/tools .

So, what I want to do is merge Bar's /www/tools into Foo's /public/bar , and pretend the rest of Bar's tree doesn't even exist.

Can do?

(I would suppose this is very similar to how you merge from a project that originally merged yours as a subtree. Which I don't know how to do, either.)

Answer 1

I've successfully done this using the following commands (rewritten for your scenario).

$ git remote add Bar /path/to/bar
$ git merge -s ours --no-commit Bar/master
$ git read-tree --prefix=public/bar -u Bar/master:www/tools/

HUGE WARNING! I'm still in the process of figuring out how to do fetch/merge from Bar. This will bring in everything just once.

My current way of merging changes is like so:

$ get fetch Bar
$ git pull -X subtree=public/bar Bar master

This will leave you with conflicts saying that a ton of files have been removed, you can just git rm them and then git commit .

I'm certainly open to suggestions on a better way to pull changes.

Since this is an old thread, I should add that I'm running Git 1.7.9.

Answer 2

Try using git subtree for this. It allows you to extract a subtree of a project into another project, which you can then merge into yours.

Answer 3

Edit: It occurs to me that you could do this just with git merge --no-commit . This will attempt the merge, and even if it does not conflict, it will stop just before committing. At this point you can remove all the junk you don't need (including restoring conflicted files if necessary) and create a merge commit only containing the desired subtree.

Original answer:

You can indeed use filter-branch for this. An outline:

Clone your source repo:

git clone --bare /path/to/bar /path/to/bar_clone

Using a bare clone will save you the time and space of creating a working directory.

Next, use filter-branch on the clone:

git filter-branch --index-filter 'git rm -rf <unwanted files/directories>' -- --all

The --all lets it know that you want to use all refs, not just the current HEAD. You will now have a repository containing only the desired subdirectory, and all of the history associated with it.

Note: Sorry, I don't know a really straightforward way to remove all but what you want. You have to be careful with wildcards because you don't want to clobber any git directories. Here's something that'll work, though it's slower, especially if you've got a lot of files:

git filter-branch --index-filter 'git rm -f `git ls-files | grep -v ^www/tools`' -- --all

Anyway, however you manage the listing of files to remove, you can go ahead with your subtree merge, pulling from bar_clone into foo .

Answer 4

I don't think this can be done using a merge, per-se, but these steps might do the trick, at least as far as the end product is concerned. First:

% git fetch Bar

This fetches the latest commit(s) from the Bar repository, but doesn't try to merge them. It records the SHA for the commit in .git/FETCH_HEAD

% cat .git/FETCH_HEAD
b91040363160aab4b5dd46e61e42092db74b65b7                branch 'Bar' of ssh://blah...

This shows what the SHA identifier for the latest commit from the remote branch is.

% git checkout b91040363160aab4b5dd46e61e42092db74b65b7 www/tools

This takes the copy of the files in www/tools for the fetched commit and overwrites what's in the working tree. You can then commit those changes to your local repository like normal. The resulting commit won't have any reference to where it came from, but it should at least get your repository with the versions of the files you want.

Answer 5

I would suggest using subtree twice - once to extract all of the /www/tools and once to extract /public - it sounds like /public should be in its own repo so, I would suggest you push /public to a new repo - with all of it's subtree history, then merge /www/tools's subtree history into the new /public repo and add that back as a subtree of Foo.

cd foo
git subtree split --prefix=public --branch=new-shared-public --annotate='(split) '
cd../bar
git subtree split --prefix=www/tools --rejoin --branch=new-shared-www-tools --annotate='(split) '

You now have two new branches on your repos. One with public's commit history and the other with www/tools. I'll omit cd from here on.

Create your new public repo (I'm assuming github here but sounds like you might want to do it locally) and push your subtree there: git checkout new-shared-public git push git@github.com:my_id/new-shared-repo.git HEAD:master

Then merge your other branch into that:

git checkout new-shared-www-tools
git remote add Foo git@github.com:my_id/new-shared-repo.git
git merge -s ours --no-commit Bar/master

Although I've specified ours as the commit policy, it probably ( probably ) doesn't matter.

Finally, after you've preformed the merge and pushed it to the new repo, go back to the Foo repo. Git rm the public history from there and add the new public repo as a subtree:

git subtree add --squash --prefix shared git@github.com:my_id/new-shared-repo.git master
git subtree pull --squash --prefix shared git@github.com:my_id/new-shared-repo.git master
git push