git add . not adding some subdirectories that are git repos

Question

I used rsync to copy over some important data files to a backup location, and when I try to commit everything, some of the git repos that were copied over aren't being added.

I get the feeling this has to do with git's submodule feature? Is there perhaps a command line switch to tell git to treat all subdirectories as normal folders?

$ git status
# On branch master
# Changes not staged for commit:
#   (use "git add <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working directory)
#   (commit or discard the untracked or modified content in submodules)
#
#   modified:   home/user/dev/ruby/vid_downloader (modified content)
#   modified:   home/user/source/htdigest (untracked content)
#   modified:   home/user/source/node (untracked content)
#
no changes added to commit (use "git add" and/or "git commit -a")

$ git add .

$ git status
# On branch master
# Changes not staged for commit:
#   (use "git add <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working directory)
#   (commit or discard the untracked or modified content in submodules)
#
#   modified:   home/user/dev/ruby/vid_downloader (modified content)
#   modified:   home/user/source/htdigest (untracked content)
#   modified:   home/user/source/node (untracked content)
#
no changes added to commit (use "git add" and/or "git commit -a")

All those folders that aren't committing are git repos with changes that haven't been committed to their working directories...

This is a part of a backup script, so any kind of cmd line workaround would be appreciated.

Answer 1

Try git add -A option in each submodule. when -A option is used, all files in the entire working tree are updated (old versions of Git used to limit the update to the current directory and its subdirectories).

or

Try git submodule foreach --recursive git add -A and git submodule foreach --recursive git commit -m "some meaningful message" .

Answer 2

Sounds like your workflow is:

1. Rsync copy a git repository with submodules to a remote server
2. On the remote server, commit the copied git repository to a backup git repository.

Your first option is, upon copy, delete the ".git" folder in each submodule. That will make git treat is as a normal folder. I do not recommend this option due to losing history of the submodule.

If I understand you correctly, I think you should stop using rsync and use git itself properly.

Set up a remote git repository for the git repository as well as its submodules, (see here: http://thelucid.com/2008/12/02/git-setting-up-a-remote-repository-and-doing-an-initial-push/ )

then in your backup script call

git push backup_server
git submodule foreach --recursive git push backup_server

After setting backup_server to their respective backup repositories using

git remote add backup_server https://backup_server_location/backup_repo.git

Answer 3

Git is reporting uncommitted changes in your submodules. The uncommitted content in home/user/dev/ruby/vid_downloader has been added but not yet committed (it's only listed in the index, not in a committed tree); the uncommitted content in the other two is wholly untracked.

This matters because the point of a submodule is that when you add the id of its checked out commit to your current index (and then commit it), you can later recreate the specific, committed build environment for your main project by checking out that specific commit from the nested repo. So git status by default warns you about any content in submodule's worktree that won't be exactly recreated by checking out its currently-checked-out commit.

If any specific untracked files in your htdigest and node submodules will not affect later rebuild either add them to the subproject's .gitignore or if you don't want their names committed to its .gitignore s you can make a repo-local list of names you don't care about in its .git/info/exclude . See the gitignore docs .

The modified (and tracked) files in the vid_downloader work tree are not necessarily a problem, but remember that the only thing in your main project commit is the id of a commit that's supposed to checked out from the nested repository (that's all a submodule is) -- but checking out its current commit will not produce the tracked content that's there now. That's flat-out dangerous in the general case, and you should be very sure it isn't a problem (maybe it's a file that really shouldn't be committed, or you did your testing with the unaltered content and have since moved on).

---

That said, are you certain your backup script isn't also replicating the nested work trees and repositories? If it is, the workaround is probably to ignore those status messages, because they're describing potential later checkouts in a different environment, not a full restore of the existing one.

Answer 4

i always use

git add --all .

to add all files