How to exclude certain directories/files from git grep search

Question

Is there a way to exclude certain paths/directories/files when searching a git repository using git grep ? Something similar to the --exclude option in the normal grep command?

I need to use git grep because using grep directly runs too slowly on large git repositories.

Answer 1

In git 1.9.0 the "magic word" exclude was added to pathspec s. So if you want to search for foobar in every file except for those matching *.java you can do:

git grep foobar -- './*' ':(exclude)*.java'

Or using the ! "short form" for exclude:

git grep foobar -- './*' ':!*.java'

Note that in git versions up to v2.12, when using an exclude pathspec , you must have at least one "inclusive" pathspec . In the above examples this is the ./* (recursively include everything under the current directory). In git v2.13 this restriction was lifted and git grep foobar -- ':!*.java' works without the ./* .

You could also use something like :(top) (short form: :/ ) to include everything from the top of the repo. But then you'd probably also want to adjust your exclude pathspec to start from the top as well: :/!*.java (otherwise it would only exclude *.java files from under your current directory).

There's a good reference for all the "magic words" allowed in a pathspec at git-scm.com (or just git help glossary ). For some reason, the docs at kernel.org are really out of date even though they often come up first in google searches.

Answer 2

Update: For git >= 1.9 there is native support for exclude patterns, see onlyone's answer .

This may seem backwards, but you can pass a list of files not matching your exclude pattern to git grep like this:

git grep <pattern> -- `git ls-files | grep -v <exclude-pattern>`

grep -v returns every path not matching <exclude-pattern> . Note that git ls-files also takes a --exclude parameter, but that is only applied to untracked files .

Answer 3

It's not possible, but has been discussed recently . Proposed workaround in link:

You can put *.dll to .gitignore file then git grep --exclude-standard .

EDIT see onlynone's answer , since git 1.9.0 it's possible.

Answer 4

You can mark files or directories as binary by creating an attributes file in your repository, eg

$ cat .git/info/attributes 
directory/to/ignore/*.* binary
directory/to/ignore/*/*.* binary
another_directory/to/also/ignore/*.* binary

Matches in binary files are listed without the including line, eg

$ git grep "bar"
Binary file directory/to/ignore/filename matches
other_directory/other_filename:      foo << bar - bazz[:whatnot]

Answer 5

With the example by @kynan as base I made this script and put it in my path ( ~/bin/ ) as gg . It does use git grep but avoids some specified filetypes.

In our repo its a lot of images so I have excluded the imagefiles, and this takes the serchtime down to 1/3 if I search the whole repo. But the script could easily be modified to exclude other filestypes or geleralpatterns.

#!/bin/bash                                                                    
#                                                                              
# Wrapper of git-grep that excludes certain filetypes.                         
# NOTE: The filetypes to exclude is hardcoded for my specific needs.           
#                                                                              
# The basic setup of this script is from here:                                 
#   https://stackoverflow.com/a/14226610/42580                                  
# But there is issues with giving extra path information to the script         
# therefor I crafted the while-thing that moves path-parts to the other side   
# of the '--'.                                                                 

# Declare the filetypes to ignore here                                         
EXCLUDES="png xcf jpg jpeg pdf ps"                                             

# Rebuild the list of fileendings to a good regexp                             
EXCLUDES=`echo $EXCLUDES | sed -e 's/ /\\\|/g' -e 's/.*/\\\.\\\(\0\\\)/'`      

# Store the stuff that is moved from the arguments.                            
moved=                                                                         

# If git-grep returns this "fatal..." then move the last element of the        
# arg-list to the list of files to search.                                     
err="fatal: bad flag '--' used after filename"                                 
while [ "$err" = "fatal: bad flag '--' used after filename" ]; do              
    {                                                                          
        err=$(git grep "$@" -- `git ls-files $moved | grep -iv "$EXCLUDES"` \  
            2>&1 1>&3-)                                                        
    } 3>&1                                                                     

    # The rest of the code in this loop is here to move the last argument in   
    # the arglist to a separate list $moved. I had issues with whitespace in   
    # the search-string, so this is loosely based on:                          
    #   http://www.linuxjournal.com/content/bash-preserving-whitespace-using-set-and-eval
    x=1                                                                        
    items=                                                                     
    for i in "$@"; do                                                          
        if [ $x -lt $# ]; then                                                 
            items="$items \"$i\""                                              
        else                                                                   
            moved="$i $moved"                                                  
        fi                                                                     
        x=$(($x+1))                                                            
    done                                                                       
    eval set -- $items                                                         
done                                                                           
# Show the error if there was any                                              
echo $err                                                                      

Note 1

According to this it should be possible to name the thing git-gg and be able to call it as a regular git command like:

$ git gg searchstring

But I can not get this working. I created the script in my ~/bin/ and made a the git-gg symlink in /usr/lib/git-core/ .

Note 2

The command can not be made into an regular sh git-alias since it will then be invoked at the root of the repo. And that is not what I want!