how to execute raw git commands from gitpython?

Question

How do I execute this command through gitpython?

git show <commit-id> --word-diff-regex="([a-zA-Z0-9_]+)" | sed -n "s/.*\[-\([a-zA-Z_]*\)-\]{+\([a-zA-Z_0-9]*\)+}.*/\1 \2/p" | sort | uniq

import git 
git.Git("<directory-name").execute('git show 610003c068d8bb5ca3c2a6b8bbde76cbc6ff281a --word-diff-regex="([a-zA-Z0-9_]+)" | sed -n "s/.*[-([a-zA-Z_]*)-]{+([a-zA-Z_0-9]*)+}.*/\1 \2/p" | sort | uniq')

I'm facing this error when running from the.py file.

raise GitCommandError(redacted_command, status, stderr_value, stdout_value)
git.exc.GitCommandError: Cmd('g') failed due to: exit code(128)
  cmdline: g i t   s h o w   2386879h237gd8yy8v8yeg8yg9b1927j   - - w o r d - d i f f - r e g e x = " ( [ a - z A - Z 0 - 9 _ ] + ) "   |   s e d   - n   " s / . * \ [ - \ ( [ a - z A - Z _ ] * \ ) - \ ] { + \ ( [ a - z A - Z _ 0 - 9 ] * \ ) + } . * / ☺   ☻ / p "   |   s o r t   |   u n i q
  stderr: 'b"fatal: ambiguous argument '|': unknown revision or path not in the working tree.\nUse '--' to separate paths from revisions, like this:\n'git <command> [<revision>...] -- [<file>...]'"'

Answer 1

You have a couple of different problems, one of which might be somewhat self-cancelling but they all add up.

First, the execute function here expects an iterable , eg, a list or a tuple of strings. Unfortunately for you, a string is iterable:

>>> list("hello")
['h', 'e', 'l', 'l', 'o']

So:

"git show"

becomes:

["g", "i", "t", " ", "s", "h", "o", "w"]

for instance and you can wind up running the g command, which doesn't exist. That's why the error message refers to Cmd('g') .

Generally speaking, what you should do here is break your arguments up yourself, into individual words. This eliminates the need to use double quotes around the regular expression:

git.Git("<directory-name").execute(["git", "show",
    "610003c068d8bb5ca3c2a6b8bbde76cbc6ff281a",
    "--word-diff-regex=([a-zA-Z0-9_]+)"])

for instance.

This leads us to the remaining problem, though:

| sed -n "s/.*\[-\([a-zA-Z_]*\)-\]{+\([a-zA-Z_0-9]*\)+}.*/\1 \2/p" | sort | uniq

The series of items here separated by vertical bars are shell pipelines . Git does not run shell pipelines for you; only a shell , or something serving as one, can run shell pipelines.

It is possible to have the execute function invoke a shell. To do so introduces a lot of possibilities for mischief, and requires putting those double quotes back in that I encouraged you to remove earlier (note that they are present in your sed -n as well here). If you like, you can do this with:

git.Git("<directory-name").execute(['git show '
    '610003c068d8bb5ca3c2a6b8bbde76cbc6ff281a '
    '--word-diff-regex="([a-zA-Z0-9_]+)" | '
    r'sed -n "s/.*\[-\([a-zA-Z_]*\)-\]{+\([a-zA-Z_0-9]*\)+}.*/\1 \2/p" | '
    'sort | uniq'], shell=True)

(I have broken up one very long line here for posting purposes and to use a raw string in the third line; you can keep it as the long line without the string concatenation trick, if you prefer, but watch out for backslash interpolation.) It's almost always better, though, to avoid shell=True .

Your invocation of git diff --word-diff followed by this sed... | sort | uniq sed... | sort | uniq sed... | sort | uniq (which can be shortened to sed... | sort -u ) suggests that you're looking for word replacements in files. This might be better done directly by opening the various files (including the old committed ones) from Python—the gitpython code will help you get access to old committed files—and using your own word-splitting and the diff engine in Python's difflib . But that's another topic entirely. The main thing I'll point out here is that:

sed -n "s/.*\[-\([a-zA-Z_]*\)-\]{+\([a-zA-Z_0-9]*\)+}.*/\1 \2/p"

is probably flawed with respect to what you want. Consider, for instance, the diff that results from these two files:

$ cat one.x
this word is different
$ cat two.x
this wooord is changed
$ git diff --word-diff one.x two.x
diff --git a/one.x b/two.x
index 54b7c85..5357828 100644
--- a/one.x
+++ b/two.x
@@ -1 +1 @@
this [-word-]{+wooord+} is [-different-]{+changed+}

Running the output of git diff through the sed above, we get:

$ git diff --word-diff one.x two.x | sed -n "s/.*\[-\([a-zA-Z_]*\)-\]{+\([a-zA-Z_0-9]*\)+}.*/\1 \2/p"
different changed

Note how we lost the change from word to wooord . The problem here is that the .* match in sed is "greedy": it matches as many characters as possible. That's why we get only the last change on each line with a matching pattern.

You can partially fix this with the sed invocation by replacing .* with [^[*] , but that just means that only the first item gets printed. It's possible to fix that in sed as well: see, eg, split lines and operate on each new line with sed or my example sed script below, but again this really calls out for some other method. (Doing it entirely in Python is one way; using awk is another.)

---

Sample complete script that produces both word changes (and tested on slightly more complicated input since I was playing with sed and wanted to be sure I caught all cases):

git diff --word-diff one.x two.x |
sed -n \
    -e "s/[^[]*\[-\([a-zA-Z_]*\)-\]{+\([a-zA-Z_0-9]*\)+}\(.*\)/\1 \2\\
\3/" \
    -e "t found" \
    -e "b" \
    -e ": found" \
    -e "P;D"

This uses the fact that the pattern space normally just holds one line. If we find a match, we replace the one line in the pattern space with two: the pair of words, and the remainder of the line. Then we branch to the commands that print and delete only the first line in the pattern space, and fall off the end of the script so that sed restarts with the current pattern space. If we didn't find a match, we b to skip to the end of the script, at which point sed -n doesn't print the current pattern space (because of the -n ) and moves on to the next line.