python program works fine from command line and as git difftool, but dies as external-diff

Question

Here is the program running normally, it is dying but for a good reason (because files a and b do not exist).

% dmp_diff a b
{'LESS': '-R', 'LC_CTYPE': 'en_US.UTF-8', 'SHELL': '/usr/local/bin/zsh', 'LOGNAME': 'lust', 'USER': 'lust', 'HOME
': '/Users/lust', 'PATH': '/Users/lust/bin:/Users/lust/util:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/loc
al/share/python:/opt/X11/bin:/Users/lust/bin:/Users/lust/util:/usr/local/share/npm/bin:/usr/local/share/npm/bin',
 'DISPLAY': '/tmp/launch-uW8956/org.macosforge.xquartz:0', 'TERM_PROGRAM': 'iTerm.app', 'LANG': 'en_US.UTF-8', 'T
ERM': 'xterm-256color-italic', 'Apple_PubSub_Socket_Render': '/tmp/launch-rPIGNa/Render', 'SHLVL': '2', 'SECURITY
SESSIONID': '186a4', 'ITERM_SESSION_ID': 'w0t0p0', '_': '/Users/lust/util/dmp_diff', 'GREP_COLOR': '1;32', 'SSH_A
UTH_SOCK': '/tmp/launch-pgXEyS/Listeners', 'TMUX': '/private/var/folders/z_/ybb_x1f96l9d2yqr18ncmg1w0000gn/T/tmux
-501/default,19104,0', 'GREP_OPTIONS': '--color=auto', 'ITERM_PROFILE': 'Default', 'TMPDIR': '/var/folders/z_/ybb
_x1f96l9d2yqr18ncmg1w0000gn/T/', 'LSCOLORS': 'Gxfxcxdxbxegedabagacad', 'OLDPWD': '/Users/lust', '__CF_USER_TEXT_E
NCODING': '0x1F5:0:0', 'Apple_Ubiquity_Message': '/tmp/launch-V9NNsf/Apple_Ubiquity_Message', 'PWD': '/Users/lust
/.vim', 'PAGER': 'less', 'TMUX_PANE': '%10', 'COMMAND_MODE': 'unix2003'}
Files: a b
Traceback (most recent call last):
  File "/Users/lust/util/dmp_diff", line 46, in <module>
    file_content = open(filename, 'r').read()
IOError: [Errno 2] No such file or directory: 'a'
> /Users/lust/util/dmp_diff(46)<module>()
     45     file_process = Popen(['file', '-'], stdin=PIPE, stdout=PIPE)
---> 46     file_content = open(filename, 'r').read()
     47     file_content = map(lambda x: x.encode('string_escape')

ipdb>

And with

[diff]
    external = dmp_diff

I get this when I run the external diff with git; it dies for a bad reason (Notice it only got to line 16 instead of 46):

% git diff
{'LESS': '-R', 'GIT_PAGER_IN_USE': 'true', 'VERSIONER_PYTHON_PREFER_32_BIT': 'no', 'LC_CTYPE': 'en_US.UTF-8', 'SH
ELL': '/usr/local/bin/zsh', 'LOGNAME': 'lust', 'USER': 'lust', 'HOME': '/Users/lust', 'PATH': '/usr/libexec/git-c
ore:/usr/bin:/Users/lust/bin:/Users/lust/util:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/share/pytho
n:/opt/X11/bin:/Users/lust/bin:/Users/lust/util:/usr/local/share/npm/bin:/usr/local/share/npm/bin', 'DISPLAY': '/
tmp/launch-uW8956/org.macosforge.xquartz:0', 'TERM_PROGRAM': 'iTerm.app', 'LANG': 'en_US.UTF-8', 'GIT_PREFIX': ''
, 'TERM': 'xterm-256color-italic', 'Apple_PubSub_Socket_Render': '/tmp/launch-rPIGNa/Render', 'VERSIONER_PYTHON_V
ERSION': '2.7', 'SHLVL': '2', 'SECURITYSESSIONID': '186a4', 'ITERM_SESSION_ID': 'w0t0p0', '_': '/usr/bin/git', 'G
REP_COLOR': '1;32', 'SSH_AUTH_SOCK': '/tmp/launch-pgXEyS/Listeners', 'TMUX': '/private/var/folders/z_/ybb_x1f96l9
d2yqr18ncmg1w0000gn/T/tmux-501/default,19104,0', 'GREP_OPTIONS': '--color=auto', 'ITERM_PROFILE': 'Default', 'TMP
DIR': '/var/folders/z_/ybb_x1f96l9d2yqr18ncmg1w0000gn/T/', 'LSCOLORS': 'Gxfxcxdxbxegedabagacad', 'OLDPWD': '/User
s/lust/util/.git', '__CF_USER_TEXT_ENCODING': '0x1F5:0:0', 'Apple_Ubiquity_Message': '/tmp/launch-V9NNsf/Apple_Ub
iquity_Message', 'PWD': '/Users/lust/util', 'GIT_DIR': '.git', 'PAGER': 'less', 'TMUX_PANE': '%9', 'COMMAND_MODE'
: 'unix2003'}
Traceback (most recent call last):
  File "/Users/lust/util/dmp_diff", line 16, in <module>
    import ipdb
ImportError: No module named ipdb
external diff died, stopping at .dmp_diff.un~.
(END)

Now this works fine if I run dmp_diff as a git difftool . However, that is not sufficient for me because I want to use this diff program for git log -p and stuff like that, so that I don't need to come up with a bunch of brittle scripts, and I'm very used to running git log -p already (but want an improved git output).

I can tell that (obviously) in the second case that python is unable to find ipdb. I am not sure what is missing to get this to work. I would appreciate it if someone can show me ways to load a module from python that can provide more control than import . imp.find_module , perhaps? I would also love it if anyone knows what craziness Git is doing that is leading to this.

I am also aware of the fact that git sends as the second and seventh args the file names to be compared (rather than as the first and second args); my dmp_diff program already accounts for that (and it has nothing to do with this current issue).

Answer 1

Thanks for the tip, @ThomasFenzl, indeed git was running python 2.7.2 while from the command line it is running 2.7.5. Still not completely sure why it can't find the ipdb lib dir, but it was a good starting point for sure.

Apparently the use of #!/usr/bin/env python as opposed to specifying the exact python binary path (which on my system is the canonical /usr/local/bin/python ) causes it when run from git to grab the system's original python.

I think I still need to fall back to the /usr/bin/env python method though because one of the environments I need this on only has python installed in my user directory.