Eclipse removing entries from the global easy_install.pth?

Question

We are having some annoying problems with Eclipse, which apparently have been always there, on Windows 7 + a custom python installation + Eclipse 3.7 and the latest Pydev.

The problem is basically that if when setting up the python interpreter you add all the libraries too, then Eclipse or Pydev or who knows what deletes all these entries from the global easy_install.pth file, which are also configured in Pydev. This means that within Eclipse everything works fine, but anything outside of it can't possibly work correctly.

Now I find it hard to believe that noone noticed, so I would think that we're doing something wrong.

I also filled a bug which didn't get much attention yet:

https://sourceforge.net/tracker/?func=detail&aid=3446052&group_id=85796&atid=577329

Any idea of what and why it could be? Already being able to know exactly who and when modify that file would help a little bit, any suggested tool?

EDIT: I'm trying with Process Monitor as suggested below. So I fire up Eclipse, I run a simple Python command and I get something like this happened the file.

"File Time","Total Events","Opens","Closes","Reads","Writes","Read Bytes","Write Bytes","Get ACL","Set ACL","Other","Path" "0.0000306","4","1","1","0","0","0","0","0","0","2","C:\\python25\\Lib\\site-packages\\easy-install.pth"

So basically it was open:1, closed:1, other:2. And the file was actually modified.

My impression is that Eclipse just stupidly overwrites the file, otherwise there should be also some read operations, right?

That, however, doesn't help me that much, I can only see Eclipse as the process modifying the file, not much else.

Answer 1

Ok, I must say I haven't been able to reproduce this so far in PyDev... do you know if that's only happening in your machine (or some other colleague also has the same problem)?

To try to reproduce it, can you give more details on your toolchain: what's your python version and what's your setuptools version? Which packages you have installed? (my feeling is that the culprit is not really Eclipse/PyDev, but some python package that's doing that).

Just to note, PyDev simply doesn't do anything related to .pth files -- all it does is launching your python script with the PYTHONPATH setup in the environment variables for the launched process (so, it's very unlikely that PyDev is writing that file -- simply because it doesn't know anything related to .pth files).

One thing that could help in getting to the root of it: you can try making that file unwritable (you should be able to change its permissions so that you can't modify it), then, try to reproduce and see if you get some error somewhere (because if someone tries to write it, there'll be an exception because you don't get permissions to do so).

Answer 2

Since you are on windows, why not use process monitor ? Turn it on, filter for your file(s), then install. It'll log who touches the file, and in what sequence. That should find your culprit.

Answer 3

I haven't really found the fix for the real problem, but at least I found a workardound.

What I do now is to store in a text file all the egg names that should be in the sys.path at run-time. Before doing anything else I take this list, join these paths with the correct site-packages path, and add what was missing to the sys.path.

It seems to work, but if I remove some libraries (like pyqt) even if it's they are added I still get som errors, so there is still something missing...

    import pkg_resources
    import sys

    from distutils.sysconfig import get_python_lib
    from os import path
    from psi.devsonly.utils import filename_to_list


    #TODO: there appears to be an issue with QT removing the entries
    def update_sys_path():
        """Make sure that the sys path contains all the entries needed
        """
        site_packages = get_python_lib()
        req = pkg_resources.Requirement.parse('psi.devsonly')
        wea = pkg_resources.resource_filename(req, 'windows_easy_install.pth')
        ppack_list = filename_to_list(wea)

        for p in ppack_list:
            full_name = path.join(site_packages, p)
            # the lower() is necessary or the entries are not found
            if full_name.lower() not in sys.path:
                print("adding to the path the entry %s" % full_name)
                sys.path.insert(0, full_name)
            else:
                print("%s already present in the path" % full_name)