Managing packages: PyCharm vs conda vs pip

Question

I'm new to Python and recently installed PyCharm 2016.3 on Windows 10. I'm also using Anaconda 3.

I don't know much about package management and would like to understand it better. Normally I just use conda update --all but I noticed (by checking the package list of my local PyCharm Interpreter) that this doesn't upgrade all packages to the latest version.

One such package is Pillow of which there's a version 4.0.0 but conda (4.3.11) won't update it past 3.4.2. I tried conda install pillow: 4.0.0 and got:

UnsatisfiableError: The following specifications were found to be in conflict:
  - pillow 4.0.0*
  - python 3.5*
  - spyder-app
Use "conda info <package>" to see the dependencies for each package.

Later I found out that Pillow is also available on conda-forge so I tried conda install -c conda-forge pillow=4.0.0 and got:

The following NEW packages will be INSTALLED:

    libiconv:  1.14-vc14_4   conda-forge [vc14]
    libxml2:   2.9.3-vc14_9  conda-forge [vc14]
    olefile:   0.44-py35_0   conda-forge
    vc:        14-0          conda-forge

The following packages will be UPDATED:

    freetype:  2.5.5-vc14_2              [vc14] --> 2.7-vc14_0    conda-forge [vc14]
    jpeg:      8d-vc14_2                 [vc14] --> 9b-vc14_0     conda-forge [vc14]
    libtiff:   4.0.6-vc14_2              [vc14] --> 4.0.6-vc14_7  conda-forge [vc14]
    pillow:    3.4.2-py35_0              --> 4.0.0-py35_2  conda-forge

The following packages will be SUPERCEDED by a higher-priority channel:

    conda:     4.3.11-py35_0             --> 4.2.13-py35_0 conda-forge
    conda-env: 2.6.0-0                   --> 2.6.0-0       conda-forge
    qt:        4.8.7-vc14_9              [vc14] --> 4.8.7-vc14_6  conda-forge [vc14]

I decided not to proceed and instead tried pip install pillow . Since this command doesn't ask for confirmation the package was simply installed. Now when I type conda list I get:

Pillow                    4.0.0                     <pip>
pillow                    3.4.2                    py35_0

The package list of the PyCharm Interpreter now shows Pillow as being version 4.0.0 but conda update pillow still returns:

# All requested packages already installed.
pillow                    3.4.2                    py35_0

My questions are:

1) What should I rely on to keep all my packages up to date, without compatibility issues?

2) Why did conda install pillow: 4.0.0 return an error but conda install -c conda-forge pillow=4.0.0 didn't?

3) What do the * next to pillow 4.0.0 and python 3.5 in the list of dependencies mean?

4) Since now I have both Pillow 3.4.2 (in /anaconda3/pkgs ) and Pillow 4.0.0 (in /anaconda3/lib/site-packages ) which one would be used if I imported Pillow?

5) Does the superseding conda: 4.3.11-py35_0 --> 4.2.13-py35_0 conda-forge mean conda is getting downgraded?

6) What is the difference between the tags pip, py35_0, py35_4, np111py35_2, etc?

7) PyCharm tells me there's a version 2.9.5 of package Jinja2 but both normal conda and conda-forge only find 2.9.4. From which channel is PyCharm getting this information?

Answer 1

Ok, I can't answer all of your questions but here goes:

1) Conda defers to the "pain up front" approach for handling dependency/conflict resolution. You'll have to get all of your packages to play nicely together in the repo's/channels that you have available to even make a package or keep them in an environment together. You can try running it with --force or --no-deps to try getting it in but ..... that can cause issues for you in the future (I don't know if that would even work with the later versions of conda, it changes a lot). Simply keeping packages up to date, and up to latest, I would just use pip. Its come a long way in the last few years ( https://glyph.twistedmatrix.com/2016/08/python-packaging.html )

2) I am not completely sure, I believe it would have something to do with providing an explicit non-url channel for conda to look at. Typically you pass it the URL to the conda-forge repo (I think, again we don't use conda-forge internally).

3) The * means you are ignoring the patch/build 4.0.0 == Major.Minor.Build. Likewise, 3.5* == any version of 3.5

4) I would import pillow in a terminal, and then print out the module to see where its getting pulled from, why guess?

5) pass (although I think so)

6)

• pip : means you installed that package via pip. It will not be picked up if you do conda list --explicit
• py35_0 : has a requirement / only available to envs / packages that use python 3.5
• py35_4 : not sure (always forget that one)
• np111py35_2 : requires python3.5 and also numpy 1.11 (I think *)

7) I tend to steer clear of pycharm, I believe that you can inspect the python interpreter that pycharm is pointing at to see what environment its in. Based on the root environment, you can do a conda info and get a list of all of the channels you are pointing to.

Note: if you are going to use conda, you may just want to add conda-forge to your channels list instead of passing the -c (but seeing how the other channels are organized should help you see how you should pass the -c flag)