Python environment maintenance

Question

This question is a bit of a rant as well - but a genuine request for help!

I am constantly battling to make sure my Python installations are up to date and as close to identical as possible between all the different machines and devices I use (eg, raspberry pi's, desktop PC's and laptops).

My latest frustration stems specifically from a practical example where I have two identical Pi Zero W's, with BME280 sensors, running the exact same Python script - one works perfectly, the other complains about urllib3 - even though both Pi's run DietPi with the same versions of Python etc, and even after uninstalling and re-installing influxdb (first started complaining about influxdb client not being available - then after uninstalling and re-installing, it found something new to moan about in the form of urllib3...), and then uninstalling and re-installing urllib3 and requests (but still not working...)

I find it similarly frustrating even just with keeping and maintaining one version of Python and libs on one PC - especially if you throw a Mac into the mix that seemingly has its own "dedicated" Python installation - try installing a "dedicated" Python environment that will not mess with the "built-in" version also causes things to get so messy after a while - you don't know what is where anymore, and even using pip/pip3 starts producing outcomes you don't expect in terms of having libraries available where you expect to see them...

Is there some elegant solution to this that is easily transferable between devices etc? Perhaps something like having a "master" python installation somewhere that I can clone or "package" and then move along with the script so I know it will work on any device I plonk it on, and will remain working regardless of some update in future? a Docker container would work - but I am pretty sure it will be too big for a Pi Zero...

I know about the venv's, but I am not sure if this is something that I can "copy and paste" between devices. Again, Ideally, I'd like to create the code on my PC, and just be able to copy the python file(s) to another device and know that it will run the same everywhere... </end rant>:)

Answer 1

Writing some code that work on any Python environment and will continue to do whatever the updates is hopeless except for really trivial code ( print ('Hello') should give same results whatever the Python version but is not really interesting).

AFAIK, the only reasonable way is to package your own code, in the sense of the Python Packaging User Guide . The packaging will manage the dependencies and the Python versions so if you can install it with some /path/to/python -m pip install (resp. py -x -m pip install for Windows without venv), then it should work with the same python that was used for installation.

It does require some initial work to understand Python Packaging and building the packages but it should save you some headaches when installing your application on a new environment... And if you want to be able to have various Python applications requiring potentially incompatible versions of libraries, venv is definitely the solution: you build a venv, install your own package in it, and find exactly the dependencies you need with the proper versions.