Running Python Scripts with Jenkins

Question

I am looking to schedule my python script runs with jenkins. The issue is, my scripts use a lot of libs like pandas etc that are installed on my mac terminal.

Is there a way to allow Jenkins to pick up these modules (or run the scripts as if it was terminal)? Also is there a way to run Python3 in jenkins?

I have already configured Jenkins to execute from custom workspace and have tried both shell and plugin executions.

Answer 1

The answer is yes, but it is detailed so I can only give you high level steps here. Jenkins can execute command line statements, and python modules can be run from the command line.

I would start by using the begins library to create a python file to run from the command line with arguments. Get it working on your local machine that way.

You will want to use either virtualenv or venv, and do all your pip installs using that virtual environment. Then you can copy the virtual environment to your Jenkins machine, or create a new one. Look into the freeze tool.

When calling your python from jenkins, you must first activate your virtual environment just as if you were working on it yourself.

You have a lot of research to do, but is very doable. I can help with follow up questions if needed.