How to implement cross-platform continuous integration for python package development?

Question

I am developing some python packages and I do want to perform proper testing before releasing them to PyPi.

This would require running the unittests across

• different python versions: 2.5, 2.6, 2.7, 3.2
• different operating systems: OS X, Debian, Ubuntu and Windows

Right now I am using pytest

Question: how can I implement this easily and preferably making the results publicly available and having integrated with github, so anyone who pushes will know the results.

Note: I am already aware about https://travis-ci.org/ but this seems to be missing the cross-platform part, which is essential in this case.

Another option I was considering was to use Jenkins, but I don't know how to provide the matrix testing on it.

Answer 1

I have used Jenkins, and I would recommend it. It has a plethora of plugins, and is very configurable.

I have used it for running projects over windows/linux/mac/mobile platforms, for sanity, unit, component, and regression tests.

It can support chaining of projects and tests, fingerprinting of items to be monitored as they progress your testing environment and also you can set up users and keep track of changes.

You can use it for production and for testing at the same time, hooking it up to your git repository, any change you make is automatically run through all the gauntlets you want.

Answer 2

You can use tox to automate setting up virtual environments and running your tests across Python versions:

[tox]
envlist = py25,py26,py27,py32
[testenv]
commands=py.test

Tox supports Python versions 2.4 and up, as well as Jython and PyPy.

If you want to look at a real-world project that uses tox , take a look at the zope.configuration tox.ini configuration . The package includes excellent documentation on how to run the tox tests . These tests are automatically run by the Zope nightly test builders .

Configuring tox to run under Jenkins is trivial and fully documented.