How to use poetry in Google Colab?

Question

Context

I am currently working on a team project, where we need to train neural.networks. Some members are working on their local computer, and some on Colab (for GPU usage). We need to have the same dependencies. I am already familiar in using poetry on a local computer, but not on Colab, and I was wondering how to use it in Colab.

So I did some tests, and I encountered some issues. Maybe I can find some answers here. Thank you in advance!

Issues

1st issue : poetry add <package> does not update pyproject.toml

I want to add a new package, suppose torch. According to poetry's documentation, to install a new package, we need to run poetry add <package> . Since I run this command for the first time, a virtual environment is created, as well as the poetry.lock . But no packages were installed. Moreover, the poetry.lock file is updated, but not the pyproject.toml . This happens only on Colab. I tried on my local computer, and the command indeed automatically update the pyproject.toml file as well.

2nd issue : poetry run pip install <package> does not update pyproject.toml

Instead, we can install the package with poetry run pip install <package> . I saw this command on the following GitHub gist . The packages are now installed in the virtual environment, but the pyproject.toml was not updated.

Here is a link to the Colab notebook I used for those tests.

Thank you again!

Answer 1

Hello for everyone reading this post.

I settled on a solution. The main issue was that pyproject.toml was not updated automatically, so I just decided to modify it by hand. Here is the steps for using poetry in Colab, whether you create your own poetry project, or cloning a repo on Github.

https://github.com/elise-chin-adway/poetry-and-colab/blob/main/Using_python_poetry_in_Google_Colab.ipynb

I don't know if it is the best solution or not, but I hope it will help other people:)