apache-beam[gcp] issue in google colab

Question

I have an issue with running apache-beam[gcp] on google colab.

If I run the following cell

!pip install -q apache-beam[gcp]

import apache_beam as beam

with beam.Pipeline() as p:
  p | beam.Create([1, 2, 3]) | beam.Map(print)

I have the error:

FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/lib/python3.6/dist-packages/cachetools-4.1.0.dist-info/METADATA'

On the second run:

AttributeError: module 'apache_beam' has no attribute 'pipeline'

I was working with beam couple weeks ago in colab and the following flow was working fine. I tried resetting back to 2.21.0 and 2.20.0 - no luck.

Thanks!

UPD. With %%writefile magic it seems to be working. Looks like the issue is with colab imports

Answer 1

Try to upgrade cachetools:

!pip install cachetools --upgrade

It'll prompt you to restart the instance, so go ahead and restart it.

Re-try the operation.