导入错误：ApacheBeam 上没有名为“google.cloud”的模块

Question

I get an import error with importing apache beam's google datastore api.I have one version of Python 3 installed on my Windows 10 64-bit system.

Can somebody help me? I have try to solve it but i can't

# -*- coding: utf-8 -*-
import apache_beam as beam
from apache_beam.io.gcp.datastore.v1new.datastoreio import ReadFromDatastore, WriteToDatastore
from apache_beam.options.pipeline_options import GoogleCloudOptions
from apache_beam.options.pipeline_options import PipelineOptions

my error message like this

Traceback (most recent call last):
  File "C:\Work\test.py", line 3, in <module>
    from apache_beam.io.gcp.datastore.v1new.datastoreio import ReadFromDatastore, WriteToDatastore
  File "C:\Work\VENV\venv\lib\site-packages\apache_beam\io\gcp\datastore\v1new\datastoreio.py", line 40, in <module>
    from apache_beam.io.gcp.datastore.v1new import helper
  File "C:\Work\VENV\venv\lib\site-packages\apache_beam\io\gcp\datastore\v1new\helper.py", line 33, in <module>
    from google.cloud import environment_vars
ModuleNotFoundError: No module named 'google.cloud'

Here is pip list

Package                  Version
------------------------ ----------
apache-beam              2.17.0
avro-python3             1.9.1
cachetools               4.0.0
certifi                  2019.11.28
chardet                  3.0.4
crcmod                   1.7
dill                     0.3.0
docopt                   0.6.2
fastavro                 0.21.24
future                   0.18.2
google-api-core          1.16.0
google-api-python-client 1.7.11
google-auth              1.10.1
google-auth-httplib2     0.0.3
googleapis-common-protos 1.51.0
grpcio                   1.26.0
hdfs                     2.5.8
httplib2                 0.12.0
idna                     2.8
mock                     2.0.0
numpy                    1.18.1
oauth2client             3.0.0
pbr                      5.4.4
pip                      19.0.3
protobuf                 3.11.2
pyarrow                  0.15.1
pyasn1                   0.4.8
pyasn1-modules           0.2.8
pydot                    1.4.1
pymongo                  3.10.1
pyparsing                2.4.6
python-dateutil          2.8.1
pytz                     2019.3
requests                 2.22.0
rsa                      4.0
setuptools               40.8.0
six                      1.14.0
uritemplate              3.0.1
urllib3                  1.25.7

Please help me with my problem. Thank you before

Answer 1

我安装了 apache-beam[gcp] 并且运行得非常好。

Answer 2

the "google-cloud"-Pacakge are missing.

Run:

pip install google-cloud

Answer 3

You need "--requirements_file" flag to your Option on ApacheBeam.

as below

options = MyOptions(flags = ["--requirements_file", "./requirements.txt"])

Answer 4

I was facing the same issue. But later the issue has been solved with the below steps:

1. Read following nice article on virtualenv, this will help in later steps,

2. Create virtual environment (venv env)

3. Activate virtual env

4. Install cloud-dataflow using following link: (this brings in apache_beam) https://cloud.google.com/dataflow/docs/quickstarts/quickstart-python

5. Run the sample