ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. Dataflow python custom template

Question

when I create a custom dataflow template(python), yesterday the template works fine but today when i execute the template return an error and i can't fixed yet.

TypeError: Descriptors cannot not be created directly If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0. (I changed version in requirements.txt but nothigs works). And then appear this text. google-cloud-vision 1.0.0 requires google-api-core[grpc]<2.0.0dev,>=1.14.0, but you have google-api-core 2.10.0 which is incompatible google-cloud-videointelligence 1.16.1 requires google-api-core[grpc]<2.0.0dev,>=1.14.0, but you have google-api-core 2.10.0 which is incompatible google-cloud-spanner 1.19.1 requires google-api-core[grpc,grpcgcp]<2.0.0dev,>=1.14.0, but you have google-api-core 2.10.0 which is incompatible google-cloud-recommendations-ai 0.2.0 requires google-api-core[grpc]<2.0.0dev,>=1.22.2, but you have google-api-core 2.10.0 which is incompatible google-cloud-language 1.3.0 requires google-api-core[grpc]<2.0.0dev,>=1.14.0, but you have google-api-core 2.10.0 which is incompatible google-cloud-datastore 1.15.3 requires google-api-core[grpc]<2.0.0dev,>=1.14.0, but you have google-api-core 2.10.0 which is incompatible google-cloud-core 1.7.2 requires google-api-core<2.0.0dev,>=1.21.0, but you have google-api-core 2.10.0 which is incompatible. google-cloud-bigtable 1.7.0 requires google-api-core[grpc]<2.0.0dev,>=1.14.0, but you have google-api-core 2.10.0 which is incompatible apache-beam 2.36.0 requires protobuf<4,>=3.12.2, but you have protobuf 4.21.5 which is incompatible.

Answer 1

I had the same issue. After looking at TypeError: Descriptors cannot not be created directly I was able to resolve it by adding the line protobuf==3.20.1 to the end of my requirements.txt file, that caused an override to the protobuf pip installation to this specific version, as some other package was upgrading it.