How to solve dependency conflicts in requirements.txt file in Django deployment

Question

I am trying to deploy my Django app on GCP using google appengine. First I deployed the app after testing on localhost by following this documentation by Google appengine. deployed the app using gcloud app deploy

But there is some issue and the server is not running showing the error as 502 Bad Gateway

Then I checked the logs and then realized that I forgot to upload the requiremts.txt file. the uploaded file and tried to deploy the app again.

But got an error as

ERROR: Cannot install -r requirements.txt (line 19), -r requirements.txt (line 21), -r requirements.txt (line 27) and grpcio==1.48.1 because these package versions have conflicting dependencies.

Here is some dependency conflict between modules Gcloud suggested a documentation https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts to solve this but I'm not getting it actually how to solve the conflict of modules in requirements.txt

Here is the requirements.txt file

APScheduler==3.6.3
asgiref==3.5.2
backports.zoneinfo==0.2.1
beautifulsoup4==4.11.1
cachetools==4.2.2
certifi==2022.6.15
charset-normalizer==2.1.1
dill==0.3.5.1
Django==4.0.6
django-environ==0.9.0
django-social-share==2.3.0
environ==1.0
google==3.0.0
google-api-core==2.10.0
google-auth==2.11.0
google-cloud-secret-manager==2.12.4
google-cloud-speech==2.15.1
googleapis-common-protos==1.56.4
grpc-google-iam-v1==0.12.4
grpcio==1.48.1
grpcio-status==1.48.1
idna==3.3
Pillow==9.2.0
proto-plus==1.22.1
protobuf==4.21.5
psycopg2==2.9.3
pulumi==3.39.3
pyasn1==0.4.8
pyasn1-modules==0.2.8
pytz==2022.2.1
pytz-deprecation-shim==0.1.0.post0
PyYAML==6.0
requests==2.28.1
rsa==4.9
semver==2.13.0
six==1.16.0
soupsieve==2.3.2.post1
sqlparse==0.4.2
tornado==6.2
tzdata==2022.1
tzlocal==4.2
urllib3==1.26.12

And an error log

Updating service [default]...failed.
ERROR: (gcloud.app.deploy) Error Response: [9] Cloud build 7ac526d7-15d5-45f6-afe7-727b4216f79d status: FAILURE
.... This could take a while.
INFO: pip is looking at multiple versions of google-api-core to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of google to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of environ to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of django-social-share to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of django-environ to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of django to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of dill to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of charset-normalizer to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of certifi to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of cachetools to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of beautifulsoup4 to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of backports-zoneinfo to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of <Python from Requires-Python> to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of asgiref to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of apscheduler to determine which version is compatible with other requirements. This could take a while.
ERROR: Cannot install -r requirements.txt (line 19), -r requirements.txt (line 21), -r requirements.txt (line 27) and grpcio==1.48.1 because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested grpcio==1.48.1
    grpc-google-iam-v1 0.12.4 depends on grpcio<2.0.0dev and >=1.0.0
    grpcio-status 1.48.1 depends on grpcio>=1.48.1
    pulumi 3.39.3 depends on grpcio==1.47

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

Please help me to solve this error Thanks

Answer 1

I would use something like poetry to manage your dependencies while you work on your app. It will actually help you not to run into conflicts and solve most of them for you automatically.

Then when you are ready to deploy your app you can do:

poetry export -f requirements.txt --output requirements.txt

And poetry will generate a requirements.txt file that you can upload to GCP