Heroku and Flask - ModuleNotFoundError: No module named 'fetch_data'

Question

I am trying to deploy my flask+react app to heroku but its crashing and giving me the following error:

ModuleNotFoundError: No module named 'fetch_data'

Here is a snippet of the logs:

2021-03-23T19:34:05.332761+00:00 app[web.1]: Traceback (most recent call last):
2021-03-23T19:34:05.332763+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.7/site-packages/gunicorn/arbiter.py",
 line 583, in spawn_worker
2021-03-23T19:34:05.332764+00:00 app[web.1]: worker.init_process()
2021-03-23T19:34:05.332765+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.7/site-packages/gunicorn/workers/base
.py", line 119, in init_process
2021-03-23T19:34:05.332765+00:00 app[web.1]: self.load_wsgi()
2021-03-23T19:34:05.332765+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.7/site-packages/gunicorn/workers/base
.py", line 144, in load_wsgi
2021-03-23T19:34:05.332766+00:00 app[web.1]: self.wsgi = self.app.wsgi()
2021-03-23T19:34:05.332766+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.7/site-packages/gunicorn/app/base.py"
, line 67, in wsgi
2021-03-23T19:34:05.332767+00:00 app[web.1]: self.callable = self.load()
2021-03-23T19:34:05.332767+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.7/site-packages/gunicorn/app/wsgiapp.
py", line 49, in load
2021-03-23T19:34:05.332767+00:00 app[web.1]: return self.load_wsgiapp()
2021-03-23T19:34:05.332768+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.7/site-packages/gunicorn/app/wsgiapp.
py", line 39, in load_wsgiapp
2021-03-23T19:34:05.332768+00:00 app[web.1]: return util.import_app(self.app_uri)
2021-03-23T19:34:05.332768+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.7/site-packages/gunicorn/util.py", li
ne 358, in import_app
2021-03-23T19:34:05.332769+00:00 app[web.1]: mod = importlib.import_module(module)
2021-03-23T19:34:05.332769+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.7/importlib/__init__.py", line 127, i
n import_module
2021-03-23T19:34:05.332770+00:00 app[web.1]: return _bootstrap._gcd_import(name[level:], package, level)
2021-03-23T19:34:05.332771+00:00 app[web.1]: File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
2021-03-23T19:34:05.332771+00:00 app[web.1]: File "<frozen importlib._bootstrap>", line 983, in _find_and_load
2021-03-23T19:34:05.332771+00:00 app[web.1]: File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
2021-03-23T19:34:05.332772+00:00 app[web.1]: File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
2021-03-23T19:34:05.332772+00:00 app[web.1]: File "<frozen importlib._bootstrap_external>", line 728, in exec_module
2021-03-23T19:34:05.332772+00:00 app[web.1]: File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_remove
d
2021-03-23T19:34:05.332773+00:00 app[web.1]: File "/app/api/api.py", line 2, in <module>
2021-03-23T19:34:05.332773+00:00 app[web.1]: from fetch_data import fetch_data
2021-03-23T19:34:05.332773+00:00 app[web.1]: ModuleNotFoundError: No module named 'fetch_data'
2021-03-23T19:34:05.332985+00:00 app[web.1]: [2021-03-23 19:34:05 +0000] [9] [INFO] Worker exiting (pid: 9)
2021-03-23T19:34:05.391634+00:00 app[web.1]: [2021-03-23 19:34:05 +0000] [4] [INFO] Shutting down: Master
2021-03-23T19:34:05.392273+00:00 app[web.1]: [2021-03-23 19:34:05 +0000] [4] [INFO] Reason: Worker failed to boot.
2021-03-23T19:34:05.544193+00:00 heroku[web.1]: Process exited with status 3
2021-03-23T19:34:05.617532+00:00 heroku[web.1]: State changed from up to crashed

Now fetch_data is a package I have made that is in the api folder. Below is the directory structure of the api folder:

.
|-- __pycache__
|   |-- api.cpython-37.pyc
|   `-- fetch_data.cpython-37.pyc
|-- api.py
|-- fetch_data
|   |-- __init__.py
|   `-- fetch_data.py
|-- static
|   `-- react
|-- templates
|   `-- index.html
|-- test.py
`-- venv
    |-- Include
    |-- Lib
    |-- Scripts
    `-- pyvenv.cfg

fetch_data.py is a flask blueprint that I have called within api.py.

this is a snippet of api.py:

from flask import Flask, jsonify, render_template
from fetch_data import fetch_data
from nsepy import get_history
from datetime import date,timedelta
import pandas as pd
import os
import json

app = Flask(__name__)
app.register_blueprint(fetch_data.fetch_api)

@app.route('/')
def index():
    return render_template("index.html", flask_token = "Hello World")
.
.
.

if __name__ == '__main__':
    port =int(os.environ['PORT'])
    app.run(host='0.0.0.0', debug=False, port= port)

This is the code for fetch_data.py :

from flask import Blueprint
from nsetools import Nse

fetch_api = Blueprint('fetch_api', __name__, url_prefix='/fetch')

@fetch_api.route('/api/<string:ticker>', methods = ['GET'])
def fetch_data(ticker):
    nse = Nse()

    return nse.get_quote(ticker, as_json = True)

@fetch_api.route('/gainers', methods = ['GET'])
def fetch_gainers():
    nse = Nse()
    return nse.get_top_gainers(as_json = True)

@fetch_api.route('/losers', methods = ['GET'])
def fetch_losers():
    nse = Nse()
    return nse.get_top_losers(as_json = True)

I have seen some answers mentioned that there may be a problem with the Procfile , but I think mine looks fine.

web: gunicorn -w 1 -b 0.0.0.0:$PORT api.api:app

I'm still new to flask and heroku so I apologise if the solution to this question is obvious. Thanks in advance for the help.

Answer 1

It is best to use this kind of structure where every concern is separate:

project
  |----- app.py
  |----- test.py
  |----- .flaskenv
  |----- app/
           |----- __init__.py
           |----- fetch_data/
                      |----- __init__.py
                      |----- routes.py
           |----- main/
                   |----- __init__.py
                   |----- routes.py
           |----- static/
                    |----- # <your-static-files>
           |----- templates/
                      |----- index.html

app.py is the entry point to your application. It will have:

from app import create_app

app = create_app()

# ...

Your routes.py will look like this(renamed fetch_api.py to routes.py to make it a bit obvious about what it does in the fetch api package):

# all your fetch_data.py code
# ...

@bp.route('/losers', methods = ['GET'])
def fetch_losers():
    nse = Nse()
    return nse.get_top_losers(as_json = True)

__init__.py from fetch_data package will define your blueprint

from flask import Blueprint

bp = Blueprint('fetch_api', __name__)

from app.fetch_data import routes

The main package will hold all other details about the app. In main 's route, you will have:

# Remember to initialize the main package appropriately 
# so as to use Flask's Blueprint
# ...

@bp.route('/')
def index():
    return render_template("index.html", flask_token = "Hello World")

# ...

The __init__.py file is your application instance:

from flask import Flask
# all your other imports go here

app = Flask(__name__)

# ...

def create_app(config_class=Config):
    app = Flask(__name__)
    app.config.from_object(config_class)

    from app.fetch_api import bp as fetch_bp
    app.register_blueprint(fetch_bp, url_prefix='/fetch')

    # ...

    return app

from app import models  # add/remove if modules exist/don't exist in your app folder

This will get rid of any importation issues you might have with the application packages.