在gunicorn上部署flask应用程序，模块对象没有属性

Question

I am trying to deploy my Flask application to gunicorn and I am getting following error.

AttributeError: 'module' object has no attribute 'MyTopics'

When I run server simply the Flask server, everything works fine:

python http_server/server.py

But when I run it under gunicorn, then I am getting the AttributeError

gunicorn -c http_server.config http_server.server:app

Loading of my server app looks like:

import logging
import sys
import os
import json
import time
from functools import wraps

from flask import Flask, request, jsonify

from my_lda import MyTopics

import config

logger = logging.getLogger(__name__)
app = Flask(__name__)

class Server(object):
    def __init__(self, lda_model_path):
        self.lda_model_path = lda_model_path
        self.lda_model = self.load_to_lda_model()

    def load_to_lda_model(self, path=None):
        """
        load the lda model from the specific path
        """

        if path is None:
            path = self.lda_model_path

        logger.info("loading LDA model from %r" % path)
        lda_model = MyTopics.load(path)
        logger.info("loaded LDA model %s", lda_model)
        return lda_model

logging.basicConfig(format='%(processName)s %(process)s:-%(asctime)s : %(levelname)s : %(module)s:%(lineno)d : %(funcName)s(%(threadName)s) : %(message)s')
logging.root.setLevel(level=logging.INFO)
logging.info("running %s" % ' '.join(sys.argv))

if 'MODEL_PATH' in os.environ:
    path = os.environ['MODEL_PATH']
else:
    path = config.MODEL_FILE_PATH
app.config['server'] = Server(path)

if __name__ == '__main__':
    app.run()

Calling load is based on gensim.utils.SaveLoad, but I've tried also standard pickling, but with no difference.

My ideas so far: Can gunicorn run different Python (I'm launching it from the same virtualenv)? Is it possible that gunicorn does not see some packages installed via pip install -e .

EDIT:

Adding the project structure:

• http_server
  • __init__.py
  • server.py
  • config.py
• data
  • model.pkl
• __init__.py
• my_lda.py
• setup.py

Answer 1

So the problem was related to the way I was pickling the MyTopics object. More details are described in pickle can't import a module that exists?