AWS Elastic Beanstalk - Flask App Cannot Import Custom Module

Question

I am trying to host a Flask app on AWS Elastic Beanstalk in Python 3.6, but no matter what I try, the application's url leads to a 500: Internal Server Error.

The structure of my app is as follows:

myApp: -application.py -mytransformers.py -requirements.txt

I took a look at the logs, and the separate python script I need that contains custom Scikit-learn transformers (mytransformers.py) cannot be found by wsgi:

AttributeError: Can't get attribute 'MyTransformer' on <module '__main__' (built-in)>

I need these because I have pickled Scikit-learn models in an S3 repo that I import in application.py that require custom transformers.

I tried to move these transformer definitions to my main application.py script but it still returns the same error.

I think that the wsgi script is unable to find these transformer definitions, and I have not found much in the way of people with a similar problem.

Edit 1

I tried adding a __init__.py file to my application directory I am still getting the same error. What I am trying to do is load a Scikit-Learn pipeline using Pickle that contains some custom transformers, however when I try to load them it throws the error about not being able to find the custom transformers, even though I imported them at the start of the script. I tried putting the class definitions for the transformers in the application.py script but this still does not solve the error.

Edit 2

I am almost certain the error is in EB using WSGI to manage the Flask app. My app does run locally, its just the WSGI that seems not able to find the helper functions. I found a similar post using Azure that had a similar problem of WSGI not finding functions defined in the application file: Error with WSGI when deploying Flask App to Azure

Answer 1

I'd start by adding a blank file named __init__.py to the root of your application. Right now the Flask app in application.py can't detect your transformers because they are not on your python path, and they're not part of the same package.

Adding __init__.py will make your transformers importable using: from mytransformers import MyTransformer because it will add them to the application package.

Edit: Does it run locally?