dev_appserver.py app.yaml produces: ImportError: Importing the multiarray numpy extension module failed

Question

I run this command:

dev_appserver.py app.yaml

and I get an error:

Traceback (most recent call last):
  File "C:\Users\sehrlich\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\google_appengine\google\appengine\runtime\wsgi.py", line 240, in Handle
    handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
  File "C:\Users\sehrlich\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\google_appengine\google\appengine\runtime\wsgi.py", line 299, in _LoadHandler
    handler, path, err = LoadObject(self._handler)
  File "C:\Users\sehrlich\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\google_appengine\google\appengine\runtime\wsgi.py", line 85, in LoadObject
    obj = __import__(path[0])
  File "C:\Users\sehrlich\Desktop\ToolKit\Application Projects\Tableau Web Data Connector Improvement\tableauextensions\main.py", line 2, in <module>
    from Get_Data import get_data, build_connection, run_sql
  File "C:\Users\sehrlich\Desktop\ToolKit\Application Projects\Tableau Web Data Connector Improvement\tableauextensions\Get_Data.py", line 1, in <module>
    import numpy as np
  File "C:\Users\sehrlich\Desktop\ToolKit\Application Projects\Tableau Web Data Connector Improvement\tableauextensions\lib\numpy\__init__.py", line 142, in <module>
    from . import add_newdocs
  File "C:\Users\sehrlich\Desktop\ToolKit\Application Projects\Tableau Web Data Connector Improvement\tableauextensions\lib\numpy\add_newdocs.py", line 13, in <module>
    from numpy.lib import add_newdoc
  File "C:\Users\sehrlich\Desktop\ToolKit\Application Projects\Tableau Web Data Connector Improvement\tableauextensions\lib\numpy\lib\__init__.py", line 8, in <module>
    from .type_check import *
  File "C:\Users\sehrlich\Desktop\ToolKit\Application Projects\Tableau Web Data Connector Improvement\tableauextensions\lib\numpy\lib\type_check.py", line 11, in <module>
    import numpy.core.numeric as _nx
  File "C:\Users\sehrlich\Desktop\ToolKit\Application Projects\Tableau Web Data Connector Improvement\tableauextensions\lib\numpy\core\__init__.py", line 26, in <module>
    raise ImportError(msg)
ImportError:
Importing the multiarray numpy extension module failed.  Most
likely you are trying to import a failed build of numpy.
If you're working with a numpy git repo, try `git clean -xdf` (removes all
files not under version control).  Otherwise reinstall numpy.

I have installed and uninstalled NumPy. They app works fine when I run something like

python -m flask run 

and it uses NumPy no problem. Can't figure out what the issue is.

Answer 1

As Dan Cornilescu said, GAE Standard can't use libraries with code compiled in C [1] [2] :

[From 1] You can use third-party libraries that are pure Python code with no C extensions

[From 2] The interpreter cannot load Python services with C code; it is a "pure" Python environment.

NumPy is one of those cases, you can see it in their Git Repo [3] and in Wikipedia [4] (Written in: Python, C). Also, check the first answer to this SO question .

Curiously enough, I found a version on NumPy only based on "pure" Python called "TinyNumPy" [5] that you could use in GAE Standard. This are its limitations according to their Git Repo:

• ndarray.flat iterator cannot be indexed (it is a generator).
• No support for Fortran order.
• Support for data types limited to bool, uin8, uint16, uint32, uint64, int8, int16, int32, int64, float32, float64.
• Functions that calculate statistics on the data are much slower, since the iteration takes place in Python.
• Assigning via slicing is usually pretty fast, but can be slow if the striding is unfortunate.

In a nutshell, either you use GAE Flex or try to avoid NumPy.

Answer 2

Since you want to use the GAE-supplied numpy then:

• you shouldn't have it installed inside your application code (the traceback indicates it's running from the app's lib dir, where vendored-in libraries go)

• you should request it in your app.yaml 's libraries section:

   libraries: - name: numpy version: "1.6.1" 

• you should also have the requested numpy version installed locally on your system (but not in the app dir) so that the development server can use it as numpy is one of the libraries with such requirement, see Using built-in bundled libraries with the local development server :

Many of the built-in libraries provided by the runtime are automatically available to the local development server. However, the following built-in libraries must be installed locally before you can use them with the local development server:

...

• numpy