如何在Windows开发环境中使用python设置气流

Question

At work we are completing a PoC of Airflow by AirBnB. Standard operating practice is to decouple the development (ie DAG creation and python scripts etc) from the runtime environment. Ie we are not allowed to script directly on the servers (bad practice anyway).

As such, I have configured intelliJ to work with Python (works as expected) in Windows (we are not allowed to have unix workstations) but I cannot find a way to install airflow in Windows. I am new to Python with a Java development background.

As such, how do I setup a local development environment IDEA ultimate for AirBnb/Airflow (there is no documentation ?

Answer 1

I have it working and compiling with the following step: The steps below are for using intelliJ Ultimate edition, download the python modules.

1. Download the python extension for InttelliJ Ultimate edition or PyCharm
2. Configure and install python 2.7 for Windows.
3. Add the SDK to IDEA
4. Configure the following proxy details in environment variables to install modules using PIP -> HTTP_PROXY=http : // user:password@your-company-proxy.com:8080 HTTPS_PROXY=http : // user:password@your-company-proxy.com:8080 (remove spaces, not allowed to post multiple links)
5. Microsoft Visual C++ 9.0 is required (Unable to find vcvarsall.bat). Get it from http://aka.ms/vcpython27
   C:_downloads>msiexec /i VCForPython27.msi ALLUSERS=1
6. pip install airflow
7. Create a new project, configure it as a python module.
8. Download the GIT repository -> airflow
9. Configure your path correctly, also set a python home and AIRFLOW_HOME.