Azure ML Environment: install a package from a file?

Question

I'm building an Environment object in the Azure Machine Learning service using the Python SDK, and everything is working fine except one Python package that installs from a URL. I'm wondering how to deal with it. This works:

my_env = Environment.from_conda_specification("trident", './environment.yml')

..but the Docker build fails on one of the packages, which installs from a file.

[91mERROR: Could not find a version that satisfies the requirement detectron2==0.1.3+cu101 (from -r /azureml-environment-setup/condaenv.s5fi23rw.requirements.txt (line 7)) (from versions: none) [0m[91mERROR: No matching distribution found for detectron2==0.1.3+cu101 (from -r /azureml-environment-setup/condaenv.s5fi23rw.requirements.txt (line 7)) [0m[91m

Here's how I would install that package manually:

python -m pip install detectron2 -f / https://dl.fbaipublicfiles.com/detectron2/wheels/cu101/torch1.5/index.html

and I have another package that should install from github, like this:

pip install -U 'git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI'

I'm pretty ignorant about yaml files: is there a way to include complicated syntax like that in a yaml file?

I'm hoping to not have to re-build the environment locally and install from it (which is an alternative option), because I would have to reinstall CUDA to do so.

Thanks

Answer 1

Updating because who likes downvotes and someone might find this useful.

AML uses same spec for installing Conda packages as per: https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#create-env-file-manually

OP could have applied something like:

# run: conda env create --file environment.yml
name: test-env
dependencies:
- python>=3.5
- anaconda
- pip
- pip:
 # works for regular pip packages
 - docx
 - gooey
 # for github 
 - git+https://github.com/facebookresearch/detectron2.git
 # and for wheels
 - https://dl.fbaipublicfiles.com/detectron2/wheels/cu101/index.html

However, I found it much easier to use a Docker image to load Detectron2 onto a container for AzureML because of CUDA/CuDNN compatibility fun.

FROM mcr.microsoft.com/azureml/base-gpu:openmpi3.1.2-cuda10.1-cudnn7-ubuntu18.04

RUN apt update && apt install git -y && rm -rf /var/lib/apt/lists/*

RUN /opt/miniconda/bin/conda update -n base -c defaults conda
RUN /opt/miniconda/bin/conda install -y cython=0.29.15 numpy=1.18.1

# Install cocoapi, required for drawing bounding boxes
RUN git clone https://github.com/cocodataset/cocoapi.git && cd cocoapi/PythonAPI && python setup.py build_ext install

RUN pip install --user tensorboard cython
RUN pip install --user torch==1.5+cu101 torchvision==0.6+cu101 -f https://download.pytorch.org/whl/torch_stable.html

RUN pip install azureml-defaults
RUN pip install azureml-dataprep[fuse]
RUN pip install pandas pyarrow
RUN pip install opencv-python-headless
RUN pip install detectron2 -f https://dl.fbaipublicfiles.com/detectron2/wheels/cu101/index.html```