When creating a component in Azure ML studio, the component is not able to save files to output folder URI. The component transforms images and saves

Question

In Azure ML studio, we can build components to do different tasks in machine learning. I am creating a component that has one input: Input image folder (URI) and two output folders (URIs). The component takes images from input folder, transforms images using Pytorch and tries to save it to output folder.I am getting following error after executing command component from a yaml file.

Execution failed. User process 'python' exited with status code 1. Please check log file 'user_logs/std_log.txt' for error details.

Error: Traceback (most recent call last): File "prep_1.py", line 177, in main(args) File "prep_1.py", line 169, in main prepare_data_component(args.input_data, args.training_data, args.val_data) File "prep_1.py", line 114, in prepare_data_component image.save(save_path) File "/azureml-envs/azureml_7e9e1abac3aeb5e2560b92cd769d118a/lib/python3.7/site-packages/PIL/Image.py", line 2428, in save fp = builtins.open(filename, "w+b") OSError: [Errno 30] Read-only file system: '/mnt/azureml/cr/j/50e4xxxxxxxx25a02xxxxxx/cap/data-capability/wd/INPUT_input_data/train/chickens/trial.jpg'

I want to know how to write/save images to an output URI from a.py file executed as a command from yaml file

Answer 1

I am creating Azure ML components from a yaml file. Here's how a simple yaml file to create a component will look like:

prep.yaml

 $schema: https://azuremlschemas.azureedge.net/latest/commandComponent.schema.json type: command name: prep_image_classification_pytorch display_name: Data Preparation for Image Classification Pytorch inputs: input_data: type: uri_folder outputs: training_data: type: uri_folder val_data: type: uri_folder code: ./ command: python prep.py --input_data ${{inputs.input_data}} --training_data ${{outputs.training_data}} --val_data ${{outputs.val_data}} environment: conda_file: ./conda.yaml image: mcr.microsoft.com/azureml/openmpi3.1.2-ubuntu18.04

The command type will run the source python file which is prep.py

The prep.py file will take arguments from given yaml file (command line arguments) which we will parse through argparse library

prep.py

 ef parse_args(): # setup argparse parser = argparse.ArgumentParser() # add arguments parser.add_argument("--input_data", type=str, help="path of input data") parser.add_argument("--training_data", type=str, default="./", help="output path of train data") parser.add_argument("--val_data", type=str, default="./", help="output path of validation data") # parse args args = parser.parse_args() # return args return args def prepare_data_component( input_data: Input(type="uri_folder"), training_data: Output(type="uri_folder"), val_data: Output(type="uri_folder") ):

As shown in code above use "./default" for ouput URI folders. These folders will be created inside Azure MLs blob storage. Mentioning path of our choice inside azure blob storage did not work for me.