从 Compute Engine 实例模板创建 Google Batch 作业，未正确导入自定义 Python 库

Question

I am trying to start a Google Batch Job, from an instance template containing some custom Python packages in it, but the Batch Job always fails, telling me the package I imported does not exist. These are the steps I followed ( read all of them carefully before rushing towards an answer ):

1. Create a VM instance from a public image ; for reference, let's call it source-vm . Start and connect to that VM through SSH.
2. In source-vm , install spaCy , running the following commands in CLI:

sudo apt update
sudo apt install python3-pip
sudo pip install spacy==3.2.1
sudo python -m spacy download en_core_web_sm

3. Build /scripts/test.py on source-vm . This can be considered as the main script to be run later on, in the Google Batch Job ( myconfig.json , on Step "7"):

import spacy
from argparse import ArgumentParser, ArgumentDefaultsHelpFormatter

# Read arguments from CLI
parser = ArgumentParser(formatter_class=ArgumentDefaultsHelpFormatter)
parser.add_argument(
    "-t", "--task_count",
    type=str,
    default="0",
    help="Task count",
    choices={"0", "1", "2", "3", "4", "5"}
)
args = vars(parser.parse_args())
task_count = args["task_count"]

# Read data from Google Cloud Storage mounted data
MOUNTED_GCS_URI="/mnt/disks/test-bucket/input-test/sample-{}.txt".format(task_count.zfill(12))
with open(MOUNTED_GCS_URI,"r") as f:
    mytext = f.read()

# Import a test spaCy model
nlp=spacy.load("en_core_web_sm")

# NLP process: Entity Extraction
doc=nlp(mytext)

# Gather all found entities
found_entities=[]
for ent in doc.ents:
    found_entities.append(
        {"word":doc.text[ent.start_char:ent.end_char], "label":ent.label_}
    )

# Print results
print({"text":doc.text, "entities":found_entities})

4. Power-off source-vm and with it, create a machine image from a VM ; for reference, let's call it base-image .
5. Create an instance template based on an existing instance ; for this purpose, I used both source-vm and base-image . For reference, let's call it my-instance-template .
6. ( OPTIONAL ) Create a VM instance from an instance template as a quick test for my-instance-template . This second VM instance was called test-vm . After creating test-vm , I started and connected to it through SSH, then ran the following commands:

printf '\nINSTALLATION LOCATIONS, FOR PYTHON, PIP & SPACY:\n'
which python3 && which pip && which spacy
printf '\nVERSIONS, FOR PYTHON, PIP & SPACY:\n'
python3 --version && pip --version && python3 -c 'import spacy;print("spaCy version:",spacy.__version__)'
printf '\nWHAT IS IN PATH:\n'
echo $PATH

Which did not trigger any error, and got the following message:

INSTALLATION LOCATIONS, FOR PYTHON, PIP & SPACY:
/usr/bin/python3
/usr/bin/pip
/usr/local/bin/spacy

VERSIONS, FOR PYTHON, PIP & SPACY:
Python 3.9.2
pip 20.3.4 from /usr/lib/python3/dist-packages/pip (python 3.9)
spaCy version: 3.2.1

WHAT IS IN PATH:
/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games

Notice how I did not have to install or do any additional setting on test-vm , this is expected.

7. After successfully testing my-instance-template , start a custom Google Batch Job :

gcloud batch jobs submit batch-job-1 \
  --location us-central1 \
  --config myconfig.json

Where myconfig.json is:

{
    "taskGroups": [
        {
            "taskSpec": {
                "runnables": [
                    {
                        "script": {
                            "text": "python3 /scripts/test.py -t ${BATCH_TASK_INDEX} >> /mnt/disks/test-bucket/output-test/output-${BATCH_TASK_INDEX}.txt"
                        }
                    }
                ],

                "volumes": [
                    {
                        "gcs": {
                            "remotePath": "test-bucket"
                        },
                        "mountPath": "/mnt/disks/test-bucket"
                    }
                ],

                "computeResource": {
                    "cpuMilli": 2000,
                    "memoryMib": 2000
                },
                "maxRetryCount": 0,
                "maxRunDuration": "600s"
            },
            "taskCount": 6,
            "parallelism": 2
        }
    ],
    "allocationPolicy": {
        "instances": [
            {
                "installGpuDrivers": false,
                "instanceTemplate": "my-instance-template"
            }
        ]
    },
    "labels": {
        "department": "my-department",
        "env": "testing"
    },
    "logsPolicy": {
        "destination": "CLOUD_LOGGING"
    }
}

This Batch Job failed, with the following error obtained from the Cloud Logs:

Traceback (most recent call last):
File "/scripts/test.py", line 1, in <module>
import spacy
ModuleNotFoundError: No module named 'spacy'

QUESTION:

Why my Google Batch Job from a Compute Engine Instance Template fails, telling me "it did not find spaCy" (Step "7"), but when the exact same Compute Engine Instance Template is used to build an isolate VM instance (Step "6"), then everything works OK and spaCy library is imported correctly?

Answer 1

I just solved this case, after realizing that all the packages installes in Step "2", were installed using sudo . Therefore, myconfig.jsonl runnable (Step "7"), should be slighlty modified as follows:

...
"taskSpec": {
    "runnables": [
                    {
                        "script": {
                            "text": "sudo python3 /scripts/test.py -t ${BATCH_TASK_INDEX} >> /mnt/disks/test-bucket/output-test/output-${BATCH_TASK_INDEX}.txt"
                        }
                    }
                ],
...