Unable to install packages in AWS EC2 instance target directory using pip install -t

Question

I am following this video on Building a Serverless Machine Learning Inference API with AWS Lambda

Inside my AWS logged in system, I made the directories, copy pasted the requirements.txt file and used the following commands:

pip3 install -t /mnt/efs/fs1/ml/lib --no-cache-dir -r /mnt/efs/fs1/ml/requirements.txt

doing so, gave me error:

ERROR: Exception:
Traceback (most recent call last):
  File "/usr/lib64/python3.7/shutil.py", line 566, in move
    os.rename(src, real_dst)
OSError: [Errno 18] Invalid cross-device link: '/tmp/pip-target-41qwbfdl/lib/python/zipp.py' -> '/mnt/efs/fs1/ml/lib/zipp.py'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 216, in _main
    status = self.run(options, args)
  File "/usr/lib/python3.7/site-packages/pip/_internal/cli/req_command.py", line 182, in wrapper
    return func(self, options, args)
  File "/usr/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 490, in run
    options.target_dir, target_temp_dir, options.upgrade
  File "/usr/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 548, in _handle_target_dir
    target_item_dir
  File "/usr/lib64/python3.7/shutil.py", line 580, in move
    copy_function(src, real_dst)
  File "/usr/lib64/python3.7/shutil.py", line 266, in copy2
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/usr/lib64/python3.7/shutil.py", line 121, in copyfile
    with open(dst, 'wb') as fdst:
PermissionError: [Errno 13] Permission denied: '/mnt/efs/fs1/ml/lib/zipp.py'

then I tried using the command and while installing torch it just prints * Killed .

I tried using

sudo chown ec2-user:ec2-user /mnt/efs/fs1/ml

but same issue. I also tried with sudo pip3 install but then could not install.

Answer 1

Seems your EC2 EFS target destiny is write-protected by default. You can undo that by following this aws tutorial: https://aws.amazon.com/pt/premiumsupport/knowledge-center/efs-enable-read-write-access/