Lambda 未加载加密共享库

Question

I'm using the cryptography library in AWS Lambda. I've compiled the package using pip in an Amazon Linux VM. I have uploaded the package as a layer. Either way, every time I call the library, I have an error which is not descriptive at all:

Unable to import module 'lambda_function': libffi-ae16d830.so.6.0.4: cannot open shared object file: No such file or directory

As you can see, the error is not about not finding the lib, is another shared module which I haven't been able to find.

Here's an example of the code I'm trying to execute on Lambda:

from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.kdf.pbkdf2 import PBKDF2HMAC
from cryptography.fernet import Fernet

password_provided = "test123" 
password = password_provided.encode() 
salt = b'test_' 
kdf = PBKDF2HMAC(
    algorithm=hashes.SHA256(),
    length=32,
    salt=salt,
    iterations=100000,
    backend=default_backend()
)
key = base64.urlsafe_b64encode(kdf.derive(password)) 


message = "message from db".encode()

f = Fernet(key)
encrypted = f.encrypt(message)

print(encrypted)

f = Fernet(key)
decrypted = f.decrypt(encrypted)

print(decrypted.decode("utf-8"))

It's not the first time I've compiled a library to work on AWS Lambda, but in this case even I compile the cryptography lib. What should I add or change?

Edit: I've found out the library was missing in the zip file I've created, as is inside a hidden folder. I zipped using '.'instead of '*' but now I'm running with a new problem: When I run the lambda, I get this:

Unable to import module 'lambda_function': /opt/cryptography/hazmat/bindings/_constant_time.so: undefined symbol: PyInt_FromLong

any idea?

Answer 1

Even I faced the same issue, while zipping I forgot to include the hidden files(.libs_cffi_backend) in the site-packages. After Including it, I didn't see this error.

Answer 2

As the library you're using requires native libraries, you have to pack the native.so files as well with the layer. I ran into a similar issue while trying to run wkhtmltopdf on aws lambda.

The binaries for the library has to be compiled in the same environment as a lambda instance. Lambda gets booted up using AWS Linux.

You can either boot up an EC2 running AmazonLinux or use docker, easiest way is to boot up a docker container.

$ sudo docker run -it amazonlinux bash

Now you need to download/unpack all.so files into a directory then zip it. Also, make sure to keep all.so files inside a folder called lib inside the zip. After zipping, the zip should look something similar to this:

.
├── lib
│   ├── libcrypto.so.10
│   ├── libcrypto.so.1.0.2k
│   ├── libfontconfig.so.1
│   ├── libfontconfig.so.1.7.0
.......

Then you can just zip it and upload it as a layer. It will be uploaded to /opt/ in your Lambda Container. AWS looks for library files under /opt/lib amongst many other locations .

The challenging part for you would be to figure out how to get all the required.so files in order for your dependency to run properly.

Answer 3

Previously I manually packed the archive, deployed with aws cloudformation deploy , and had the same problem. I then switched to using sam build --use-container and it worked. My suspicion is that it works because SAM uses a specific container. This is the container it uses: https://gallery.ecr.aws/sam/build-python3.7