MongoDB field level encryption in AWS Lambda

Question

I try to use MongoDB field level encryption with Lambda. I packaged the binary mongocryptd for Amazon Linux 2 with the lambda and made sure it exists in the current path. However connecting to the server does not work. I get the following error "MongoServerSelectionError: connect ECONNREFUSED 127.0.0.1:27020"

Running the code of the handler inside a docker image of Amazon Linux 2 works as expected. Running the lambda without the Autoencryption option also works as expected. So basically the lambda has access to the mongo cluster. I also tried to use a local master key provider – to no avail.

UPDATE: Spawning the mongocryptd process manually inside the lambda const child = spawn("mongocryptd", []) resulted in errors due to missing libraries that were missing in Lambda container (although present at the Amazon Linux 2 docker image). I added all the missing libraries, so I can manually spawn the process. However I still get the error above.

here is the code of my lambda

const { Binary, MongoClient } = require("mongodb");
const connectionString = "mongodb+srv://OMMITED.mongodb.net/development?retryWrites=true&w=majority";
const keyVaultNamespace = "development.__keyVault";
const base64KeyId = "OMMITED";
const path = require("path");

// add mongocryptd to $PATH
process.env.PATH = `${process.env.PATH}:${path.resolve(__dirname, "bin")}`;

const kmsProviders = {
  aws: {
    accessKeyId: "OMMITED",
    secretAccessKey: "OMMITED",
  },
};

const createSchema = () => {
  return {
    "development.test": {
      bsonType: "object",
      encryptMetadata: {
        keyId: [new Binary(Buffer.from(base64KeyId, "base64"), 4)],
      },
      properties: {
        foo: {
          encrypt: {
            bsonType: "string",
            algorithm: "AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic",
          },
        },
      },
    },
  };
};

module.exports.hello = async (event) => {
  const secureClient = new MongoClient(connectionString, {
    useNewUrlParser: true,
    useUnifiedTopology: true,
    autoEncryption: {
      keyVaultNamespace,
      kmsProviders,
      schemaMap: createSchema(),
    },
  });

  try {
    await secureClient.connect();
    const collection = secureClient.db("development").collection("test");
    const resp = await collection.find({}).toArray();
    console.log("RESP", JSON.stringify(resp));
  } catch (error) {
    console.log(error);
  }

  return {
    statusCode: 200,
    body: JSON.stringify(resp),
  };
};

UPDATE 2 Spawning the mongocryptd process manually in a lambda and logging stdout gives me the following output. There seems to be something wrong in the first output.

START RequestId: 52efeae4-3da9-46d2-b04a-72ba18c92d87 Version: $LATEST
2020-08-29T07:44:54.379Z    52efeae4-3da9-46d2-b04a-72ba18c92d87    INFO    {"t":{"$date":"2020-08-29T07:44:54.372+00:00"},"s":"I",  "c":"CONTROL",  "id":23103,   "ctx":"SignalHandler","msg":"Ignoring error from setting thread name","attr":{"error":"Operation not permitted"}}

2020-08-29T07:44:54.381Z    52efeae4-3da9-46d2-b04a-72ba18c92d87    INFO    {"t":{"$date":"2020-08-29T07:44:54.380+00:00"},"s":"I",  "c":"CONTROL",  "id":4615669, "ctx":"initandlisten","msg":"MongoCryptD starting","attr":{"pid":20,"port":27020,"socketFile":"/tmp/mongocryptd.sock","architecture":"64-bit","host":"169.254.13.13"}}

2020-08-29T07:44:54.381Z    52efeae4-3da9-46d2-b04a-72ba18c92d87    INFO    {"t":{"$date":"2020-08-29T07:44:54.381+00:00"},"s":"I",  "c":"CONTROL",  "id":23403,   "ctx":"initandlisten","msg":"Build Info","attr":{"buildInfo":{"version":"4.4.0","gitVersion":"563487e100c4215e2dce98d0af2a6a5a2d67c5cf","openSSLVersion":"OpenSSL 1.0.2k-fips  26 Jan 2017","modules":["enterprise"],"allocator":"tcmalloc","environment":{"distmod":"amazon2","distarch":"x86_64","target_arch":"x86_64"}}}}
{"t":{"$date":"2020-08-29T07:44:54.381+00:00"},"s":"I",  "c":"CONTROL",  "id":51765,   "ctx":"initandlisten","msg":"Operating System","attr":{"os":{"name":"Amazon Linux release 2 (Karoo)","version":"Kernel 4.14.177-104.253.amzn2.x86_64"}}}
{"t":{"$date":"2020-08-29T07:44:54.381+00:00"},"s":"I",  "c":"CONTROL",  "id":21951,   "ctx":"initandlisten","msg":"Options set by command line","attr":{"options":{"processManagement":{"idleShutdownTimeoutSecs":60}}}}

2020-08-29T07:44:54.381Z    52efeae4-3da9-46d2-b04a-72ba18c92d87    INFO    {"t":{"$date":"2020-08-29T07:44:54.381+00:00"},"s":"I",  "c":"CONTROL",  "id":24225,   "ctx":"initandlisten","msg":"Using lock file","attr":{"file":"/var/task/mongocryptd.pid"}}
{"t":{"$date":"2020-08-29T07:44:54.380+00:00"},"s":"I",  "c":"CONTROL",  "id":23103,   "ctx":"SignalHandler","msg":"Ignoring error from setting thread name","attr":{"error":"Operation not permitted"}}

2020-08-29T07:44:54.382Z    52efeae4-3da9-46d2-b04a-72ba18c92d87    INFO    {"t":{"$date":"2020-08-29T07:44:54.382+00:00"},"s":"E",  "c":"CONTROL",  "id":24231,   "ctx":"initandlisten","msg":"Failed to open pid file, exiting","attr":{"error":{"code":98,"codeName":"DBPathInUse","errmsg":"Unable to create/open the lock file: /var/task/mongocryptd.pid (Read-only file system). Ensure the user executing mongod is the owner of the lock file and has the appropriate permissions. Also make sure that another mongod instance is not already running on the /var/task directory"}}}

END RequestId: 52efeae4-3da9-46d2-b04a-72ba18c92d87
REPORT RequestId: 52efeae4-3da9-46d2-b04a-72ba18c92d87  Duration: 3014.10 ms    Billed Duration: 3100 ms    Memory Size: 1024 MB    Max Memory Used: 137 MB Init Duration: 360.02 ms    

Answer 1

The drivers are required to silence output from mongocryptd, making spawn problems difficult to debug. You can:

• Pass --logpath argument to mongocryptd as specified here
• Patch the driver to remove stdout/stderr redirects of mongocryptd process