Lambda INSERT INTO MySql function is adding to DB but still timing out

Question

I have a Lambda POST function which takes http request JSON body data and adds it to connected MySql database.

Code below:

var mysql = require("mysql");
var config = require("./config.json");

var pool = mysql.createPool({
  host: config.dbhost,
  user: config.dbuser,
  password: config.dbpassword,
  database: config.dbname,
  port: 3306,
  ssl: true,
});

exports.handler = (event, context, callback) => {
  context.callbackWaitsForEmptyjsonLoop = false;
  pool.getConnection(function (err, connection) {
    const col1 = event.col1;
    const col2 = event.col2;
    const col3 = event.col3;

    // Use the connection
    connection.query(
      "INSERT INTO table (col1, col2, col3) VALUES(?, ?, ?)",
      [col1, col2, col3],
      function (error, results, fields) {
        // And done with the connection.
        connection.release();

        var response = {
          statusCode: 200,
          body: JSON.stringify("Account added successfully!"),
        };

        // Handle error after the release.
        if (error) callback(error);
        else callback(null, response.body);
      }
    );
  });
};

I have a very similar lambda function which updates a record in the database from JSON, and it runs perfectly.

When testing the code, the records are successfully added to the DB. However, the lambda function will time out no matter how long I set the timeout limit to.

The lambda returns this error:

{ "errorMessage": "2021-10-14T10:00:25.159Z 62f12d68-c648-4f0f-9e86-3cd23f0f90c6 Task timed out after 5.01 seconds" }

The last two lines of the Logs are:

Thu Oct 14 10:00:25 UTC 2021 : Successfully completed execution
Thu Oct 14 10:00:25 UTC 2021 : Method completed with status: 200

I have tested the code as a local file running it using: node local.js and it runs quickly and without errors.

Not sure what I am missing. Maybe the return type of an INSERT INTO query is different from an UPDATE ?

Any help would be appreciated. Thanks

Answer 1

An answer has already been given in the comments on the question, but I want to go a bit more in depth here, since using a connection pool in a lambda function is (mostly) not useful and can even be harmful.

Why it is not useful

A connection pool is designed to handle multiple parallel connections to the database in applications where you might want to handle more than one different action at the same time (eg, an express API that handles multiple requests simultaneously). However, AWS Lambda is designed to only handle a single request at the same time and to have a different execution environment for each parallel execution. This means that you will be requesting far more connections from your database that you actually need.

Why it can be bad

You have no control over how a lambda execution environment is terminated. This also means that you cannot ensure that the connections in your connection pool will be closed gracefully, nor that the database server will close them for you. If you then get a lot of increase in lambda execution (by an increase in traffic or by creating a loop in lambda - which is very easily done) you can very quickly eat up all the available connections that the database has available, which can impact all other customers of the database as well. This usually ends with a very uncomfortable talk with a manager (been there, done that).

How to do it then?

Well, actually pretty straightforward, create a single connection, use it and close it again. Below is an example (using postgres db, but that should be easily adapted to use MySQL) of such a handler function:

const pgp = require('pg-promise')();
const getDBConfiguration = require('./db-config');

let dbOptions; // dbOptions are retrieved from SSM param store, we can cache those
const handler = async (event) => {
  dbOptions = dbOptions || { ...(await getDBConfiguration()), max: 1 }; // restrict number of connections in pool to one

  const client = pgp(dbOptions);
  try {
    await client.any(/*Execute some query*/);
  } finally {
    await client.$pool.end();
  }
};