Serverless Cron Job firing twice

Question

I am not sure why but my webhook is being fired twice in my cron job. So this cron job is suppose to run once every 15 min which it does, but it is firing off twice. I will post the logs, handler and yml file to help out.

Basically my cron job will make some request to a salsify api to store a url inside a mongodb. Once that file has been completed and built the next time the cron job runs it should trigger the webhook for netlify. Then the process starts all over again.

In my netlify I noticed the build was being ran twice and have pin pointed the source to the serverless cron job.

---

EDIT: Something I should add in here is that even if my cron job runs twice it still should still only technically call the webhook once if there is a file in the MongoDB. Yet it is still calling it twice somehow which is causing my netlify build to fail because it needs that file in order to build.

---

function part of serverless.yml:

functions:
  salsifyCron:
    handler: src/handler.salsifyCron
    events:
      - schedule: 
          rate: cron(*/15 * * * ? *)
          enabled: true

Logs:

2018-05-17 10:00:41.121 (-05:00)    10d87735-59e3-11e8-be56-69e06899fa1f    Trigger Webhook
2018-05-17 10:01:45.941 (-05:00)    10d87735-59e3-11e8-be56-69e06899fa1f    Trigger Webhook

handler:

require('envdotjs').load();
import fetch from 'isomorphic-fetch';
import axios from 'axios';
import middy from 'middy';
import { jsonBodyParser, httpErrorHandler, cors } from 'middy/middlewares';

import { connectToDatabase } from '../utils/db';
import Sheet from '../models/Sheet';
import config from '../utils/config';

module.exports.salsifyCron = middy(async (event, context, callback) => {
  context.callbackWaitsForEmptyEventLoop = false;
  let sheetId;
  const options = {
    url: `https://app.salsify.com/api/orgs/${
      process.env.SALSIFY_ORG_ID
    }/export_runs`,
    headers: {
      Authorization: `Bearer ${process.env.SALSIFY_API_KEY}`,
      'Content-Type': 'application/json'
    }
  };
  await connectToDatabase();

  const storedData = await Sheet.find({});
  if (
    storedData.length > 0 &&
    storedData[0] &&
    storedData[0].status === 'completed' &&
    storedData[0].url !== null
  ) {
    console.log('Trigger WebHook');
    axios.post('https://api.netlify.com/build_hooks/*****************');
    process.exit(0);
    return;
  }
  if (storedData[0]) {
    sheetId = storedData[0].sheetId;
  }
  if (storedData.length === 0) {
    const res = await fetch(options.url, {
      method: 'POST',
      headers: options.headers,
      body: JSON.stringify(config)
    }).then(res => res.json());
    if (res.id && res.status) {
      await Sheet.create({
        sheetId: res.id,
        url: null,
        status: res.status
      });
      sheetId = res.id;
    } else {
      console.log(res);
      process.exit(1);
    }
  }

  const resWithId = await fetch(`${options.url}/${sheetId}`, {
    method: 'GET',
    headers: options.headers
  }).then(res => res.json());

  if (resWithId.status === 'running') {
    console.log('running cron job');
    console.log(resWithId.estimated_time_remaining);
  }
  if (resWithId.status === 'completed') {
    console.log('completed cron job');
    await Sheet.findByIdAndUpdate(
      storedData[0],
      { status: resWithId.status, url: resWithId.url },
      { new: true }
    );
  }
})
  .use(cors())
  .use(jsonBodyParser())
  .use(httpErrorHandler());

Answer 1

Lambda timeout. This might not have been the problem in your case, but it is a common problem that causes this result.

Your lambdas are not getting executed simultaneously but with a bit of a delay. This is a clue that it is not just getting a duplicate execution.

I would guess that your lambda is first terminating with an error (for example timing out - the default lambda timeout is quite small) and the lambda is being rerun after it fails.

I have had this problem with timeouts and it is quite confusing if you don't notice that the lambda has timed out.