Can't connect to MongoDB Atlas from Google Function

Question

I already have an express server on App engine that is using my Mongo Atlas Database and everything is fine.

I want to launch a cloud function that is using the same database and is handling pubsub events. However, I can't connect to the database when I deploy the function.

I tried with the private URI and the public one. With the public, I some time manage to connect but it's not all the time (and I have to whitelest all IPs on the cluster).

Here is the function code (simplified)

const path = require('path')
require('dotenv').config({ path: path.join(__dirname, '/.env') })
const config = require('./libs/config/config.js')
const mongoose = require('mongoose')
const sanitize = require('mongo-sanitize')
const Utils = require('./libs/services/utils')

// Mongo connection string
const mongo = config.get('db.protocol') + '://' + config.get('db.user') + ':' + config.get('db.password') + '@' + config.get('db.host') + '/' + config.get('db.name') + '?retryWrites=true&w=majority'

// Database connexion
mongoose.connect(mongo, { useNewUrlParser: true, useCreateIndex: true, useUnifiedTopology: true }).then(() => {
  console.log('Connected to mongoDB')
}).catch(e => {
  console.log('Error while DB connecting')
  console.log(e)
})

/**
 * Background Cloud Function to be triggered by Pub/Sub.
 * This function is exported by index.js, and executed when
 * the trigger topic receives a message.
 *
 * @param {object} data The event payload.
 * @param {object} context The event metadata.
 */
exports.scenarioManagerPushPubSub = (gcpEvent, context) => {
  const event = Utils.parseEntryInfo(gcpEvent, context)
  console.log('scenarioManager : ', event, event.constructor.name)
 // my Function
}

I use the same connection string on my Express app on GAE.

I have the VPC serverless activated and I whitelisted the IP range on Mongo Atlas. The peering between Mongo Atlas and GCP is up and running.

The Error I get:

A 2020-08-18T09:16:03.547Z scenarioManager-staging Error while DB connecting scenarioManager-staging  
A 2020-08-18T09:16:03.547Z scenarioManager-staging { MongooseServerSelectionError: Could not connect to any servers in your MongoDB Atlas cluster. One common reason is that you're trying to access the database from an IP that isn't whitelisted. Make sure your current IP address is on your Atlas cluster's IP whitelist: https://docs.atlas.mongodb.com/security-whitelist/ scenarioManager-staging  
A 2020-08-18T09:16:03.547Z scenarioManager-staging     at NativeConnection.Connection.openUri (/workspace/node_modules/mongoose/lib/connection.js:830:32) scenarioManager-staging  
A 2020-08-18T09:16:03.547Z scenarioManager-staging     at Mongoose.connect (/workspace/node_modules/mongoose/lib/index.js:335:15) scenarioManager-staging  
A 2020-08-18T09:16:03.547Z scenarioManager-staging     at Object.<anonymous> (/workspace/smsGateway.js:16:10) scenarioManager-staging  
A 2020-08-18T09:16:03.547Z scenarioManager-staging     at Module._compile (internal/modules/cjs/loader.js:778:30) scenarioManager-staging  
A 2020-08-18T09:16:03.547Z scenarioManager-staging     at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10) scenarioManager-staging  
A 2020-08-18T09:16:03.547Z scenarioManager-staging     at Module.load (internal/modules/cjs/loader.js:653:32) scenarioManager-staging  
A 2020-08-18T09:16:03.547Z scenarioManager-staging     at tryModuleLoad (internal/modules/cjs/loader.js:593:12) scenarioManager-staging  
A 2020-08-18T09:16:03.547Z scenarioManager-staging     at Function.Module._load (internal/modules/cjs/loader.js:585:3) scenarioManager-staging  
A 2020-08-18T09:16:03.547Z scenarioManager-staging     at Module.require (internal/modules/cjs/loader.js:692:17) scenarioManager-staging  
A 2020-08-18T09:16:03.547Z scenarioManager-staging     at require (internal/modules/cjs/helpers.js:25:18) scenarioManager-staging  
A 2020-08-18T09:16:03.547Z scenarioManager-staging     at Object.<anonymous> (/workspace/index.js:1:20) scenarioManager-staging  
A 2020-08-18T09:16:03.547Z scenarioManager-staging     at Module._compile (internal/modules/cjs/loader.js:778:30) scenarioManager-staging  
A 2020-08-18T09:16:03.547Z scenarioManager-staging     at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10) scenarioManager-staging  
A 2020-08-18T09:16:03.547Z scenarioManager-staging     at Module.load (internal/modules/cjs/loader.js:653:32) scenarioManager-staging  
A 2020-08-18T09:16:03.547Z scenarioManager-staging     at tryModuleLoad (internal/modules/cjs/loader.js:593:12) scenarioManager-staging  
A 2020-08-18T09:16:03.547Z scenarioManager-staging     at Function.Module._load (internal/modules/cjs/loader.js:585:3) scenarioManager-staging  
A 2020-08-18T09:16:03.547Z scenarioManager-staging   message: scenarioManager-staging  
A 2020-08-18T09:16:03.547Z scenarioManager-staging    'Could not connect to any servers in your MongoDB Atlas cluster. One common reason is that you\'re trying to access the database from an IP that isn\'t whitelisted. Make sure your current IP address is on your Atlas cluster\'s IP whitelist: https://docs.atlas.mongodb.com/security-whitelist/', scenarioManager-staging  
A 2020-08-18T09:16:03.547Z scenarioManager-staging   reason: scenarioManager-staging  
A 2020-08-18T09:16:03.547Z scenarioManager-staging    TopologyDescription { scenarioManager-staging  
A 2020-08-18T09:16:03.547Z scenarioManager-staging      type: 'ReplicaSetNoPrimary', scenarioManager-staging  
A 2020-08-18T09:16:03.547Z scenarioManager-staging      setName: null, scenarioManager-staging  
A 2020-08-18T09:16:03.547Z scenarioManager-staging      maxSetVersion: null, scenarioManager-staging  
A 2020-08-18T09:16:03.547Z scenarioManager-staging      maxElectionId: null, scenarioManager-staging  
A 2020-08-18T09:16:03.547Z scenarioManager-staging      servers: scenarioManager-staging  
A 2020-08-18T09:16:03.547Z scenarioManager-staging       Map { scenarioManager-staging  
A 2020-08-18T09:16:03.547Z scenarioManager-staging         '*-shard-00-00-pri.xgtnk.gcp.mongodb.net:27017' => [ServerDescription], scenarioManager-staging  
A 2020-08-18T09:16:03.547Z scenarioManager-staging         '*-shard-00-01-pri.xgtnk.gcp.mongodb.net:27017' => [ServerDescription], scenarioManager-staging  
A 2020-08-18T09:16:03.547Z scenarioManager-staging         '*-shard-00-02-pri.xgtnk.gcp.mongodb.net:27017' => [ServerDescription] }, scenarioManager-staging  
A 2020-08-18T09:16:03.547Z scenarioManager-staging      stale: false, scenarioManager-staging  
A 2020-08-18T09:16:03.547Z scenarioManager-staging      compatible: true, scenarioManager-staging  
A 2020-08-18T09:16:03.548Z scenarioManager-staging      compatibilityError: null, scenarioManager-staging  
A 2020-08-18T09:16:03.548Z scenarioManager-staging      logicalSessionTimeoutMinutes: null, scenarioManager-staging  
A 2020-08-18T09:16:03.548Z scenarioManager-staging      heartbeatFrequencyMS: 10000, scenarioManager-staging  
A 2020-08-18T09:16:03.548Z scenarioManager-staging      localThresholdMS: 15, scenarioManager-staging  
A 2020-08-18T09:16:03.548Z scenarioManager-staging      commonWireVersion: null } } scenarioManager-staging  

How I deploy my function:

gcloud functions deploy scenarioManager-staging --entry-point scenarioManagerPushPubSub --runtime nodejs10 --trigger-topic scenarioManager-staging --region europe-west1 --set-env-vars NODE_ENV=staging --vpc-connector=default-serverless-vpc 

Can you help me please?

[EDIT] I added a Firewall entry on my default VPC to allow 27015-2017 outbound. I added rights on the service account to access the.network. .. BUT stil KO on private URI. On the public one, with a full whitelist, i sometimes connect but not always.

Answer 1

Try running the function locally and make sure it works.

if it works locally and not after deployment make sure your firebase project is upgraded to Blaze plan.

Answer 2

While looking into it, I have found a similar issue reported here [1]. Using Google DNS resolved the issue for the user. I would recommend checking this link if it helps.

[1] Not able to connect to MongoDB atlas database

Answer 3

I did the dev with a full local config (pubsub + functions + database). I just tested with a connection to Mongo Atlas from my computer and it works well.

$ functions-framework --target=scenarioManagerPushPubSub --signature-type=event --source=./scenarioManager.js
Serving function...
Function: scenarioManagerPushPubSub
URL: http://localhost:8080/
Connected to mongoDB
scenarioManager :  { call: '5f3a39ec79c84c000a063802' } Object
Call  5f3a39ec79c84c000a063802  found
Nothing to do for this scenario 5f353c0319947a000afe9bde