AWS Mongo QuickStart never completes

Question

Problem

I am trying to complete the MongoDB on AWS quickstart to create a simple MongoDB cluster. Unfortunately it never completes the rollout, cancelling after one last installation part (PrimaryReplicaNodeXYWaitForNodeInstallGP2) has not been completed within an hour.

Background

My Settings were the following:

AvailabilityZone0   eu-central-1a
AvailabilityZone1   eu-central-1b
AvailabilityZone2   eu-central-1b
BuildBucket quickstart-reference/mongodb/latest
ClusterReplicaSetCount  0
ClusterShardCount   1
ConfigServerInstanceType    t2.micro
Iops    100
KeyName my_definitely_working_keypair
MongoDBVersion  3.2
NATInstanceType t2.small
NodeInstanceType    m3.medium
PrimaryReplicaSubnet    10.0.2.0/24
PublicSubnet    10.0.1.0/24
RemoteAccessCIDR    XXX.XXX.0.0/16
SecondaryReplicaSubnet0 10.0.3.0/24
SecondaryReplicaSubnet1 10.0.4.0/24
ShardsPerNode   0
VolumeSize  40
VolumeType  gp2
VPCCIDR 10.0.0.0/16

Which caused a rollback in the same behaviour, as named in the AWS forum :

In "Ressources", all but one subtask never gets completed and stays on forever as "PrimaryReplicaNode0WaitForNodeInstallGP2 - PrimaryReplicaNode0WaitForNodeInstallWaitHandle - Created in Progress - Ressource creation initiated"

So, I was further researching on the issue. The post referred to another forum thread , where users with the problem should try to delete their DynamoDB entries and set ClusterReplicaSetCount to 3.

Problem here: In DynamoDB there are no entries and changing ClusterReplicaSetCount to 3 also causes a rollback with a similar error:

ConfigServer2WaitForNodeInstall WaitCondition timed out. Received 0 conditions when expecting 1

and later

MONGODBSTACK1 The following resource(s) failed to create: [ConfigServer1WaitForNodeInstall, PrimaryReplicaNode00WaitForNodeInstallGP2, ConfigServer0WaitForNodeInstall, SecondaryReplicaNode00WaitForNodeInstallGP2, SecondaryReplicaNode01WaitForNodeInstallGP2, ConfigServer2WaitForNodeInstall].

Summary

In both cases there is a fail on PrimaryReplicaNodeXYWaitForNodeInstallGP2 (where XY is the number of the node) while all other parts of the installation completed successfully. I am totally in the dark.

Anyone got around this? The quick start is from 2016 and I think there must be people, who have successfully created this mongo stack??

Answer 1

After days and days of hard struggle and no solution there was an update (since over a year, feels as if my prayers were heard) on the manual and the template:

https://docs.aws.amazon.com/quickstart/latest/mongodb/welcome.html

So this also comes with a completely revised infrastructure and a more sophisticated setup form, changes are described as:

Upgraded MongoDB to version 3.4; removed sharding configuration; updated security groups and added database security; updated parameters

Following the tutorial is quite similar to the former versions, so no struggle here.

Everything went out fine and I got my stack completed now consisting of a

mongoDB mongoDB Replica bastion stack vpc stack

So this part is basically done. If something else comes up, I will open a new question for that.

Answer 2

I noticed this after tearing down a dev cluster and attempting to stand up a new one with the same name.

The torn down cluster orphaned a dynamodb table with the name the new stack was trying to publish the worker nodes status onto. I deleted this dynamo table manually and reattempted to spin up the stack with the same name a third time and had success.