MongoDb Replica Set write concern issues on Secondary IP, while Primary IP is Shutdown

Question

I am using three nodes for my MongoDB replica set, one as the primary node, one as a secondary node, and one as an arbitrator node. I am using the following connection string in Golang to connect with MongoDB.

opts = options.Client().ApplyURI("mongodb://" + ipPort + "," + secIpPort + "/?replicaSet=" + repSetName).SetConnectTimeout(5 * time.Second).SetAuth(serverAuth).SetWriteConcern(writeconcern.New(writeconcern.WMajority()))

Ip used to make connection are, ipPort= 192.168.1.237 secipPort= 192.168.1.239

I have setted the Default RW Concern as following,

db.adminCommand({ "setDefaultRWConcern" : 1, "defaultWriteConcern" : { "w" : "majority", "j":true, "wtimeout" : 5000 }, writeConcern: { "w" : "majority", "j":true, "wtimeout" : 5000 }, })

Problem:

When the MongoDb service of the first IP(192.168.1.237) of the connection string is shut down, the secondary IP(192.168.1.239) performs all the read operations well. But does not perform the write operations.How can I deal with this problem? If the first IP of the connection string is down, the secondary IP/node should be able to perform both read and write operations.

How can I deal with this problem? If the first IP of the connection string is down, the secondary IP/node should be able to perform both read and write operations.

Answer 1

The reason is writeConcern: { w: "majority" }

In MongoDB you have two majorities. The voting majority and the write majority. You can interrogate them with rs.status()

In a PRIMARY-SECONDAY-ARBITER (PSA) Replica Set, it should look like this:

"majorityVoteCount" : 2
"writeMajorityCount" : 2
"votingMembersCount" : 3
"writableVotingMembersCount" : 2

If the SECONDARY goes down, then you have 2 voting member which is the majority. They can still elect a new PRIMARY.

However, only 1 writing member is left, 1 out of 2 is not the majority. Thus operation with { w: "majority" } are not possible unless the missing member comes back again.

Some background information can be found here: MongoClient to connect to multiple hosts to handle failover?

Answer 2

When the primary node of the MongoDB replica set is down, the secondary node should take over as the new primary and continue to perform both read and write operations. However, it seems that in your case, the secondary node is not able to perform write operations when the primary node is down.

There are a few possible reasons for this:

The replica set configuration is not correct. Make sure that the replica set name and the list of nodes in the replica set match what is configured in MongoDB.

The secondary node is not able to reach a quorum of nodes to perform a write operation. In your case, you are using an arbiter node that does not store data and can't vote on the write operation. You may want to consider removing the arbiter node from the replica set or adding another secondary node to ensure that a quorum of nodes is always available for write operations.

The write concern is not configured correctly. In your code, you are setting the write concern to "majority" and "j=true". The write concern "j=true" means that a write operation only returns after the write is written to the on-disk journal. Make sure that the journaling is enabled on the MongoDB and the disk space is enough.

The.network connectivity between the nodes is not stable. Make sure that the nodes can communicate with each other and that there are no firewalls or other.network issues that are preventing the nodes from communicating.

The secondary node is not up-to-date with the primary node. Make sure that the secondary node is catching up with the primary node's oplog and that the secondary node is in a healthy state.

It's recommended to check the MongoDB logs and running the command "rs.status()" to check the state of the replica set and see if there are any error messages. You can also try to connect to the secondary node directly and check the state of the replica set with the command "rs.status()" and check the state of the replica set and see if there are any error messages.

If the problem persists, you may want to consider adding another secondary node to ensure that a quorum of nodes is always available for write operations.