MongoDB replicaset configuration

Question

Could you please tell me if this will cause any issues with failover? For example, what would happen if host mongo2.local is down? (assuming the original host and the arbiter go down and only 2 members are left). Will the rest of the members be able to elect a new primary ever?

I know that there shouldn't be an arbiter here as it makes things worse but I wanted to know if a failover will occur in case of this setup and mongo2.local go down.

mongo:ARBITER> rs.status()
{
        "set" : "mongo",
        "date" : ISODate("2015-02-12T09:00:08Z"),
        "myState" : 7,
        "members" : [
                {
                        "_id" : 0,
                        "name" : "mongo1.local:27017",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",
                        "uptime" : 2572473,
                        "optime" : Timestamp(1423731603, 4),
                        "optimeDate" : ISODate("2015-02-12T09:00:03Z"),
                        "lastHeartbeat" : ISODate("2015-02-12T09:00:07Z"),
                        "lastHeartbeatRecv" : ISODate("2015-02-12T09:00:07Z"),
                        "pingMs" : 0,
                        "syncingTo" : "mongo2.local:27017"
                },
                {
                        "_id" : 1,
                        "name" : "mongo2.local:27017",
                        "health" : 1,
                        "state" : 1,
                        "stateStr" : "PRIMARY",
                        "uptime" : 12148099,
                        "optime" : Timestamp(1423731603, 4),
                        "optimeDate" : ISODate("2015-02-12T09:00:03Z"),
                        "lastHeartbeat" : ISODate("2015-02-12T09:00:08Z"),
                        "lastHeartbeatRecv" : ISODate("2015-02-12T09:00:08Z"),
                        "pingMs" : 0,
                        "electionTime" : Timestamp(1423711411, 1),
                        "electionDate" : ISODate("2015-02-12T03:23:31Z")
                },
                {
                        "_id" : 2,
                        "name" : "mongo3.local:27017",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",
                        "uptime" : 5474488,
                        "optime" : Timestamp(1423731603, 4),
                        "optimeDate" : ISODate("2015-02-12T09:00:03Z"),
                        "lastHeartbeat" : ISODate("2015-02-12T09:00:07Z"),
                        "lastHeartbeatRecv" : ISODate("2015-02-12T09:00:08Z"),
                        "pingMs" : 139,
                        "syncingTo" : "mongo2.local:27017"
                },
                {
                        "_id" : 3,
                        "name" : "mongo2.local:27020",
                        "health" : 1,
                        "state" : 7,
                        "stateStr" : "ARBITER",
                        "uptime" : 12148101,
                        "self" : true
                }
        ],
        "ok" : 1
}

and:

mongo:ARBITER> rs.config()
{
        "_id" : "mongo",
        "version" : 5,
        "members" : [
                {
                        "_id" : 0,
                        "host" : "mongo1.local:27017",
                        "priority" : 0.5
                },
                {
                        "_id" : 1,
                        "host" : "mongo2.local:27017"
                },
                {
                        "_id" : 2,
                        "host" : "mongo3.local:27017",
                        "priority" : 0.5
                },
                {
                        "_id" : 3,
                        "host" : "mongo2.local:27020",
                        "arbiterOnly" : true
                }
        ]
}

Answer 1

If you have less than a majority of the votes in a replica set available, the replica set will not be able to elect or maintain a primary and the replica set will be unhealthy and will be read-only. Ergo, if you only have 2 of your 4 embers up, you will not have a primary. No automatic failover will occur as there aren't enough votes for an election.

Don't have an even number of nodes in a replica set. It increases the chances that there will be problems, just because there are more servers, without increasing the failure tolerance of the set. With 3 or 4 replica set members, 2 down servers will render the set unhealthy.