mongodb: OperationFailure: Query for sharded findAndModify must contain the shard key

Question

I'm encountering this error, does anyone know why, under the hood, the shard key must be included in findAndModify query?

Answer 1

Most likely due to stated consistency and causality guarantees .

When a find-and-modify operation involves finding by shard key, the entire operation is known to be performable on one shard (or, technically, zero or one shards) since the shard key uniquely determines the shard that holds data with that shard key. In other words, the entire operation can be routed to a single replica set that would perform the find-and-modify with the consistency and causality guarantees obtainable from a replica set in general.

Without the shard key in conditions, the query may involve data stored on multiple shards. A read query that does not involve restriction by shard key is sent to all shards which independently return matching documents, which are then assembled into a single result set by a mongos router (this is also known as scatter-gather read). When performing a find-and-modify, the update must also be sent to all of the shards that had matching documents which could be multiple shards. Without a distributed coordinator, writes across shards may be inconsistent. Such a distributed coordinator did not exist prior to 4.2 server when transaction support was implemented in sharded clusters.

I imagine in the future the restriction to include shard key in find-and-modify may be relaxed if the find-and-modify is performed in a transaction, since the transaction does already perform distributed coordination (though the individual updates would still be more efficient if restricted to a single shard).