Sharding is MongoDB’s approach to scaling out. Sharding partitions a collection and stores the different portions on different machines. When a database’s collections become too large for existing storage, you need only add a new machine. Sharding automatically distributes collection data to the new server.
Sharding automatically balances data and load across machines. Sharding provides additional write capacity by distributing the write load over a number of mongod instances. Sharding allows users to increase the potential amount of data in the working set.
Sharding occurs within a sharded cluster. A sharded cluster consists of the following components:
Applications do not access shards directly but instead access mongos instances.
mongos instances. The mongos instances route the reads and writes from applications to the shards. Applications do not access the shards directly.
Within a sharded cluster, you enable sharding on a per-database basis. When you enable sharding on a database, MongoDB distributes the collections across the shards. MongoDB does not yet distribute data within a collection.
After enabling sharding for a database, you choose which collections to shard. For each sharded collection, you specify a shard key.
To set up a sharded cluster, see Deploy a Sharded Cluster.
The shard key determines the distribution of the collection’s documents among the cluster’s shards. The shard key is a field that exists in every document in the collection. MongoDB distributes documents according to ranges of values in the shard key. A given shard holds documents for which the shard key falls within a specific range of values. Shard keys, like indexes, can be either a single field or multiple fields.
Within a shard, MongoDB further partitions documents into chunks. Each chunk represents a smaller range of values within the shard’s range. When a chunk grows beyond the chunk size, MongoDB splits the chunk into smaller chunks, always based on ranges in the shard key.
To select a shard key, see Select a Shard Key.
New in version 2.4.
The field you choose as your hashed shard key should have a good cardinality, or large number of different values. Hashed keys work well with fields that increase monotonically like ObjectId values or timestamps.
If you shard an empty collection using a hashed shard key, MongoDB will automatically create and migrate chunks so that each shard has two chunks. You can control how many chunks MongoDB will create with the numInitialChunks parameter to shardCollection or by manually creating chunks on the empty collection using the split command.
To shard a collection using a hashed shard key, see Hashed Sharding.
Balancing is the process MongoDB uses to redistribute data within a sharded cluster. When a shard has too many chunks when compared to other shards, MongoDB automatically balances the shards. MongoDB balances the shards without intervention from the application layer.
The balancing process attempts to minimize the impact that balancing can have on the cluster, by:
You may disable the balancer on a temporary basis for maintenance and limit the window during which it runs to prevent the balancing process from impacting production traffic.
To disable the balancer, see Disable the Balancer.
The balancing procedure for sharded clusters is entirely transparent to the user and application layer. This documentation is only included for your edification and possible troubleshooting purposes.
While sharding is a powerful and compelling feature, it comes with significant infrastructure requirements and some limited complexity costs. As a result, use sharding only as necessary and when indicated by actual operational requirements.
You should consider deploying a sharded cluster if:
If these attributes are not present in your system, sharding will only add additional complexity to your system without providing much benefit. When designing your data model, if you will eventually need a sharded cluster, consider which collections you will want to shard and the corresponding shard keys.
It takes time and resources to deploy sharding, and if your system has already reached or exceeded its capacity, you will have a difficult time deploying sharding without impacting your application.
As a result, if you think you will need to partition your database in the future, do not wait until your system is overcapacity to enable sharding.
For information on requirements, see the following: