Sharding
In one line
Sharding splits data across multiple database servers — horizontal scaling for datasets too large for one machine.
In simple words
Sharding distributes a database's data across multiple servers (shards). Each shard holds a subset of the data, and the shard key determines which shard a row goes to.
Types:
- Range-based — shard by value ranges (user IDs 1-10000 on shard A, 10001-20000 on shard B)
- Hash-based — hash the shard key, distribute evenly
- Geo-based — shard by geographic region (US users on US shard)
Benefits:
- Horizontal scaling — add more shards as data grows
- Performance — each shard handles less data
- Availability — one shard down doesn't kill the whole system
Challenges:
- Cross-shard queries — joins and aggregations across shards are complex
- Rebalancing — adding shards requires redistributing data
- Consistency — distributed transactions are hard
MongoDB, Cassandra, and Vitess (MySQL sharding) are popular sharding solutions.
Example
// Sharding concept — distribute users across 4 shards by hash
function getShard(userId) {
const shards = ['shard-1', 'shard-2', 'shard-3', 'shard-4'];
const hash = hashCode(userId);
return shards[hash % shards.length];
}
// User 'alice' → hash → shard-2
// User 'bob' → hash → shard-3
// Application routes queries to the right shard
const shard = getShard(userId);
const user = await dbConnections[shard].query(
'SELECT * FROM users WHERE id = ?', [userId]
);
// Cross-shard query (e.g., count all users):
// Must query ALL shards and aggregate results
const counts = await Promise.all(
shards.map(s => dbConnections[s].query('SELECT COUNT(*) FROM users'))
);
const total = counts.reduce((a, b) => a + b, 0);Sharding distributes data across servers by shard key. Single-shard queries are fast. Cross-shard queries must fan out to all shards and aggregate.
Common confusions
Confused with: partitioning
The difference: Partitioning splits a table within one database. Sharding splits data across multiple database servers. Partitioning is single-node; sharding is multi-node. Partitioning improves query performance; sharding enables horizontal scaling beyond one machine's capacity.
Related terms
Related Resources
Related Lessons
- Database Scaling
Sharding.
Decode error messages
Plain-English explanations of common errors — what they mean, why they happen, and how to fix them.