Skip to content

Sharding

In one line

Sharding splits data across multiple database servers — horizontal scaling for datasets too large for one machine.

In simple words

Sharding distributes a database's data across multiple servers (shards). Each shard holds a subset of the data, and the shard key determines which shard a row goes to.

Types:

  1. Range-based — shard by value ranges (user IDs 1-10000 on shard A, 10001-20000 on shard B)
  2. Hash-based — hash the shard key, distribute evenly
  3. Geo-based — shard by geographic region (US users on US shard)

Benefits:

  1. Horizontal scaling — add more shards as data grows
  2. Performance — each shard handles less data
  3. Availability — one shard down doesn't kill the whole system

Challenges:

  1. Cross-shard queries — joins and aggregations across shards are complex
  2. Rebalancing — adding shards requires redistributing data
  3. Consistency — distributed transactions are hard

MongoDB, Cassandra, and Vitess (MySQL sharding) are popular sharding solutions.

Example

javascript
// Sharding concept — distribute users across 4 shards by hash
function getShard(userId) {
  const shards = ['shard-1', 'shard-2', 'shard-3', 'shard-4'];
  const hash = hashCode(userId);
  return shards[hash % shards.length];
}

// User 'alice' → hash → shard-2
// User 'bob' → hash → shard-3

// Application routes queries to the right shard
const shard = getShard(userId);
const user = await dbConnections[shard].query(
  'SELECT * FROM users WHERE id = ?', [userId]
);

// Cross-shard query (e.g., count all users):
// Must query ALL shards and aggregate results
const counts = await Promise.all(
  shards.map(s => dbConnections[s].query('SELECT COUNT(*) FROM users'))
);
const total = counts.reduce((a, b) => a + b, 0);

Sharding distributes data across servers by shard key. Single-shard queries are fast. Cross-shard queries must fan out to all shards and aggregate.

Common confusions

  • Confused with: partitioning

    The difference: Partitioning splits a table within one database. Sharding splits data across multiple database servers. Partitioning is single-node; sharding is multi-node. Partitioning improves query performance; sharding enables horizontal scaling beyond one machine's capacity.

Related terms

Related Resources

Related Lessons

Decode error messages

Plain-English explanations of common errors — what they mean, why they happen, and how to fix them.

View errors

← Back to glossary