Let's Build a URL Shortener That Survives a Billion Links

A URL shortener is a hash table with a domain name. That is the joke, and it is why the problem is a good one: the feature list fits in a sentence, so every hour you spend on it is spent on the parts that actually decide whether a system works.

Two endpoints. Create a short code for a URL. Follow a short code to a URL. Let us build it properly.

What are we actually building?

Start with the numbers, because they change the answers.

Figure 1

Back of the envelope, before any code

Writes / second: —
Reads / second: —
Peak reads / second: —
Storage after 5 years: —

New links / day1,000,000Reads per link100 : 1Cache hit rate90%

Row size assumes a 7-character code, a URL averaging 120 bytes, a creator id, timestamps and an index — roughly 200 bytes on disk. Peak traffic is modelled at five times the daily average, which is a conservative multiplier for anything with a human audience in one set of time zones.

Three things fall out of that panel, and they set everything that follows.

The data is small. Even at a million new links a day, five years of rows fit comfortably on one machine. There is no sharding problem here, and any design that starts with sharding has skipped the basics.

Reads dwarfs writes. A hundred to one is a modest ratio for this workload. Every engineering decision should be biased towards making the read path boring.

The cache is the architecture. Watch what happens to the database load as the cache hit rate moves from 80% to 95%. That single number matters more than your choice of database.

How do we generate the codes?

This is the only genuinely interesting design decision in the system, and there are three defensible answers.

Option 1: encode a counter

Keep an auto-incrementing integer, encode it in base62, and you are done.

Figure 2

One integer, one short code

Database id1,000,000

Short URLcbz.to/4c92

The division

Code length	Distinct URLs
1	62
2	3,844
3	238,328
4	14,776,336
5	916,132,832
6	56,800,235,584
7	3,521,614,606,208

Sequential idDrag to move between 1 and a trillion

Base62 uses 0-9, a-z and A-Z — 62 symbols that survive a URL with no escaping. Encoding is repeated division by 62; the remainders, read backwards, are the code. Nothing is stored to make this work, which is exactly why it is worth understanding before reaching for random keys.

It is fast, it never collides, and the codes are as short as they can possibly be.

Though there are some problems with it. Codes are guessable — 4c92 tells you 4c93 probably exists, so anything private is enumerable. Codes leak your growth rate, which is a business-intelligence gift to competitors. And a single global counter is a coordination point, which is the thing you were trying to avoid.

The counter problem has a simple fix: hand each application instance a range of ids — say a million at a time — from a small allocator, and let it burn through them locally. One coordination round trip per million links, no shared state on the hot path, and gaps in the sequence that does not affect the system.

Option 2: random keys

Generate seven random base62 characters and check whether they are taken.

Figure 3

How soon does a random key collide?

Keyspace56.8 billion

Collision chance0.9%

50/50 at about281,000 links

Code lengthLinks created100,000

Probability of at least one collision among n randomly chosen keys from a keyspace of 62^L, using the standard birthday approximation 1 − exp(−n²/2N). It is an approximation, and it is accurate enough to make the point: with random keys you meet your first duplicate at roughly the square root of the keyspace, not near the end of it.

Codes are unguessable, and there is no counter to coordinate. The catch is the birthday problem, and the panel above puts numbers on it — collisions occurs at roughly the square root of the key-space, not near the end of it.

This however, does not make random keys wrong. It makes the unique index on the code column mandatory, and the insert becomes a retry loop instead of a single statement. As long as the key-space is large enough that retries are rare, this is a perfectly good scheme, and it is what most public shorteners use.

Option 3: hash the URL

Take a SHA-256 of the target and use the first seven base62 characters.

It is deterministic, which means submitting the same URL twice gives the same code and saves a row. It also means you cannot ever give two people different codes for the same target, which breaks per-user analytics, and hash collisions still need handling. Useful as a deduplication layer, not as your primary scheme.

What to actually do: ranged counters encoded in base62, with a random component mixed in if guessability matters. You get the short codes and no coordination on the hot path, and you keep the option of unguessable codes for the customers who ask.

What does a request look like?

Figure 4

Two paths, wildly different requirements

1POST /api/linksAuthenticated, rate limited per account
2ValidateScheme is http or https, host resolves, not on a blocklist, not a shortener loop≈2 ms
3Allocate a codeTake the next id from a range this instance already reserved, encode base62≈0 ms
4INSERTUnique index on the code; on conflict, take the next id and retry≈3 ms
5201 CreatedReturn the short URL. Nothing is cached — nobody has asked for it yet

Writes are rare and can afford to be careful. Validation, blocklists and rate limits all belong here, because this is the only path an abuser can use to put something into your system.

Path

Latencies are indicative. The asymmetry is the design: one path runs a few hundred times a second and must never be interesting, the other runs a few times a second and is where every abuse control lives.

Two details on that read path repay attention.

301 or 302? A 301 Moved Permanently response is cached by browsers indefinitely, which is ideal for latency but terrible for everything else, you can never change the target, and you never see the click again, so the analytics go to zero. A 302 Found response costs you a request every time and keeps both. Default to 302 and offer 301 as an option for links that are genuinely permanent.

Analytics must be asynchronous. Writing a click row before responding puts your analytics database on the critical path of every redirect. Emit an event to a queue, or batch in memory and flush, and let the redirect return. A shortener that is down because its stats table is locked is an embarrassing way to fail.

Which database?

Almost anything, which is the honest answer for a workload this shape.

The access pattern is a primary key lookup returning one row. Postgres does that in under a millisecond at this scale, and gives you transactions, a unique index, and the ability to run a report without exporting anything. Start there.

The refinement is to separate the two jobs. Keep the durable record — code, target, owner, timestamps, options — in Postgres. Serve reads from something in front of it: Redis for a single region, or a replicated key-value store like DynamoDB, Cloudflare KV or Workers KV if the traffic is genuinely global. The mapping is immutable once created, which makes caching trivial: no invalidation logic, just a long TTL and a delete on the rare occasions someone edits or removes a link.

If you do reach a scale that needs sharding, shard on the code itself — it is the only key anything looks up by, and it is uniformly distributed by construction.

What breaks when real people find it?

Everything on this list has taken down a real shortener.

It becomes a phishing tool. Somebody will shorten a link to a credential-harvesting page and mail it to ten thousand people, and the domain that gets blacklisted is yours. Check submissions against Google Safe Browsing or a similar feed, at creation and again on a schedule, because a URL that was clean on Tuesday can become malicious by Friday.

It becomes an open redirect. Your domain, its destination. That is a redirect gadget attackers actively look for, so it is worth an interruption for links that have not been vetted, and it is worth never allowing a shortener to point at itself.

Someone shortens the shortener. Two loops pointing at each other will happily consume a request handler. Resolve one hop at creation, refuse anything that lands back on your own domains, and cap redirect chains.

A single link goes viral. One code takes ninety per cent of your traffic. This is the case a cache handles beautifully and a database does not, and it is the reason the edge cache is not an optimization but the design.

Enumeration. Sequential codes get scraped, and now somebody has a list of every URL your users have shortened, including the ones they assumed were private. Rate limit by IP on the redirect path, and use unguessable codes wherever the target is not public.

Custom aliases collide with your routes. The moment you let a user pick cbz.to/pricing, they can pick cbz.to/api or cbz.to/admin. Reserve a namespace before you launch the feature, not after.

What would we build on a Monday?

For a real product, this is the shape we would ship:

Postgres for the durable record: code primary key, target, owner_id, created_at, expires_at, is_active.
Ranged id allocation, a million at a time, encoded in base62, with a random suffix on links flagged private.
A cache in front of every read — an edge cache first, a key-value store behind it, the database only on a miss. Immutable data, so a long TTL and an explicit delete.
302 by default, 301 opt-in per link.
Clicks to a queue, aggregated on a schedule. Never on the response path.
Safe Browsing at creation and on a weekly sweep, plus per-account rate limits and a reserved-prefix list.

That is a system one engineer can build in a week and that will hold a billion links without a redesign. The interesting part was never the hash table — it was noticing that reads and writes are different systems that happen to share a table.