HLDintermediatePremium

BitTorrent HLD: Downloading From Strangers, With No Server

A BitTorrent system design: splitting a file into hashed pieces, the peer swarm and tracker, rarest-first piece selection, tit-for-tat incentives, and why P2P scales up as demand grows.

By fiveyearsdevJune 14, 20269 min read

"Design BitTorrent." Every other system on this site has a server somewhere — a database, a cache, an origin. BitTorrent's whole premise is that there isn't one. A million people download the same file with no central machine paying for the bandwidth, because each downloader becomes an uploader: the crowd serves itself. That inverts the usual scaling fear — here, the more people who want the file, the more capacity there is to serve it. Two problems fall out of "no server, download from strangers," and they're the design. How do you trust bytes from an anonymous peer (they could be corrupt or …

What’s inside

Let's start nowhere near a computer

Where this exact shape shows up

Step 1 — Functional requirements (sentences first)

Step 2 — Non-functional requirements

Step 3 — Pieces and hashes (where the data lives)

Step 4 — The swarm and the tracker

Step 5 — Rarest-first piece selection

Step 6 — Tit-for-tat: keeping it fair

Step 7 — Trade-offs (each one keeping an NFR)

The complete implementation

Step 8 — Scaling: the design that gets stronger under load

Step 9 — When a piece fails: designing for failure

The interview corner

Where to go from here

Read this one free

HLD