Replicating BitTorrent PropShare

BitTorrent is usually described as tit-for-tat, but it actually behaves more like an auction. Peers bid for bandwidth by uploading to their neighbors, the top bidders get unchoked, and then every winner gets the same share no matter how much they bid.

That last part is easy to exploit. If every winner gets the same share, the best strategy is to upload just enough to stay a winner. BitTyrant clients do exactly that, and BitThief clients barely upload at all and live off optimistic unchoking.

PropShare was the proposed fix: hand out bandwidth in proportion to what each peer uploads, so uploading twice as much gets you twice the bandwidth, and gaming the auction stops paying off. We set out to replicate the paper that showed this worked.

You have 90 KB/s to give. Who gets it?

Peer A 40 30

Peer B 35 30

Peer C 10 30

Peer D 9 0

Peer C uploads 10 KB/s and gets the same 30 KB/s as Peer A, who uploads 40.

Try:

Simplified to three regular unchoke slots and no optimistic unchoking. All numbers are KB/s.

Rebuilding the testbed

The original experiments ran on PlanetLab, a global research testbed that has been shut down for years. So before we could test anything, we had to build something like it ourselves: a multi-region AWS EC2 deployment with our own orchestration, node roles declared in YAML, a controlled launch order (leechers first, then seeders, so nobody gets a head start), live log streaming, and central collection of download times.

Then we implemented both a vanilla BitTorrent client and a PropShare client, including seeding, peer discovery, optimistic unchoking, auction-based unchoking, and bandwidth accounting. Most of the project ended up being infrastructure, and the experiment itself was the smaller part.

Results

We got three runs in, at PropShare-to-BitTorrent ratios of 0.33, 0.6, and 1.0.

Performance comparison between PropShare and vanilla BitTorrent — PropShare against vanilla BitTorrent across three configurations. Notice how much the confidence intervals overlap.

PropShare performed about the same as vanilla BitTorrent. The variance was high, and the confidence intervals overlapped too much for us to honestly call it a win.

We think the difference from the original comes from three places. PlanetLab was a much messier system than a set of EC2 instances, and some of the original effect may have depended on that. EC2 latency also looks different from PlanetLab’s geographically scattered nodes. And the original paper left some implementation details unspecified, so our PropShare and theirs aren’t guaranteed to be the same algorithm.

What we learned about replication

You have to make assumptions. The original never said which BitTorrent client it used as the baseline, and the spec allows several that don’t behave the same way.
Infrastructure takes most of the time. Writing our own AWS orchestration gave us control, but it ate our schedule. Kubernetes would probably have been the better call.
Inherited code has a cost. We built on PyTorrent, and what it claimed to support and what it actually did turned out to be different things.
Small details matter. An unspecified detail, like who gets bandwidth when one node monopolizes a piece, can change the whole result.