Changing the engine mid-flight

On 15 September 2022, Ethereum completely changed the way it works. A network that moves billions every day replaced the piece that decides which transactions are valid, and it did so without stopping for a single minute. Hardly any of its users had to do anything.

The community itself described it with a vivid image: changing a spaceship’s engine mid-flight. How do you pull off something like that? With far less magic than it seems.

Two engines

Until then, Ethereum ran on proof of work, the same system as Bitcoin. To add a block, miners have to carry out computational work that costs real energy and that the rest of the network can easily check. That cost is exactly what makes it secure: it anchors the network to the physical world, and cheating would mean spending that energy for nothing in return. Bitcoin has worked this way since 2009.

The new engine is proof of stake. Miners are replaced by validators who, instead of energy, put their own funds on the line as collateral. If one of them tries to cheat the network, they lose part of what they deposited.

Swapping one for the other was no minor tweak: it meant replacing the heart of the system. Whether it was a good decision is another debate, and there are strong arguments in favour of proof of work. What interests us here is something else: how it was done.

Build alongside, not on top

The first decision was to leave the working engine alone. In December 2020, the Beacon Chain went live: a new proof-of-stake chain that ran in parallel with the original for almost two years.

During that time it didn’t move anyone’s transactions. Its job was to prove that the new engine worked: thousands of validators, blocks every few seconds, real failures and real fixes. When the time came to join the two chains, the new engine had been running for real for 21 months.

Rehearse until it’s boring

Before the real day there were many practice days. First on test networks created for the occasion, such as Kintsugi, in late 2021. Then, in mid-2022, on the test networks developers were already using every day: Ropsten in June, Sepolia in July, Goerli in August.

On top of that came dozens of shadow forks: exact copies of the real network on which the migration was rehearsed with real data. Some rehearsals went wrong, and that was the point. Every failure found in a rehearsal was one that wouldn’t make it to the big day.

The day nobody noticed

On 15 September 2022, the original network and the Beacon Chain merged. Transactions kept flowing, balances stayed where they were and applications kept working.

For most users, the biggest change in Ethereum’s history was just another day. That was exactly the goal.

Not every plan turned out as announced back in 2021. The original idea of splitting the network into 64 chains changed along the way into a different approach. But the order of priorities held: first, make the change safe.

What works for any migration

Few companies will ever change the engine of a global network. Almost all of them will migrate something one day: a database, a provider, a payment system. And failed migrations are expensive.

In April 2018, the British bank TSB moved its customers onto a new platform. The data was migrated, but the platform failed as soon as it went live, and a significant share of its 5.2 million customers had problems for months. In 2022, regulators fined it 48.65 million pounds, pointing out, among other things, that the migration hadn’t been properly planned.

Ethereum’s experience leaves four habits that are easy to apply:

Build alongside. The new system runs in parallel before it replaces the old one.
Rehearse with real data. A faithful copy of production uncovers what test environments miss.
Ask nothing of the user. The best migration is one that doesn’t make them do anything.
Have a way back. Know in advance how to undo the change if something goes wrong.

No magic involved

Big migrations don’t go well by luck. They go well because they’re built alongside, rehearsed until they’re boring and designed so that nobody needs to find out.

The best migrations are the ones nobody remembers.