software-architecture · 3 min read
The five-percent ceiling
The monitoring system was designed to watch thousands of live streams at once, flagging glitches before they reached viewers. When Marcin Kolny and his team at Prime Video ran it at scale in late 2022, it hit a wall at five percent of expected load.
The technical post describing what happened — and what fixed it — spread across the industry in May 2023. The original architecture was built on the standard cloud-native playbook of its era: AWS Step Functions to orchestrate analysis across Lambda functions, with video frames shuttling in and out of S3 on each hop. Each component was independently scalable. The whole thing was serverless. It was microservices all the way down.
The problem was cost. Step Functions charge per state transition, and the monitoring service ran multiple transitions for every second of every stream. S3 charges per API call, and the system called it constantly, at tier-1 prices, to pass frames between stages. Scale it up and the bill scaled accordingly — except the architecture never got to scale up, because it hit account limits at five percent of projected load first.
The fix was conceptually simple. As Kolny later put it: “We realized that distributed approach wasn’t bringing a lot of benefits in our specific use case, so we packed all of the components into a single process.” Everything moved to EC2 and ECS, transferring video frames through shared memory instead of object storage. Infrastructure cost fell by over 90 percent.
The industry’s reaction was disproportionate to the size of the change. Amazon is not a startup forced to cut corners. Amazon is the company whose 2002 API mandate reorganized an entire codebase around service boundaries and whose engineers built AWS almost as a side effect of solving their own distributed-systems problems. When Amazon publishes a post arguing, in effect, that it over-engineered a workload by going serverless, the implicit admission carries weight.
Kolny was careful to say the decision applied to this specific workload. But careful qualifications don’t travel as fast as the headline. Within a day, DHH had a response published calling the original pattern “madness in almost all cases” for teams without Amazon’s scale. He had made this argument before: in a February 2016 Signal v Noise essay he titled The Majestic Monolith, he had written “don’t distribute your computing. At least if you can in any way avoid it.” The 2023 Prime Video post gave him a seven-year-delayed co-author.
The debate that followed was not really about monoliths or microservices. It was about a question that any architectural choice eventually forces: what does this cost you to operate, not just to deploy? Microservices make each component easier to change independently, but they make the whole system harder to reason about, harder to monitor, and expensive when every function call becomes a network hop with a price tag.
By 2024, surveys found roughly 40 percent of organizations that had adopted microservices were consolidating services back into larger units. The pattern acquired a name — the modular monolith — and a rough formula: deploy as one, design as many.
The word “monolith” had been an insult in engineering circles since at least 2014. After May 2023, it was permitted, again, to be a compliment.
Sources
- Prime Video Switched from Serverless to EC2 and ECS to Save Costs — InfoQ — Marcin Kolny’s case study: the original architecture, the five-percent scaling wall, the consolidation, and the 90% cost reduction.
- Even Amazon can’t make sense of serverless or microservices — DHH — DHH’s May 2023 response calling the distributed-within-a-team approach “madness in almost all cases.”
- The Majestic Monolith — DHH / Signal v Noise — February 2016 essay coining the term and arguing against premature distribution for small teams.
- The Modular Monolith 2026 Complete Guide — DEV Community — Survey data on the 2024 industry consolidation trend and the modular monolith pattern.