What exactly are scaling laws, and is this the same thing as "bigger models are always better"?
Scaling laws are a quantifiable, predictive mathematical relationship — not the vague intuition that "bigger is always better." OpenAI's 2020 Kaplan paper demonstrated that for a fixed model architecture, loss decreases as a predictable power law as parameters, dataset size, and training compute increase. In other words, you can roughly forecast, before training even starts, how much performance gain a given amount of compute will buy.
The difference from "bigger is always better" is that scaling laws have diminishing returns built in from the start — a power-law curve looks like a straight line on a log-log plot, but on a linear scale it's actually a curve with fast early gains and slow later ones. Scaling laws never promised unlimited linear improvement; they simply made the rate of diminishing returns predictable.
Why did the industry invent "test-time compute" as a new direction? Why not just keep making models bigger?
You certainly can keep scaling models up directly, but the marginal returns have clearly declined — and there's a hard limit behind it: the data wall. Training a one-trillion-parameter model at the industry-standard Chinchilla-optimal ratio (roughly 20 training tokens per parameter) requires about twenty trillion tokens, while the total stock of high-quality text data available on the internet is estimated at somewhere between ten and fifty trillion tokens. In other words, simply making models bigger will soon run into a physical ceiling of "not enough clean data to feed it" — a problem money alone can't solve.
Test-time compute offers a different path: instead of dumping a massive one-time compute budget into training a bigger fixed model, compute gets spread across every individual inference — the model spends a few extra seconds reasoning at the moment it answers a question, trading a tunable amount of compute for accuracy. And this path hasn't visibly hit the same data ceiling yet.
What exactly happened with GPT-4.5 being pulled from the API, and how does it connect directly to scaling laws hitting a wall?
GPT-4.5 (internally codenamed Orion) launched in February 2025 as OpenAI's largest non-reasoning model in terms of training data and compute at the time, and it did outperform its predecessor GPT-4o on benchmarks like factual accuracy. But its inference pricing was $75 per million input tokens and $150 per million output tokens — roughly 38 times more expensive than GPT-4.1, which launched later at $2/$8 per million tokens. OpenAI announced GPT-4.5's API deprecation on the very same day it launched GPT-4.1, with a shutdown date of July 14, 2025 — just four and a half months after release, making it the shortest-lived model in OpenAI's commercial history.
This case is frequently cited as a textbook example of scaling laws hitting a wall precisely because it demonstrates so cleanly that "technical improvement" and "commercially worthwhile" are two different things. GPT-4.5 did achieve a measurable performance gain by throwing more compute at the problem — but that gain couldn't justify a proportional increase in inference cost, and the market ultimately voted with its feet for a cheaper alternative that performed nearly as well at roughly 1/38th the price.
How does this debate over scaling laws hitting a wall relate to how ordinary readers should think about AGI timeline progress?
If you've been tracking "how far away is AGI" through media headlines, this debate is a useful reminder: the total amount of compute being invested continuing to rise is not the same thing as the rate of improvement along any single dimension continuing to accelerate. The pre-training scaling curve has genuinely flattened over the past couple of years, but that doesn't mean overall AI progress has stalled — the industry has simply spread its resources across multiple independent axes (training scale, data efficiency, test-time reasoning, post-training technique). One axis slowing down doesn't mean the whole picture is slowing down, but it also doesn't guarantee the whole picture is accelerating either — there's real room for judgment here.
A more practical way for readers to evaluate this is: don't take a lab's announcement of "we trained an even bigger model" as a standalone signal of progress. Instead, watch whether that lab is making solid gains across multiple axes simultaneously — a company that can only generate headlines by making its model bigger is, if anything, showing a relatively fragile position under diminishing returns to scale.
For most of the last decade, AI labs ran on one formula that seemed to work almost universally: make the model bigger, feed it more data, throw more compute at it, and performance would improve in a predictable way. This relationship became known as scaling laws — first demonstrated systematically in OpenAI’s 2020 Kaplan paper, which showed that for a fixed architecture, loss decreases as a predictable power law as parameters, dataset size, and training compute increase. The finding turned compute investment into something you could actually forecast, rather than a bet made on intuition.
The problem is that this curve has visibly flattened over the past two years. Multiple analyses published in 2026 point out that doubling training compute now buys noticeably less improvement than the previous doubling did — which is exactly what a power law predicts once you look at it on a linear scale rather than a log-log plot: massive early gains that flatten out quickly, with each further doubling of compute returning less and less. The most frequently cited example is OpenAI’s GPT-4.5: trained with more data and compute than its predecessor, it did post real gains on benchmarks like factual accuracy, yet it was pulled from the API just four and a half months after launch because inference costs were too high relative to the actual gains delivered. OpenAI chief scientist Ilya Sutskever said at NeurIPS in late 2024 that “pre-training as we know it will end” — a line that has since been quoted repeatedly as the moment the industry publicly acknowledged the curve was flattening.
That doesn’t mean compute investment has lost its point. The industry’s response wasn’t panic; it was redirecting compute toward a new axis: letting models think longer at the moment they generate an answer, rather than only spending more during training. This approach is called test-time compute — running more reasoning steps during inference, trading inference-time compute for accuracy. Between 2024 and 2026, this newer curve still looks comparatively steep, without the sharp flattening signal that pre-training scaling has already shown — which is also why data center buildout in 2026 has been shifting away from pure training-GPU stacking and toward infrastructure that can support large-scale, real-time inference.
The research community remains split on this. One camp argues that brute-force scaling along a single axis has genuinely hit a practical ceiling — the clearest evidence being the “data wall”: training a one-trillion-parameter model at the Chinchilla-optimal ratio would require roughly twenty trillion tokens, while the total stock of high-quality text data on the internet is estimated at somewhere between ten and fifty trillion tokens. The ceiling is visible. The other camp points out that scaling laws themselves haven’t broken down — only the brute-force single-axis approach has. As long as compute is split across multiple independently stackable axes — training scale, data efficiency, post-training, and test-time reasoning — the overall progress curve can stay steep, it just no longer depends on “making the model bigger” as the only lever.
If you’re evaluating an AI-related investment or procurement decision, a company still competing purely on raw parameter count is itself a dated signal — the sharper question is how many independent axes its compute strategy spans (training scale, data efficiency, test-time reasoning, post-training technique) rather than whether it’s betting everything on one. The GPT-4.5 story is also a reminder that improving on technical benchmarks doesn’t automatically mean the economics work out: a model can post record scores and still get pulled from the market in under half a year if inference costs never come down.