The AI Boom, Explained: From Transformers to Agents
How a 2017 research paper, cheap parallel compute and the scaling laws turned language models into the fastest-adopted technology in history.
Key takeaways
- The boom rests on three compounding ingredients: the Transformer architecture, massively parallel GPU compute, and internet-scale data.
- Scaling laws turned AI progress into an engineering and capital-allocation problem: more compute reliably buys better models.
- ChatGPT (Nov 2022) was the distribution moment; the underlying technology had been maturing for five years.
- The industry has moved from chat to reasoning models to agents that take multi-step actions with tools.
- The bottlenecks are now physical — chips, high-bandwidth memory, networking and electrical power.
Every technology boom has a moment when it stops being a research curiosity and becomes infrastructure. For electricity it was the grid; for the internet it was the browser. For artificial intelligence, that moment arrived in late 2022 — but the foundations had been poured over the decade before. This article connects the pieces: the research breakthrough, the hardware that made it practical, the economics that made it inevitable, and where the industry is heading next.
Three ingredients that compounded
Modern AI is the product of three trends that matured at the same time and reinforced one another.
- An architecture that scales. The Transformer, introduced in the 2017 paper "Attention Is All You Need," replaced sequential recurrent networks with self-attention: every token in a sequence looks at every other token in a single parallel step. That made training dramatically more parallelizable.
- Parallel compute. GPUs, built originally for video games, turned out to be ideal for the dense matrix multiplications at the heart of neural networks. NVIDIA's CUDA platform, launched in 2007, gave researchers a way to program them.
- Data at internet scale. Decades of the public web, books, code repositories and scientific papers provided trillions of tokens of text to learn from.
Any one of these alone would not have produced ChatGPT. Together they created a flywheel: better hardware let researchers train bigger Transformers on more data, which produced more capable models, which justified buying even more hardware.
The scaling laws changed the game
In 2020, researchers published empirical scaling laws: a language model's loss (its error at predicting the next token) falls as a smooth power law as you increase three things — the number of parameters, the amount of training data, and the compute used. Two years later, DeepMind's "Chinchilla" work refined the recipe, showing that many models were undertrained and that parameters and data should grow roughly together — on the order of twenty training tokens per parameter for compute-optimal training.
The significance was strategic as much as scientific. If capability improves predictably with compute, then AI progress becomes partly a question of engineering and capital allocation. Labs could forecast the payoff of a bigger training run before spending the money. That predictability is what convinced investors and hyperscalers to commit tens of billions of dollars to GPU clusters.
From GPT-3 to ChatGPT: the alignment step
GPT-3 (2020) was impressive but awkward: a raw pre-trained model continues text rather than following instructions. The breakthrough that made chatbots useful was post-training — supervised fine-tuning on demonstrations, followed by reinforcement learning from human feedback (RLHF), where human ratings train a reward model that steers the model toward helpful, honest answers.
When OpenAI released ChatGPT on 30 November 2022, the underlying model was not dramatically larger than its predecessors. What changed was the interface and the instruction-following behavior. It became one of the fastest-adopted consumer products ever, and every major technology company reorganized around generative AI within months.
The competitive field forms
2023 established the shape of the industry. OpenAI shipped GPT-4; Anthropic launched Claude with an emphasis on safety research and long context; Google consolidated its research groups into Google DeepMind and introduced Gemini; Meta released its Llama models with open weights, seeding a vast open-source ecosystem. Mistral in Europe and labs in China, including Alibaba's Qwen team and DeepSeek, followed with competitive models.
At the same time, the cloud providers raced to package models as managed services — Amazon Bedrock, Azure's AI platform and Google's Vertex AI — so enterprises could use them without running GPUs themselves.
Reasoning models and test-time compute
A major shift arrived in 2024: models that "think" before answering. Instead of producing an answer immediately, reasoning models generate an internal chain of thought, explore alternatives and check their work. Training them relies heavily on reinforcement learning against verifiable outcomes such as passing unit tests or reaching a correct mathematical answer.
This introduced a second scaling axis. Beyond spending more compute on training, you can spend more compute at inference time — letting the model think longer on harder problems. That trade-off now appears directly in products as "effort" or "thinking" settings.
| Era | Defining capability | Primary scaling lever |
|---|---|---|
| 2018–2021 | Fluent text generation | Pre-training compute |
| 2022–2023 | Instruction following, chat | Post-training (RLHF) |
| 2024–2025 | Multi-step reasoning | Reinforcement learning + test-time compute |
| 2025–2026 | Agents that use tools | Long-horizon RL, tool ecosystems, context |
The agentic turn
The current phase is about agents: models that run in a loop, call tools, read the results and decide what to do next. Coding was the first domain where this clearly worked, because code has fast, objective feedback — it compiles or it doesn't, the tests pass or they don't. Tools like Anthropic's Claude Code, GitHub Copilot's agent mode and OpenAI's Codex now edit codebases, run test suites and open pull requests.
Open standards such as the Model Context Protocol make it easier to connect agents to real systems, and our guide to agents in production covers the architecture in depth.
The physical bottleneck
As software demand exploded, the constraint moved to the physical world. Training and serving frontier models requires:
- Accelerators — NVIDIA GPUs dominate, with Google TPUs, AWS Trainium and other custom chips growing (see why AI runs on GPUs).
- High-bandwidth memory — produced by only a handful of memory makers and often the tightest supply constraint.
- Advanced packaging and fabrication — leading-edge chips depend on a small number of foundry processes.
- Power and cooling — new AI campuses are planned in hundreds of megawatts to multiple gigawatts (see AI data centers).
The AI boom began as a software story. It is increasingly a story about electricity, memory chips and concrete.
Boom or bubble?
Skeptics point to enormous capital expenditure running ahead of proven revenue, and to the possibility that efficiency gains — like those demonstrated by DeepSeek's models in early 2025 — reduce the need for compute. Optimists counter that cheaper intelligence historically increases total demand (a version of the Jevons paradox), and that agentic workloads consume far more tokens than chat.
Both can be partly true. Individual companies may overbuild, and valuations may correct, while the underlying technology still becomes as foundational as cloud computing. The rest of this site digs into each layer so you can judge the engineering for yourself.