# Inference vs Training **Training** is the process of building an AI model — feeding it data and adjusting its [[Parameters]] over weeks or months on massive GPU clusters. **Inference** is the process of using that trained model — generating a response to a user's query, analyzing a document, producing an image. Training happens once (per model version); inference happens millions of times per day. The distinction matters because training and inference have different economic structures, different hardware requirements, and different strategic implications — and the balance between them is shifting in ways that reshape the AI industry's economics and geopolitics. ## The Economics **Training costs** are massive but one-time: $100M–$1B+ for a [[Frontier Model]], paid upfront before the model generates a single dollar of revenue. Training is the capital expenditure of AI — the factory construction. Only organizations with access to enormous capital can afford frontier training runs. **Inference costs** are smaller per query but continuous: every time a user sends a message to ChatGPT or Claude, the provider pays for the GPU time to process it. At scale (hundreds of millions of users), inference costs dwarf training costs. OpenAI reportedly spends more on inference than training in any given month. Inference is the operating expenditure of AI — the factory's electricity bill. This creates a distinctive economic pattern: AI labs spend billions to train a model, then must serve it cheaply enough to attract users while expensively enough to recover costs. The per-token pricing wars of 2024–2025 (Google, Anthropic, and OpenAI aggressively cutting API prices) reflect competition to capture inference volume — the recurring revenue stream. ## The Inference-Time Scaling Shift A major development in 2024–2025 was the discovery that you can improve model performance by spending more compute at inference time rather than only at training time. OpenAI's o1 and o3 models, and Anthropic's extended-thinking features, give the model more time to "think" — generating and evaluating multiple solution paths before committing to an answer. This partially decouples capability from training cost: a smaller, cheaper-to-train model can produce better results by thinking longer at inference time. The trade-off is that inference-time scaling makes each query more expensive — sometimes dramatically so (complex reasoning queries can cost 10–100× more than simple ones). The strategic implication is significant. If capability requires enormous training runs, only the richest labs can compete. If capability can be achieved through inference-time scaling, smaller models become viable competitors — including open-weight models and models trained by labs with limited compute budgets (including Chinese labs under GPU export controls). ## KB Significance 1. **The business model foundation.** The training/inference split is the fundamental economics of AI. Training costs create barriers to entry (only rich labs can build frontier models). Inference costs determine whether AI products are profitable at scale. The pricing wars, margin pressures, and capital requirements documented in the KB's tech-industry coverage all flow from this split. 2. **The energy driver.** Both training and inference consume enormous amounts of energy, but at different scales and patterns. Training is a burst (months of maximum GPU utilization). Inference is continuous (24/7 serving of user queries). The [[The Quest for 90 Gigawatts]] documented in the KB is driven by both — but increasingly by inference as AI adoption scales. 3. **The geopolitical variable.** Inference-time scaling changes the compute sovereignty equation. If smaller models with more inference compute can match frontier models, then GPU export controls become less effective at maintaining the U.S. advantage — nations and labs with modest training compute can compensate at inference time. ## See Also - [[Scaling Laws]] — The relationship between compute and capability - [[Parameters]] — What training produces - [[Context Window]] — A major driver of inference cost - [[Compute Sovereignty]] — How training/inference requirements map to national capability