# Parameters In AI, **parameters** (also called **weights**) are the numerical values inside a neural network that encode everything the model has learned during training. When a [[Large Language Model]] processes text and produces a response, every word choice is determined by billions of mathematical operations across these parameters. They are the model's knowledge, compressed into numbers. Parameter counts have been a crude proxy for model capability: GPT-2 had 1.5 billion, GPT-3 had 175 billion, GPT-4 reportedly has ~1.8 trillion (across a mixture-of-experts architecture), and Meta's Llama 3.1 405B has 405 billion. Anthropic does not publicly disclose Claude's parameter count. ## What Parameters Actually Are Each parameter is a floating-point number (typically 16-bit or 32-bit) that represents one learned weight in the neural network. During training, these numbers are adjusted billions of times through gradient descent — the mathematical process of incrementally improving the model's predictions. After training, the parameters are frozen: they encode the model's learned patterns and are used as-is during [[Inference vs Training]]. The total parameter count is a function of the model's architecture — the number of layers, attention heads, and hidden dimensions in the [[Transformer Architecture]]. More parameters mean more capacity to store patterns, but also more compute required for training and inference, more memory required to run the model, and more energy consumed per query. ## The Parameters Arms Race Parameter count was the primary axis of AI competition from 2019–2023, driven by [[Scaling Laws]] showing that larger models performed better. This drove the progression from GPT-2 (1.5B) to PaLM (540B) to GPT-4 (~1.8T), with each generation requiring exponentially more compute and capital. The arms race shifted after 2023. Mixture-of-experts architectures (where only a fraction of parameters activate for any given input) made raw parameter count less meaningful as a capability indicator. DeepSeek's R1 demonstrated strong performance from smaller models with efficient training. "Distillation" techniques allow smaller models to learn from larger ones. The focus moved from "how many parameters" to "how efficiently are the parameters used." This shift matters for the KB's power-concentration analysis: if capability depends purely on parameter count, only the richest labs can compete. If efficiency innovations can substitute for scale, the frontier becomes more accessible — to smaller labs, open-weight projects, and nations under GPU export controls. ## KB Significance 1. **The unit of AI capital.** Parameters are the stored product of training compute — the crystallized result of millions of dollars in GPU time, energy, and data. When Meta releases Llama's weights as [[Open-Weight AI]], it is releasing the product of that capital expenditure for free. When AI companies keep weights proprietary, they are protecting their most valuable asset. 2. **The memory and energy cost.** Running a model requires storing its parameters in GPU memory. A 70B-parameter model in 16-bit precision requires ~140GB of GPU memory — multiple high-end GPUs just to load. A 1.8T-parameter model requires entire GPU clusters. These hardware requirements are the physical reason that frontier AI is expensive to deploy and concentrated in the hands of well-capitalized organizations. ## See Also - [[Scaling Laws]] — The relationship between parameters and performance - [[Open-Weight AI]] — Releasing parameters publicly - [[Fine-Tuning]] — Modifying parameters after initial training - [[Transformer Architecture]] — The architecture that defines parameter structure