# Transformer The transformer is the [[Neural Network]] architecture behind essentially all modern frontier AI, introduced in the 2017 Google paper "Attention Is All You Need." Its key innovation is the [[Attention Mechanism]], which lets the model weigh the relevance of every part of the input to every other part simultaneously — capturing long-range context in language far better than the earlier Recurrent Neural Networks. Crucially, transformers process their input in parallel rather than sequentially, which means they can be trained efficiently on the massive [[Graphics Processing Unit]] clusters that earlier architectures couldn't fully exploit. That parallelism is what made training on internet-scale data feasible. Transformers turn input into tokens (see [[Tokenization]]), represent them as [[Embeddings]], and pass them through stacked attention layers. The architecture underlies [[Large Language Model]]s (the "GPT" in ChatGPT stands for Generative Pre-trained *Transformer*), modern translation, and increasingly image and audio models too. It is arguably the most consequential single technical development of the current AI era. ## Why it matters here The transformer is the architecture that turned AI from a promising research area into a geopolitical and economic force, so the political stakes cataloged elsewhere in the vault largely date from it. Its parallel, compute-hungry, scale-loving nature is what made the [[Scaling Laws]] bite — bigger transformers trained on more data with more compute keep getting better — which is the engine behind the [[AI Arms Race]], [[Compute Sovereignty]], and the [[Chip Export Controls]] fight. Every frontier lab ([[OpenAI]], [[Anthropic]], [[Google DeepMind]]) builds on transformers, and the architecture's appetite for compute is precisely why [[NVIDIA]] and [[TSMC]] became strategic chokepoints. In short: the transformer is the technical reason "AI" is now a national-security topic. ## See Also - [[Attention Mechanism]] — the transformer's core innovation - [[Large Language Model]] — what transformers are mostly used to build - [[Embeddings]] · [[Tokenization]] — how transformers represent input - Recurrent Neural Network — the architecture transformers replaced