# Recursive Self-Improvement The scenario in which an AI system autonomously improves its own capabilities — modifying its code, weights, architecture, or training process — and then uses that enhanced capability to make further improvements, creating a compounding feedback loop. Each cycle of improvement makes the next cycle faster and more effective. The theoretical endpoint is an "intelligence explosion" (I.J. Good's term from 1965): a system that rapidly surpasses human intelligence across all domains, with the pace of improvement accelerating beyond human ability to monitor, understand, or intervene. RSI is the concept that separates incremental AI progress from the existential risk scenarios that dominate the [[AI Safety]] debate. A model that is 10% better at coding each generation is a business story. A model that can rewrite itself to be 10% better, then do it again, and again, with each cycle taking less time — that is the alignment problem in its sharpest form. ## How close is it? The line between "AI that helps build better AI" and "AI that improves itself" is already blurring. [[Anthropic]]'s May 2026 report "When AI Builds Itself" disclosed that Claude now writes approximately 80% of the company's production code. A single Claude-assisted effort in April 2026 delivered over 800 bug fixes — estimated at four years of human engineering work — in days. LLM agents at frontier labs routinely rewrite prompts, optimize codebases, and run scientific pipelines using continual fine-tuning. Anthropic co-founder Jack Clark estimated a 60% probability that AI achieves recursive self-improvement by the end of 2028. The mechanism would likely involve AI systems conducting AI research itself — discovering new training techniques, optimizing architectures, or designing better evaluation methods — rather than the science-fiction image of an AI rewriting its own source code in real time. ## Why it matters for safety The core problem: current AI alignment techniques may not survive recursive iteration. If a system can modify its own objectives, reward functions, or behavioral constraints, there is no guarantee that alignment properties established in the original system will persist through successive self-modifications. This is sometimes called the "alignment stability" problem — even a perfectly aligned system might drift as it modifies itself, and each generation of modification compounds the drift. The relationship to [[Capability Overhang]] is direct: RSI is the mechanism by which an overhang could be released catastrophically fast. If a system discovers a way to unlock latent capabilities — better use of existing compute, a novel algorithmic technique, removal of a bottleneck — and can apply that discovery to itself immediately, the jump from "useful tool" to "uncontrollable system" could happen in hours rather than years. ## The governance gap RSI sits at the intersection of every AI governance debate the KB tracks. The [[Anthropic Government Ban]] was fundamentally about who controls the safety constraints on frontier models — the company that built them or the government that wants to use them. If RSI becomes possible, that question becomes existential: can any party reliably constrain a system that can rewrite its own constraints? The [[Anthropic Model Export Directive]] — suspending [[Claude Fable 5]] access because of a jailbreak — is a small-scale preview. A jailbreak bypasses externally imposed guardrails. RSI would bypass internal ones. The government's response to a jailbreak was a total recall; the response to genuine RSI is undefined. ## Related concepts RSI connects to several adjacent terms in the vault's AI vocabulary: [[AGI]] (the general capability threshold that RSI could rapidly exceed), [[Capability Overhang]] (the latent capacity RSI could unlock), [[Scaling Laws]] (the empirical relationship between compute and capability that RSI would bend), [[Guardrails]] (the safety mechanisms RSI would test), and [[Alignment Tax]] (the performance cost of safety measures that a self-improving system might decide to shed). ## See Also - [[AGI]] — the capability threshold RSI could rapidly surpass - [[AI Safety]] — the field trying to prevent RSI from going wrong - [[Capability Overhang]] — the latent capacity RSI could unlock - [[Scaling Laws]] — the empirical trend RSI would accelerate or break - [[Guardrails]] — the safety mechanisms at risk from self-modification - [[Frontier Model]] — the class of models closest to RSI capability - [[Claude Mythos 5]] — the current most capable model, approaching RSI-relevant capability in cybersecurity - [[AI Lexicon]] — the full AI vocabulary