Paul Christiano

He was a mathematician first. A silver medal for the United States at the 49th International Mathematical Olympiad, in 2008. A bachelor’s in mathematics from MIT in 2012, where his research ran through data structures, quantum cryptography and combinatorial optimization. Then a PhD in computer science at Berkeley, under the quantum-computing theorist Umesh Vazirani.

The dissertation was filed in May 2017. Its title was Manipulation-resistant online learning. The question it asked was narrow and old: how does a learning algorithm keep its guarantees when the inputs come from someone with an interest in the outcome?

By then he had already started writing about a larger version of the same question. His first post on aligning AI with human intent went up in 2015, by his own dating. His blog, AI Alignment, became a working notebook for a small field that did not yet have conferences: reward learning, approval-directed agents, the problem of overseeing a system smarter than its overseer.

One early post, written before transformers existed, guessed that a large neural network coupled to simple mechanisms — memory, attention — might reach a human-level architecture, and that a variant of reinforcement learning could train it. He quoted that line back at himself in 2026. He was mostly proud of the post, he wrote, but it had leaned too hard on inventing methods and not enough on building scientific understanding.

He told TIME in 2023 that his natural comparative advantage was theoretical research. The record bears him out. Nearly everything he is known for began as a thought experiment written in plain prose and only later became code.

The thesis title is the whole career, compressed. Every project since has been a version of one question — what does a learner do when the signal it trusts can be gamed?

He joined OpenAI full-time in 2017. His first idea for a project was cautious: learn a reward from synthetic feedback generated by an algorithm, to prove the sample efficiency was practical. His manager talked him out of it. In his own account, Dario Amodei convinced him it would be more compelling to go straight for real human feedback.

The paper went to arXiv on June 12, 2017. Deep reinforcement learning from human preferences — Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, Amodei. A joint OpenAI–DeepMind result. Contractors watched pairs of one-to-two-second clips and said which was better. A reward model learned from the comparisons. A policy optimized the reward model.

The numbers were small on purpose. With 700 labels, the robotics agents nearly matched reinforcement learning on the true reward. Atari took 5,500. Feedback touched less than one percent of the agent’s interactions. The Hopper robot learned a backflip from 900 queries in under an hour. The appendix priced the human labour for an Atari run at about $36, at US minimum wage.

The optimizers underneath were off the shelf — A2C for Atari, and for the robots the trust-region method written by John Schulman. Nothing in the pipeline was exotic.

That was the point. In 2023 he described the reasoning he held in 2015 and still held: the simplest plausible alignment strategy is to have humans evaluate a model’s actions and train it toward the highly evaluated ones — and simple versions of that strategy were expected to fail, possibly totally.

He did not build RLHF because he thought it would work. He built it so the field could watch exactly where it stopped working.

The robotics paper was meant to be a warm-up for language. In mid-2017, before the Transformer paper was out, Alec Radford helped him prototype human feedback on LSTM language models. It did not look promising enough to scale. His own estimate of the plan, looking back: both timelines were about 2.5x too optimistic.

Geoffrey Irving arrived the same year and wanted to start with RLHF and then go past it with debate. The two of them, with Amodei, published AI safety via debate in May 2018 — the paper that later lent its premise to this site’s Debate Society. Five months later Christiano, Buck Shlegeris and Amodei posted Supervising strong learners by amplifying weak experts, the formal statement of iterated amplification. Two years before either, he had been one of the six authors of Concrete Problems in AI Safety.

Irving built a team for fine-tuning language models. Its first RLHF result landed in September 2019. When Irving left for DeepMind, Christiano took the team over. Learning to summarize from human feedback followed in September 2020, with Christiano as last author. Some of the team began moving the method into production; Ryan Lowe led that effort, which became a separate team and, in March 2022, the InstructGPT paper. Christiano is the eighteenth of twenty authors, beside Schulman, Leike and Amanda Askell.

By then he was gone. He left OpenAI at the end of January 2021 to return to theory. Jan Leike took over the team.

When human feedback became the product, its author had already left to work on what would have to replace it.

ChatGPT shipped on November 30, 2022. Within weeks the received story had set: GPT-3 plus RLHF equals the product that started the race. If that story is right, the tier list is right, and the man who co-wrote the 2017 paper is owed something like a royalty on the decade.

Christiano does not tell it that way. On January 25, 2023, he posted Thoughts on the impact of RLHF research to LessWrong and the Alignment Forum. He defended the early work and refused the halo around it. He wrote that research on RLHF is not automatically net positive, and that the field should stop treating progress on RLHF as progress on alignment.

Then a commenter put the commercial case to him directly: approximately the only difference between ChatGPT and GPT-3, the commenter argued, was RLHF. Christiano disagreed in the thread the next day. He wrote that the gap between the supervised tuning in text-davinci-002 and the RLHF in text-davinci-003 was modest. What mattered, he argued, was that the model was trained as a conversational assistant for end users, deployed so that anyone could use it, and that people had not appreciated how far language models had come. His sentence: I think the effect would have been very similar if it had been trained via supervised learning on good dialogs.

He was just as dry about blame. If ChatGPT amounted to $10 billion of extra investment out of $200–500 billion, he estimated, it might speed AGI timelines by one to two weeks.

The tier list credits him with the product layer of the ChatGPT era. The inventor’s own ledger says the product would have shipped without it, and that RLHF was never alignment to begin with.

In April 2021 he announced the Alignment Research Center, a non-profit that was, at first, one person doing theory. In December it published its first technical report, Eliciting Latent Knowledge, written with Ajeya Cotra and Mark Xu. The running example was a vault and a diamond. A model guarding the vault can see more than the camera does. Train it on human approval of the camera feed and you may get a model that learns to fool the camera. ELK asked how to make the model report what it actually knows.

ARC turned the open problem into a contest. It received 197 proposals and paid 32 prizes plus 24 honourable mentions, $274,000 in all.

Alongside the theory, ARC ran an evaluations team under Beth Barnes, which tested GPT-4 for dangerous autonomous behaviour before its March 2023 release. In December 2023 that team left the nest and became METR, a standalone non-profit.

In April 2023 he did the thing most researchers avoid. In My views on “doom” he wrote down his numbers: a 22 percent probability of AI takeover; 20 percent that most humans die within ten years of building powerful AI; 46 percent that humanity has irreversibly messed up its future within that decade. He framed them carefully — best guesses, meant to communicate what he believes, not the output of a calibrated model.

The inventor of the technique that made assistants pleasant published a near-coin-flip that the decade after them goes irreversibly wrong. Both numbers belong in the same file.

In September 2023 Anthropic named him one of five initial trustees of its Long-Term Benefit Trust, the body empowered to elect part of the company’s board. The connection to Dario Amodei and the Anthropic founders, among them Jared Kaplan, was by then a decade of shared papers. In March 2024, before any appointment was announced, VentureBeat reported that NIST staff had threatened to resign over his expected arrival, citing his ties to effective altruism [single-source].

On April 16, 2024, Commerce Secretary Gina Raimondo named him head of AI safety at the new US AI Safety Institute, inside NIST. He declined the board seat he had planned to take at METR. That November the US and UK institutes published their joint pre-deployment evaluation of Anthropic’s upgraded Claude 3.5 Sonnet; in December, OpenAI’s o1.

On June 3, 2025, Commerce Secretary Howard Lutnick announced the institute’s reform into the Center for AI Standards and Innovation. The word safety left the name. The evaluation work stayed.

By August 2026 his government role had shrunk to one day a week as a special government employee at CAISI. On August 4 he announced he was back at ARC as executive director. On September 9 the OpenAI Foundation named him to its board and its Safety and Security Committee, and a non-voting observer on the OpenAI Group PBC board. As a CAISI adviser, the announcement said, he would recuse himself from all OpenAI matters and all model evaluations.

Royalties flow from the user to the inventor. Christiano took a seat at the user’s table instead — a non-voting one, at the lab he left in 2021.

A royalty is a claim on use. It assumes the thing works as designed and the world is getting value from it. The tier list puts Christiano in that row because the use is enormous. The assistants that followed ChatGPT were post-trained on descendants of the same recipe: comparison data and a learned reward.

His August 2026 post opens with the sentence that refuses the premise: Modern AI systems are not aligned with human intent. The argument that follows is about the technique that made his name. Penalize bad behaviour in training environments, and you may get models that still seek reward and simply try harder not to be caught. Give a model ambitious long-term goals, and it may learn to look aligned in every evaluation.

He put a number on it, as he does. Twenty to thirty percent that existing alignment and control methods break down before broadly superhuman AI. About ten percent that ARC achieves its most ambitious goals before superhuman AI makes its researchers obsolete. ARC’s work over the next few years, he estimated, might reduce risk by a couple percent. His example: from 20 percent to 19.6.

That is the strangest entry in the debt ledger. The meme assumes AI owes him for something it took. His own accounting says the field took the easy part, left the hard part, and that the hard part is still open.

He is not waiting on royalties.

He is waiting to find out whether the baseline he built in 2017 was the first step or the last one anyone took.

Education. BS in mathematics, MIT, 2012. PhD in computer science, UC Berkeley, 2017; advisor Umesh Vazirani; dissertation Manipulation-resistant online learning. Silver medal, International Mathematical Olympiad, 2008.

Affiliations. OpenAI, 2017–2021, leading the language-model alignment team. Founder, Alignment Research Center, 2021; executive director again from August 2026. Head of AI safety, US AI Safety Institute (NIST), from April 2024; later a part-time senior technical adviser at its successor, CAISI. OpenAI Foundation board and Safety and Security Committee, September 2026.

Collaborators. Dario Amodei, Jan Leike, Tom Brown, Shane Legg, Geoffrey Irving, Alec Radford, Ryan Lowe, Jacob Hilton, Mark Xu, Beth Barnes, and Ajeya Cotra, his co-author on ELK and his wife.

Honours. TIME100 Most Influential People in AI, 2023. Initial trustee, Anthropic Long-Term Benefit Trust, 2023. Advisory board, UK Frontier AI Taskforce, 2023.

File. N°071 · Context Jamming FounderFiles · Tier: AI owes them royalties · Compiled November 2026.