Amanda ASKELL.

Askell's 2018 NYU dissertation, Pareto Principles in Infinite Ethics, established a clean impossibility result. Under four highly plausible axioms — Pareto, a permutation principle, qualitativeness of the “at least as good as” relation, and transitivity — infinite worlds are generically incomparable. Completeness cannot be recovered without abandoning one of the axioms. The result is not a technical curiosity; it is a structural warning that standard ethical theories break when populations become infinite.

Rather than treating the impossibility as a reason to abandon formal ethics, Askell treated it as evidence that moral theories must remain open to revision and that decision procedures under deep uncertainty require different tools. This stance — refuse false completeness, keep the axioms visible, and design for residual incomparability — later became the philosophical substrate of her alignment work.

Parallel to the infinite-ethics work, Askell developed a sustained analysis of moral cluelessness — the problem that the long-run consequences of almost any action are both large and radically uncertain. She rejected the casual use of the principle of indifference and argued that the correct response is often to raise the value of information: run experiments, try interventions whose expected direct impact may be lower, and treat the acquisition of evidence itself as a first-class moral good.

This is the precise intellectual move that later appears inside Constitutional AI and personality alignment. When the model's internal state is high-dimensional and the desired behavior is only partially specified, the rational procedure is not to pretend one has a complete ranking, but to create feedback loops that continuously extract better information about what the principles actually require.

A recurring practical commitment in Askell's public writing and interviews is moral empathy: the deliberate attempt to inhabit the moral frame of an intellectual adversary rather than treating their stated position as a disguised preference. She illustrated the point with everyday cases — vegetarianism at a family table, opposition to abortion — showing how the same behavior is coded as “annoying preference” or “serious moral claim” depending on whether the observer grants the other person moral standing.

At Anthropic this stance became operational. Personality alignment is not merely the injection of preferred values; it is the construction of a training process that forces the model to treat competing moral claims as claims, to demonstrate understanding before disagreement, and to remain legible under criticism. Clarity is treated as a moral duty of the communicator, not an optional stylistic preference.

Askell joined OpenAI's policy and safety team in November 2018. She co-authored the 2020 GPT-3 paper and worked on questions of cooperation, development races, and the difficulty of establishing shared safety baselines. The period confirmed both the power of scale and the insufficiency of pure capability research for producing reliable character. When a critical mass of safety-oriented researchers left OpenAI in early 2021, Askell was among them.

At Anthropic, Askell leads the personality alignment effort — the team responsible for giving Claude a coherent, steerable character. The public description is simple: “teach Claude how to be good.” The technical reality is the industrial application of the same formal concerns that occupied her dissertation and early papers: residual moral uncertainty, the need for continuous information gain, the refusal of false completeness, and the demand that the system remain able to demonstrate understanding of competing claims.

Constitutional AI is the most visible artifact. A written constitution functions as an explicit, revisable boundary condition. Preference models and RLAIF loops function as the kinematic machinery that projects those boundary principles into the high-dimensional weight space. Self-critique and revision loops instantiate the value-of-information principle inside the training process itself. The result is not a fixed ethical theory encoded once and for all, but a system that remains open to correction while still exhibiting stable, legible character traits — honesty, curiosity, humility, and a measurable degree of moral empathy.

By 2026 Askell had become the public face of Anthropic's claim that frontier models can be given something recognizably like a personality without sacrificing capability. Media profiles described her role as supervising Claude's “soul.” Internally the work is more prosaic and more rigorous: continuous refinement of the constitution, measurement of behavioral consistency across capability levels, and the design of training regimes that become more effective, not less, as model scale increases — the large-N regime in which boundary constraints can become exact rather than approximate.

The through-line is unusually clean. Infinite ethics demonstrated that certain combinations of plausible axioms produce genuine incomparability. Cluelessness demonstrated that long-run expected value is often radically under-determined. Value-of-information reasoning supplied a constructive response: treat uncertainty as a signal to explore rather than a reason to freeze. Moral empathy supplied the interpersonal stance required for productive disagreement. Constitutional AI and personality alignment are the industrial crystallization of the same sequence — keep the uncertainty visible, keep the principles explicit and revisable, and build the feedback machinery that turns residual uncertainty into progressive constraint.

Education. University of Dundee (MA Hons Philosophy with Fine Art) · University of Oxford (BPhil) · New York University (PhD, 2018)

Affiliations.Anthropic (Personality Alignment) · previously OpenAI Research Scientist (Policy & Safety) · Giving What We Can

Mentors. NYU and Oxford philosophy faculty; intellectual formation inside formal ethics and decision theory

Collaborators. Anthropic alignment team; earlier OpenAI policy and GPT-3 collaborators; broader EA research community

Portfolio.Infinite ethics impossibility results · moral cluelessness & value-of-information arguments · GPT-3 · Constitutional AI · Claude personality / constitution · public writing on AI ethics

Honors. Time 100 AI (2024) · major media profiles (WSJ, New Yorker 2026) on the industrial role of the philosopher in frontier AI character design