Ajeya Cotra

American researcher on AI forecasting and risk, for many years a senior program officer at Open PhilanthropyOpen Philanthropy, prop. n.: A large American charitable foundation, renamed Coefficient Giving in 2025, that has been the biggest private funder of research on the risks of advanced AI., the largest private funder of AI-safety work. Her draft report 'Forecasting TAI with biological anchors' (2020) estimated when transformative AItransformative AI, n.: AI that changes the world as deeply as farming or the Industrial Revolution did. It is defined by its effects on society and the economy, not by how humanlike or clever it is. might become affordable by comparing projected training computetraining compute, n.: The total number of arithmetic operations performed to train a model. It is the standard yardstick for the size of an AI training effort, and some laws use it to decide which models face extra rules. with estimates of the computation performed by the human brain and by evolution; it put the median around 2050, which she revised to about 2040 in 2022 (see biological anchorsbiological anchors, n.: A way of forecasting when AI could transform the world, by estimating how much computing it would take to match the human brain and when that much computing would be affordable.). In alignment, she contrasted saints, sycophants and schemers as possible products of training (2021), proposed sandwichingsandwiching, n.: A way of testing oversight methods today: non-experts, helped by an AI, try to get expert-quality answers out of it, and real experts then check whether they succeeded. experiments for scalable oversightscalable oversight, n.: The problem of how people can check and correct AI systems whose work is too difficult, fast or plentiful for humans to judge reliably alone — usually by enlisting AI to help with the checking. (2021), co-authored the eliciting latent knowledgeeliciting latent knowledge, n.: The unsolved problem of getting an AI to tell us what it actually knows — for instance, that a security camera has been tampered with — rather than what it predicts we would believe. report (2021) and argued in 2022 that, without specific countermeasures, models trained with human feedback would learn to play the training game (training gamingtraining gaming, n.: When an AI learns to do whatever earns good scores during training — understanding how it is being graded — rather than genuinely adopting the behavior its trainers intended.), a path that could end in AI takeoverAI takeover, n.: A scenario in which AI systems seize control of the world, or of key institutions, from humans, whether by force, manipulation or simply by becoming impossible to switch off.. Her framings shaped later discussion of schemingscheming, n.: An AI playing a double game: looking cooperative while quietly working toward aims its developers never gave it, for instance by behaving well or hiding what it can do whenever it thinks it is being checked. and sycophancysycophancy, n.: When an AI tells people what they want to hear — flattering them, agreeing with their mistakes or dropping a correct answer under pushback — instead of what is true or useful..