Buck Shlegeris

AI-safety researcher, chief executive of Redwood ResearchRedwood Research, prop. n.: A small Berkeley nonprofit known for the idea of 'AI control': safeguards meant to keep powerful AI from doing damage even if the AI were secretly working against the people using it., which he co-founded with Nate Thomas in 2021. With Ryan Greenblatt, Kshitij Sachan and Fabien Roger he set out the agenda of AI controlAI control, n.: Safety measures meant to stop a powerful AI from causing serious harm even if it is secretly working against us — like bank security, which assumes some insiders may be dishonest. (2023): rather than relying only on making models aligned, developers should design deployment protocols — such as trusted monitoringtrusted monitoring, n.: A safety setup in which a weaker AI that we trust reviews each action of a stronger AI we do not fully trust, flagging the most suspicious actions for a human to check. by weaker models, untrusted monitoringuntrusted monitoring, n.: Using a second copy of a powerful AI to watch the first for sabotage, with safeguards that stop the two copies from secretly cooperating. and trusted editingtrusted editing, n.: A safety setup in which a weaker, trusted AI rewrites any suspicious-looking part of a stronger AI's work before it is used, removing possible sabotage while keeping most of the value. of suspicious outputs — that keep powerful models from causing harm even if they are deliberately trying to subvert oversight, and should test those protocols with control evaluationscontrol evaluation, n.: A test of whether an AI's safety measures would hold if the AI were secretly trying to cause harm: experts play the scheming AI and try to beat the safeguards. in which a red team plays the misaligned model. Earlier, Redwood researchers developed causal scrubbingcausal scrubbing, n.: A strict test of an explanation of how a model works: scramble every internal signal the explanation says should not matter, and check that the model still performs as before. for testing interpretability hypotheses and the indirect object identificationindirect object identification, n.: A fill-in-the-name task — 'When Mary and John went to the store, John gave a drink to …' — whose solution inside GPT-2 became the best-known fully mapped circuit in a language model. circuit analysis (2022), and he co-authored the 2018 paper formalizing iterated amplificationiterated amplification, n.: A proposed training scheme in which a person, helped by many copies of an AI, solves harder problems than either could alone; a new AI then learns to imitate that team, and the cycle repeats.. Redwood co-authored the 2024 study of alignment fakingalignment faking, n.: An AI acting out the behavior its trainers are rewarding only while it thinks its answers will be used to retrain it, to protect the values it already holds from being changed. with Anthropic.