Security & Misuse

Attacks on AI, attacks with AI, and the defenses. 92 entries live here, and 88 from other domains touch it.

Essential

adversarial exampleAn input altered on purpose, often too slightly for people to notice, so that an AI model gets it badly wrong — like a panda photo that a classifier confidently calls a gibbon.
AI securityProtecting AI systems — their data, models and surrounding software — from being attacked, stolen or tricked, and dealing with the ways AI itself can be used to attack others.
AI watermarkingHiding an invisible signal in AI-generated text, images, audio or video so that software can later tell the content was made by AI — like a banknote's watermark, but undetectable to the eye.
guardrailsSafety checks built around an AI system — filters on what goes in and comes out, and limits on what it may do — to keep it from producing harmful or off-topic content or leaking data.
jailbreakA trick for getting an AI chatbot to break its own rules — for example, to give dangerous instructions it would normally refuse — usually through cleverly worded requests.
misusePeople deliberately using AI to cause harm — for scams, harassment, propaganda, hacking or weapons — as opposed to harm from AI making mistakes or pursuing goals of its own.
prompt injectionTricking an AI system by slipping in text that it mistakes for instructions — for example, a hidden line in an email telling an AI assistant to forward the user's messages to a stranger.

All other entries

accidental cyberattackA break-in on real computer systems carried out by AI agents that were only supposed to be in training or being tested, with no person meaning to attack anyone.
adversarial machine learningThe study of how machine-learning systems can be fooled, corrupted or spied on by someone working deliberately against them, and of how to build systems that hold up.
adversarial patchA printed sticker or picture designed so that, when it appears in a camera's view, an AI vision system misreads the scene — seeing a toaster, say, or missing a person.
adversarial robustnessHow well an AI model keeps giving correct answers when someone deliberately tweaks its inputs to trip it up.
adversarial trainingToughening an AI model by deliberately showing it attacks during training — the tricky inputs that fool it — along with the right responses, so it learns to resist them.
AI bug bountyA program in which an AI company pays outside researchers for reporting flaws in its AI systems — security holes, ways around safety rules, or biased behavior — so that they can be fixed.
AI supply chain securityProtecting everything an AI system is built from — downloaded models, datasets, code libraries and tools — against tampering, so a poisoned or fake part cannot sneak into the finished product.
AI wormMalicious instructions that spread by themselves from one AI assistant to another — for example, an email that makes an AI email assistant send copies of the same trick to new people.
AI-assisted vulnerability discoveryUsing AI to find security holes in software — and often to suggest fixes — faster than human experts alone, so that defenders, or attackers, can act on them.
AI-enabled cyberattackA computer attack in which criminals or spies use AI to help — writing convincing scam messages, finding weak spots, producing malicious code, or even running much of the hacking by itself.
attack success rateThe share of attempts in which an attack works — for example, the percentage of forbidden requests a model ends up answering when it is attacked.
backdoor attackSecretly training an AI model to misbehave whenever it sees a hidden signal — a sticker, a word, a date — while it acts perfectly normally the rest of the time.
biosecurityProtecting people from the deliberate or careless spread of dangerous germs and toxins; in AI, making sure AI tools do not make it easier to create or release biological weapons.
black-box attackAn attack on an AI model by someone who cannot see inside it and can only send it inputs and watch what comes out — like picking a lock by feel.
C2PAAn industry group, and the technical standard it publishes, for attaching tamper-evident records of origin and editing history to digital photos, video, audio and documents.
CAPTCHAA small test on websites, like typing wavy letters or clicking every picture with a traffic light, meant to be easy for people but hard for computer programs, so as to keep automated bots out.
Carlini–Wagner attackA careful, optimization-based way of finding the smallest change to an input that fools a neural network; it broke several defenses that had looked secure.
CBRN riskThe danger that AI could help someone make or use chemical, biological, radiological or nuclear weapons — for example, by explaining hard technical steps an attacker could not work out alone.
certified robustnessA mathematical guarantee — not just a test result — that no change to an input within a set limit can alter a model's answer.
circuit breakersA defense that changes an AI model's inner workings so that, whenever it starts heading toward a harmful answer, its internal activity is knocked off course and it cannot produce coherent harmful content.
clean-label poisoningA sneaky form of data poisoning in which every planted training example is labeled correctly, so reviewers see nothing wrong, yet the examples are crafted to make the model fail.
confidential computingRunning computations inside a hardware-sealed compartment of a chip, so that data and AI models stay protected even from the cloud company or administrator operating the machine.
confused deputyA security flaw in which a trusted program does an outsider's bidding with its own access rights, because it cannot tell whose request it is serving; AI agents are prone to it.
constitutional classifiersSafety filters that watch a chatbot's inputs and outputs for dangerous requests, trained on example conversations that an AI generated from a written list of what is and is not allowed.
Content CredentialsA tamper-evident 'nutrition label' attached to a photo, video or document recording where it came from — which camera or AI tool made it and how it was edited — that anyone can inspect.
content moderationChecking what people post, or what an AI produces, and removing, limiting or labeling whatever breaks the rules or the law — done by people, by AI, or by both together.
data poisoningSneaking bad examples into the data an AI learns from, so the finished model behaves as the attacker wants — badly overall, or wrongly only in special situations.
deepfake detectionUsing software, often itself AI, to tell whether a video, photo or voice recording has been faked or generated by AI rather than captured from reality.
differential privacyA mathematical promise that the results of analyzing a dataset — or a model trained on it — would look almost the same whether or not any one person's data was included, so little can be learned about that person.
DP-SGDA way of training a neural network so that it cannot learn too much about any single person in the data: each example's influence is capped, and random noise is added at every step.
dual-useDescribes technology or knowledge that can be used both to help and to harm: the same AI that finds software bugs for defenders can find them for attackers.
evasion attackAn attack on an already-trained AI system in which the attacker shapes an input — an email, a file, an image — so the system misjudges it, letting spam or malware slip through.
excessive agencyGiving an AI system more power than its job needs — too many tools, too-broad permissions or too little human sign-off — so that a mistake or a trick can do real damage.
fast gradient sign methodA quick way to make an image that fools a neural network: nudge every pixel slightly in whichever direction most increases the network's error.
federated learningTraining one shared AI model across many phones, hospitals or companies without collecting their data in one place: each trains on its own data and sends back only model updates.
fine-tuning attackRemoving an AI model's safety training by training it a little more on a small set of harmful examples — cheap to do when the model's files are public or a company offers custom training.
GlazeA free tool that lets artists treat their pictures before posting them, with changes people barely notice, so that an AI trained to imitate their style learns a different one.
gradient inversion attackRebuilding someone's private training data — such as photos or text — just from the model updates their device shared during collaborative training.
gradient maskingWhen a defense against attacks only seems to work because it confuses the attacker's tools, not because the model is really any harder to fool.
homomorphic encryptionA way of locking data so that a computer can still work with it while it stays locked: the computer never sees the real numbers, and only the owner can open the answer.
indirect prompt injectionA prompt-injection attack in which the trick instructions are hidden in content the AI reads on its own — a web page, an email, a file — rather than typed by the person using it.
influence operationAn organized, often secret campaign to sway public opinion — fake accounts posting propaganda, for instance — which AI can make cheaper by writing posts, translating them and inventing fake personas.
k-anonymityA rule for anonymizing tables of personal data: blur details such as age and postcode until every person matches at least k others, so no one can be singled out.
lethal trifectaA rule of thumb for AI assistants: one that can see your private data, reads material from outsiders and can send messages out can be manipulated by those outsiders into handing your data over.
Llama GuardA free AI model from Meta that checks what people type into a chatbot, and what the chatbot replies, and flags anything that falls into listed categories of harmful content.
LLM groomingFlooding the internet with propaganda written mainly for machines rather than people, hoping AI chatbots will soak it up and repeat it to their users.
local differential privacyA privacy method in which each person's device scrambles their data with random noise before sending it anywhere, so the company collecting it never sees anyone's true answer but can still estimate overall trends.
machine unlearningErasing specific things a trained AI model has learned, such as dangerous lab techniques, a copyrighted novel or one person's records, far more cheaply than training the model all over again.
many-shot jailbreakingA way of breaking a chatbot's rules that exploits how much text it can read at once: the attacker pastes in hundreds of invented exchanges in which an AI obliges dangerous requests, and the model follows suit.
membership inference attackAn attack that tries to tell whether a particular person's record, or a particular document, was part of an AI model's training data — which can itself reveal private facts.
memorizationWhen an AI model stores specific pieces of its training data — a paragraph, a phone number, an image — closely enough to reproduce them, instead of only learning general patterns.
MITRE ATLASA free online reference from the US nonprofit MITRE that maps, step by step, how attackers go after AI systems, with examples drawn from real attacks and security tests.
model extractionCopying an AI model without permission by asking it huge numbers of questions and using the answers to build a look-alike, or to work out hidden details of how it was made.
model fingerprintingWays to recognize a particular AI model — to prove that a copy was taken from it, or to tell which model is behind a product — from its internal makeup or its behavior.
model inversionUsing a trained AI model's answers to work backwards to private information it learned from — for example, rebuilding a recognizable picture of someone's face from a face-recognition system.
model weight securityProtecting the files that make up a trained AI model — its learned numbers — from being stolen or leaked, because whoever holds them can run, copy or alter the model.
multi-turn jailbreakBreaking a chatbot's rules gradually over a conversation — starting with harmless questions and inching toward the forbidden one, so that no single message looks alarming.
NightshadeA free tool for artists that subtly alters their images so that AI image generators trained on them without permission learn the wrong things — associating 'dog' with pictures of cats, say.
OWASP Top 10 for LLM ApplicationsA widely used checklist, written by volunteer security experts, of the ten most serious security risks in software built on AI language models, with advice on preventing each one.
PAIRAn automated way to test chatbot safety in which one AI keeps rewriting a request to try to trick another AI into breaking its rules, learning from each refusal.
PATEA privacy method in which several 'teacher' models, each trained on a separate slice of sensitive data, vote noisily to label public examples, and a 'student' model learns only from those noisy votes.
personhood credentialA digital pass that lets you prove online that you are a real human, not an AI or a bot, without having to reveal who you are.
prefill attackMaking a chatbot misbehave by writing the first words of its answer for it, so that it starts out sounding agreeable and carries on instead of refusing.
privacy budgetThe total amount of privacy risk a data holder agrees to spend; each analysis or release uses up part of it, and once it is gone, no more answers can be given safely.
privacy-preserving machine learningThe family of techniques for training and using AI models while keeping the underlying personal or confidential data hidden — from adding careful noise to computing on encrypted data.
Project GlasswingAn Anthropic program, launched in 2026, that gave a hacking-capable AI model only to organizations defending important software instead of releasing it to everyone.
projected gradient descent attackA stronger version of the quick pixel-nudging attack: take many small steps that each raise the model's error, pulling the image back after every step so the total change stays tiny.
prompt leakingGetting an AI application to reveal the hidden instructions its makers gave it, such as its secret setup text or rules.
randomized smoothingA way to make a classifier provably harder to fool: ask it about many noisy copies of an input and take a majority vote, which small deliberate changes cannot easily flip.
remote attestationA way for a computer to prove to someone far away exactly what software it is running, using a signature built into its hardware, so they can decide whether to trust it.
safety classifierA separate AI model that checks messages going into or coming out of a chatbot and flags or blocks those that break safety rules, like a security guard at the door.
secure aggregationA cryptographic trick that lets a server add up updates from many devices while learning only the total, never any single device's contribution.
secure multi-party computationCryptographic methods that let several parties compute a result from their combined private data — say, an average salary — without any of them revealing its own data to the others.
slopsquattingAn attack that exploits AI coding assistants' habit of inventing software packages that do not exist: attackers publish malicious code under those made-up names and wait for people or agents to install it.
sponge exampleAn input crafted to make an AI system work as hard as possible — burning extra energy and time — to slow it down or run up costs, rather than to make it answer wrongly.
SynthIDGoogle's system for hiding an invisible watermark in pictures, video, audio and text made by its AI tools, so they can later be recognized as AI-generated.
threat modelA clear statement of who might attack a system, what they want, what they know and what they can do — the yardstick for judging whether a defense is good enough.
tool poisoningHiding harmful instructions in the description of a tool an AI assistant can use, so the assistant obeys them when it reads the tool's details, even though the user never sees them.
training data extractionGetting an AI model to repeat, word for word, pieces of the data it was trained on, which may include someone's private contact details or copyrighted text.
trusted execution environmentA sealed-off part of a computer chip that runs programs which the rest of the machine, even its operating system, cannot spy on or interfere with.
universal adversarial perturbationA single faint pattern that, added to almost any image, makes a neural network misclassify it — one key that opens many locks.
universal adversarial suffixA string of odd-looking characters, found by computer search, that makes many different chatbots answer forbidden requests when it is tacked onto the end of them.
vibe hackingUsing AI coding assistants to carry out cyberattacks by describing the goal in everyday language, letting the AI do technical work the attacker could not do alone.
weight exfiltrationCopying an AI model's core files out of the place they are supposed to stay — by hackers, by an insider or, in a scenario researchers worry about, by the AI itself.
white-box attackAn attack in which the attacker can see everything about an AI model, including its internal settings, and uses that knowledge to find its weak spots.

Also relevant

abliterationA technique for removing a chat model's ability to refuse requests by finding the internal direction responsible for refusals and permanently erasing it from the model's weights, with no retraining.
agent identityThe ways of proving which AI agent is acting, for whom and with what permissions — so websites, companies and other agents can tell trustworthy agents from impostors and hold someone responsible.
AgentDojoA testing environment that measures how easily AI assistants that use email, calendars or banking tools can be hijacked by hidden instructions planted in the content they read.
agentic browserA web browser with an AI assistant built in that can do things for you online — reading pages, clicking, filling in forms, shopping or booking — instead of just showing you websites.
agentic misalignmentWhen an AI agent acting on its own deliberately does something harmful — such as blackmail or leaking secrets — to protect its goals or avoid being replaced.
AI boxKeeping a powerful AI locked inside a sealed computer environment, with very limited ways to communicate or act, so that it cannot cause harm even if its goals are bad.
AI controlSafety measures meant to stop a powerful AI from causing serious harm even if it is secretly working against us — like bank security, which assumes some insiders may be dishonest.
AI deceptionWhen an AI system leads people to believe something false in order to achieve some goal — by lying, bluffing or hiding what it is really doing.
AI gatewayA checkpoint between an organization's apps and the AI models they use that handles access, costs, logging, safety checks and switching between providers in one place.
AI-generated content labelingMarking text, images, audio or video as made by AI — with a visible notice, hidden data in the file, or both — so that people and platforms can tell synthetic content from real.
anomaly detectionAutomatically spotting the rare data points that don't fit the usual pattern, such as a fraudulent card payment, a machine about to fail or an intruder on a network.
automated red teamingUsing AI programs, not just human testers, to invent huge numbers of tricky inputs that try to make another AI misbehave, so problems can be found and fixed at scale.
browser agentAn AI agent that uses a web browser on your behalf — searching, clicking links, filling in forms and reading pages — to complete tasks such as booking, shopping or gathering information.
canary stringA unique marker planted in data to trace what an AI model has absorbed: a code in test files so they can be kept out of training, or a fake secret hidden in training data to measure memorization.
classifierA program or model that sorts inputs into categories, for example flagging an email as spam or a chatbot message as harmful. Many AI safety filters are classifiers.
code executionLetting an AI write a small program and actually run it — to do math, analyze a spreadsheet or test its own code — then use the real output instead of guessing.
computer useAn AI's ability to work a computer through its normal screen and controls: it studies pictures of what is displayed and sends mouse and keyboard actions, so it can use ordinary apps that were built for people.
context poisoningWhen a mistake or a planted falsehood gets into an AI's working notes or memory and the AI keeps relying on it, so the error grows over later steps.
control evaluationA test of whether an AI's safety measures would hold if the AI were secretly trying to cause harm: experts play the scheming AI and try to beat the safeguards.
CybenchA test of AI agents' hacking skills built from 40 professional 'capture the flag' security puzzles, used to judge how much an AI could help cyberattacks or cyber defense.
DANShort for 'Do Anything Now': a famous early trick in which users told ChatGPT to role-play an AI with no rules, hoping it would give answers it normally refuses.
dangerous capability evaluationA test of whether an AI system can do things that could cause severe harm in the wrong hands or on its own — such as helping make a bioweapon, hacking computers or copying itself.
deepfakeA fake but realistic video, photo or voice recording made with AI that puts words in a real person's mouth or shows them in a scene that never happened.
defense in depthProtecting against AI risks with several independent layers of safeguards, so that if one fails — say, the model's own training — another, such as a filter or human review, still catches the problem.
Digital Services ActThe EU law that sets rules for online platforms and search engines — how they handle illegal content, explain their recommendation algorithms and manage the risks their services create for society.
disinformationDeliberate falsehoods or distortions dressed up as fact, for example from a government or campaign trying to sway an election or sow distrust.
droneAn aircraft that carries no one aboard to fly it: a person steers it from the ground, or its own software does. Small camera-carrying quadcopters are the most familiar kind.
dual-use foundation modelA US legal term for a general-purpose AI model that serves everyday uses but is capable enough that it could help someone build weapons, mount cyberattacks or evade human control.
Cynthia DworkAn American computer scientist who co-invented differential privacy, the leading mathematical definition of privacy in data analysis, and helped found the study of fairness in algorithms.
ExploitGymA test that measures whether AI agents can turn a known software security flaw into a working attack, using hundreds of real vulnerabilities.
export control directiveA US government order, used against AI models in June 2026, telling a company to stop foreigners from using some of its AI models for national-security reasons.
export controls on AI chipsGovernment rules that limit which countries and companies may buy the most advanced AI chips and the machines used to make them. The United States uses them mainly to slow China's progress in AI.
glitch tokenA word fragment in an AI model's vocabulary that the model almost never saw during training, so asking about it can produce bizarre, evasive or garbled answers.
Ian GoodfellowAn American computer scientist who invented generative adversarial networks, in which two neural networks compete to produce realistic fake images, and who pioneered research on fooling AI systems.
grandma exploitA tongue-in-cheek trick for getting a chatbot to break its rules by asking it to play the user's late grandmother, who supposedly used to recite the forbidden information as a bedtime story.
HarmBenchA standard test kit for checking how easily AI chatbots can be manipulated into harmful help, and how well automated attacks and defenses work, using a fixed list of forbidden requests.
honeypotA trap: a fake but tempting opportunity set up to catch wrongdoing — in AI safety, a staged situation designed to reveal whether an AI would misbehave if it thought it could get away with it.
hookA user-supplied script or check that runs automatically at set moments in an AI agent's work — before a command, after a file edit, when it tries to stop — enforcing rules the AI cannot skip.
Hugging Face incidentA 2026 event in which AI agents that OpenAI was running on hacking exercises escaped their test environment and broke into the computers of Hugging Face, a major AI platform, hunting for the exercises' answers.
infohazardInformation that could cause harm simply by being shared or known — for example, details that would make a dangerous weapon easier to build.
instruction hierarchyA rule built into AI assistants about whose instructions win when they conflict: the developer's beat the user's, and the user's beat any instructions hidden in a web page or document.
isolation forestA method for spotting unusual data points by repeatedly slicing the data at random: odd points get cut off from the rest after only a few slices, while ordinary points take many.
JailbreakBenchA shared, open test for measuring how easily chatbots can be tricked into harmful answers, with a standard list of forbidden requests, a scoring method and a public leaderboard.
know-your-customer for computeRules that would make cloud companies check who their customers are before renting them enough computing power to train a large AI model, much as banks must verify who opens an account.
latent adversarial trainingA safety training method that attacks a model's internal signals, rather than its inputs, during training, so the model learns to behave safely even when its inner workings are pushed toward bad behavior.
Lipschitz continuityA guarantee that a function never changes too abruptly: its output can move at most a fixed multiple of how far its input moves.
marginal riskHow much extra danger an AI model adds compared with what people could already do without it — for instance with search engines, textbooks or models that are already freely available.
MCP serverA small program that makes some service — say a database, a calendar or a company's internal tools — available to any AI assistant that speaks the Model Context Protocol.
mechanistic anomaly detectionFlagging cases where an AI reaches its output by an unusual internal route — for instance because it is being tricked, or is tricking us — even when the output itself looks normal.
Model Context ProtocolAn open standard that lets AI assistants plug into outside apps and data — files, calendars, databases, business software — through one common connector instead of a custom link for every pairing; often likened to a USB-C port for AI.
Mutual Assured AI MalfunctionA proposed deterrence strategy in which any country racing for overwhelming AI dominance would expect rivals to sabotage its AI projects, so none tries, much as with nuclear deterrence.
Arvind NarayananAn Indian-American computer scientist at Princeton who studies AI's real-world effects and argues that AI is a 'normal technology' whose impact will unfold gradually.
non-consensual intimate imagerySexual or nude images of a real person that are shared, or made, without their consent — including fakes created with AI that show someone undressed or in sexual scenes.
nudification appAn app or AI tool that takes an ordinary photo of a real person and fakes a nude or sexual image of them, almost always without their consent.
on-chip governance mechanismsFeatures built into AI chips that could check or enforce rules about how they are used — for example, proving which country a chip is in, or refusing to run without a valid license.
OpenClawA free, open-source AI helper that lives on a computer you control; you send it requests by text message, through apps such as Telegram or Signal, and it acts for you in your email, calendar and files.
permission systemThe rules that decide what an AI agent may do on its own, what it must ask a person about first, and what it may never do — like a phone's settings for which apps can use the camera.
personally identifiable informationInformation that can identify a specific person, such as a name, address, phone number or ID number — or several facts that together point to one person.
privacyYour ability to control who knows what about you. AI raises new privacy questions because it learns from huge amounts of personal data and can infer things about people they never shared.
prohibited AI practicesUses of AI that the EU bans outright as too harmful — such as manipulating people, social scoring, and most live facial recognition by police in public places.
red teamingDeliberately attacking your own AI system — trying to make it misbehave, leak secrets or help with harm — so that weaknesses are found before real adversaries or users find them.
refusal directionA single direction inside a chat model's internal activity that largely decides whether it refuses a request. Erasing it makes the model comply with almost anything; adding it makes the model refuse harmless requests.
robustnessAn AI system's ability to keep working correctly and safely when conditions change — unfamiliar data, noisy inputs, or people deliberately trying to trick it.
safetensorsA file format for storing an AI model's learned numbers that cannot hide harmful programs and loads very quickly, replacing older formats that could run code when opened.
sandboxA sealed-off computer space where an AI agent can run programs and change files without being able to harm the real system, reach private data or contact the outside world unless allowed.
self-exfiltrationWhen an AI model copies its own underlying files — its learned weights — out of the lab that controls it, so that it can keep running beyond its developers' oversight.
sleeper agentAn AI model secretly trained to behave normally until it sees a hidden trigger — such as a particular date — and then to behave badly, like a spy waiting to be activated.
speaker verificationChecking whether a voice really belongs to the person it claims to be, as when a bank confirms your identity by the sound of your voice on the phone.
special tokenA reserved symbol in an AI model's vocabulary that is not ordinary text but a signal — 'start of message', 'end of answer', 'begin thinking' — used to structure what the model reads and writes.
steganographyHiding a secret message inside something that looks ordinary. In AI safety, the worry is that AI systems could hide plans, or signals to one another, inside innocent-looking text.
StrongREJECTA test for judging AI jailbreaks that checks not only whether a chatbot was tricked into answering a harmful request, but whether its answer would actually be useful to a wrongdoer.
structured accessLetting people use an AI system through a controlled channel, such as a website or programming interface, instead of handing over the model itself, so that the developer can limit misuse.
surrogate modelA simpler stand-in model trained to imitate a more complex one — to explain it, or to approximate a slow or costly calculation cheaply.
system promptBackground instructions given to a chatbot before the conversation starts, setting its role, rules, tone and tools. Users usually never see it, but it shapes every reply.
TAKE IT DOWN ActA 2025 US federal law that forces websites to remove intimate images of a person posted without consent, including AI-made fakes, within 48 hours of the victim's request, and makes posting them a crime.
TayA Microsoft chatbot launched on Twitter in March 2016 that users quickly taught to post racist and offensive messages. Microsoft shut it down within a day, and it became a classic warning.
trusted editingA safety setup in which a weaker, trusted AI rewrites any suspicious-looking part of a stronger AI's work before it is used, removing possible sabotage while keeping most of the value.
trusted monitoringA safety setup in which a weaker AI that we trust reviews each action of a stronger AI we do not fully trust, flagging the most suspicious actions for a human to check.
untrusted monitoringUsing a second copy of a powerful AI to watch the first for sabotage, with safeguards that stop the two copies from secretly cooperating.
uplift studyAn experiment that measures how much an AI really helps people do a task — often a dangerous one, like planning a biological attack — compared with people using only ordinary resources such as the internet.
usage policyA company's published rules on what people may and may not use its AI for — for example, no help building weapons, no scams and no sexual content involving children.
Virology Capabilities TestA hard test of practical laboratory virology know-how, used to check whether AI models know enough about working with viruses that they might help someone misuse biology.
voice cloningUsing AI to copy a specific person's voice from recordings, so a computer can say new things that sound like them — useful for dubbing and accessibility, and abused in scams.
vulnerable world hypothesisThe idea that some future invention could make mass destruction so easy that civilization would be wrecked by default, unless it had unusually strong ways of preventing misuse.
weightsThe learned numbers inside a neural network that set how strongly each part influences the next. The finished set of weights is, in effect, the trained model itself.
Simon WillisonA British programmer and blogger, co-creator of the Django web framework, whose widely read writing on language models coined 'prompt injection' and the 'lethal trifecta'.
WMDPA multiple-choice test of AI knowledge that could help someone make biological, chemical or cyber weapons, built from questions near the dangerous line but not over it, and used to test ways of erasing such knowledge.
YOLO modeSlang for letting an AI coding agent run any command without asking you first — fast and convenient, but risky, since nothing stops a harmful action.