AI Engineering & MLOps

Building AI products: retrieval, APIs, deployment, monitoring and tooling. 109 entries live here, and 231 from other domains touch it.

Essential

GitHub CopilotAn AI assistant from GitHub, built into programmers' code editors, that suggests code as they type, answers questions about code and can carry out coding tasks on its own.
Hugging FaceA company whose website is the main public library for sharing AI models and datasets — often called the GitHub of machine learning — and which makes widely used free AI software.
model APIA web service through which programs send requests to an AI model run by a company and receive its answers, usually paying per use instead of running the model themselves.
open-source AIAI released, ideally with its code and details of its training data and not just the finished model, so that anyone may use, inspect, alter, share and even rebuild it. How open is open enough is debated.
open-weight modelAn AI model whose trained internal numbers are published for anyone to download and run on their own computers, even if the data and code used to make it stay private.
proprietary modelAn AI model kept private by the company that made it; outsiders can use it only through the company's apps or paid online services and can never download it.

All other entries

103

A/B testingComparing two versions of something — two models, two prompts, two designs — by randomly showing each to different groups of real users and measuring which works better.
AI coding assistantAI software that helps people write computer programs — suggesting the next lines, explaining code, finding bugs or making changes on request — inside the tools programmers already use.
AI engineerA software developer who builds products using ready-made AI models — connecting them to data and tools and testing that they work well — rather than inventing or training the models.
AI engineeringThe craft of building useful, reliable products on top of AI models — choosing models, writing prompts, connecting data and tools, testing quality and running it all safely at scale.
AI gatewayA checkpoint between an organization's apps and the AI models they use that handles access, costs, logging, safety checks and switching between providers in one place.
AI stackThe layers of technology an AI product depends on, from chips and data centers at the bottom, through the models, up to the apps people use, each built on the layer below.
approximate nearest neighbor searchA family of shortcuts for finding the most similar items in a huge collection without checking every one — accepting an occasional miss in exchange for answers thousands of times faster.
AutoGenA free Microsoft software framework for building teams of AI agents that solve problems by messaging one another, writing and running code, and asking people for help when needed.
bi-encoderAn AI design that turns a question and a document into number lists separately, so documents can be processed once in advance and compared with any new question in a flash.
BM25A decades-old formula that search engines use to score how well a document matches a query, rewarding rare query words and repeated matches — with diminishing returns — and correcting for long documents.
CaffeAn early free deep-learning toolkit from the University of California, Berkeley, popular for image recognition in the mid-2010s, in which networks were described in simple text files rather than code.
canary deploymentReleasing a new model or software version to a small slice of users first and watching closely, so that any problem hits only a few people before the change reaches everyone.
chunkingCutting long documents into smaller pieces before storing them for AI search, so the system can find and hand the model just the relevant part instead of a whole manual.
ColBERTA search model that compares a question and a document word by word — matching each question word to its best partner in the document — while still letting documents be processed ahead of time for speed.
compound AI systemAn AI product built from several working parts — models, search tools, code and checks — designed to work together, rather than a single model answering on its own.
concept driftWhen the real-world link between what a model sees and the right answer changes over time — as when spammers change tactics — so lessons the model learned no longer hold.
contextual retrievalA fix for searching cut-up documents: before storing each piece, an AI writes a short note saying where it came from and what it is about, so the piece still makes sense on its own.
Core MLApple's software framework that lets apps on iPhones, iPads and Macs run AI models directly on the device, using its own chips instead of sending data to the cloud.
CrewAIA free software framework for setting up teams of AI agents, each given a job title, goal and backstory, that divide up tasks and work through them together like a small staff.
cross-encoderAn AI model that reads a question and a candidate passage together, side by side, and outputs one relevance score. Accurate but slow, so it is used to reorder a short list.
CursorA code editor built around AI, in which programmers can ask a model to write, change or explain code across a whole project, or hand it tasks to finish on its own.
dark factoryA way of building software in which AI agents write, test and ship all the code and no human writes or reads it, like a factory run with the lights off.
data driftWhen the information flowing into an AI model in real use gradually stops resembling the data it was trained on — new customers, new slang, new cameras — making its predictions less reliable.
data versioningKeeping a record of every version of the data used to train and test AI models, so that any past result can be traced back to exactly the data that produced it.
dense retrievalFinding relevant documents by turning the question and every document into lists of numbers that capture meaning, then picking the documents whose numbers are closest to the question's.
DSPyA free software framework in which programmers describe what each step of an AI pipeline should do, and the framework automatically works out good prompts and examples instead of hand-writing them.
embedding modelAn AI model whose only job is to turn text (or images, or code) into lists of numbers that capture meaning, so a computer can measure how similar two things are.
evals-driven developmentBuilding an AI product by first writing tests that show what good answers look like, then accepting changes to prompts, models or code only when the test scores improve.
ExecuTorchMeta's free software for running PyTorch AI models directly on phones, wearables and other small devices, without needing an internet connection or a data center.
experiment trackingKeeping an automatic logbook of every model-training attempt — settings, data, code and results — so a team can compare attempts, repeat the good ones and see what changed.
FAISSA free software library from Meta for searching huge collections of embeddings very fast, on ordinary processors or graphics cards. Many AI search systems use it under the hood.
feature storeA shared library of ready-made input signals for models — such as a customer's spending last week — computed one consistent way and supplied both when models are trained and when they run live.
forward deployed engineerAn engineer from an AI or software company who works directly inside a customer's organization, building and tailoring the company's systems to that customer's real problems.
GradioA free Python library for quickly wrapping an AI model in a simple web page — text boxes, image uploads, chat windows — so that others can try it in a browser.
GraphRAGA way of letting AI answer questions about a big pile of documents by first turning them into a map of people, things and their connections, then using that map to find and summarize answers.
HNSWA popular way to organize millions of embeddings for fast similarity search: a layered web of links where the search starts with long jumps on sparse top layers, then takes ever shorter steps.
hybrid searchSearching two ways at once — by exact keywords and by meaning — and merging the results, so you catch precise matches like product codes as well as differently worded matches.
HyDEA search trick: ask an AI to write a made-up answer to the question first, then search for real documents that resemble that imagined answer.
information retrievalThe study and engineering of search: how to organize huge piles of documents so that the few that matter to someone's question can be found and ranked quickly. AI systems rely on it to look things up.
inverted indexA lookup table that lists, for every word, all the documents containing it — like the index at the back of a book — so a search engine can find matches almost instantly.
JAXA free Google library that lets researchers write mathematics in familiar Python and automatically get derivatives and very fast code for graphics chips and Google's AI chips.
JSON modeA setting on some AI services that makes the model reply only in JSON, a standard text format for data, so that programs can read the reply reliably.
JSON SchemaA standard way of writing down exactly what a piece of JSON data must look like — which fields, of which types — so that software and AI models can check it or follow it.
Jupyter notebookAn interactive document, opened in a web browser, that mixes runnable code with its results, charts and notes, so people can experiment step by step and share their work.
KerasA free, beginner-friendly Python library for building neural networks from ready-made layers in a few lines of code, while a larger toolkit underneath does the heavy computing.
LangChainA popular free software toolkit, and the company behind it, for building applications that connect AI language models to documents, databases, tools and multi-step workflows.
LangGraphA free software library for building AI agents as flowcharts of steps that share a memory, so developers control exactly how an agent moves between steps, pauses and resumes.
late interactionA search design that keeps a separate number-list for every word of a document and compares them with the question's words at the last moment — more precise than one list per document, yet still fast.
LiteRTGoogle's free software for running AI models on phones, small gadgets and microcontrollers; it was called TensorFlow Lite until 2024 and is used inside many Android and iPhone apps.
LlamaIndexA free software toolkit, and the company behind it, for connecting AI language models to an organization's own documents and data so the models can find and use that information.
LLM observabilityRecording and inspecting what an AI application actually did on each request — prompts sent, documents found, tools used, answers given, time and cost — so problems can be found and fixed.
LLMOpsThe everyday work of running chatbots and other products built on large language models: tracking prompts, testing answers, watching cost and quality, and updating safely when models change.
machine learning engineerAn engineer who turns machine-learning ideas into working systems: preparing data, training and testing models, and running them reliably inside real products.
machine learning pipelineAn automated chain of steps that takes raw data, prepares it, trains and tests a model and hands it over for use, so the whole process can be rerun the same way.
machine learning technical debtThe hidden future cost of quick shortcuts in machine-learning systems — tangled data dependencies, messy glue code, unchecked feedback loops — that later make them hard and risky to change.
Matryoshka embeddingAn embedding built like a set of Russian nesting dolls: its first few numbers already give a rough summary, so you can cut it short to save space and still search reasonably well.
mean reciprocal rankA score for search: for each question, note how far down the first correct answer appears, take one over that position, and average across questions. First place scores 1, second place 0.5.
MLflowA free software platform that records machine-learning experiments, packages the resulting models and keeps track of which versions are ready to be used.
MLOpsThe working habits and tools that keep machine-learning models dependable inside real products once the experiment is over: automated testing, release, monitoring and retraining; the factory and maintenance crew around a model, not the model itself.
MLXA free Apple software toolkit for running and training AI models efficiently on Mac computers with Apple's own chips, popular for running language models locally on a laptop.
model deploymentPutting a trained AI model to work in a real product so that people or other software can use it — and doing so carefully enough that the change can be undone if something goes wrong.
model deprecationWhen an AI company announces that an older model will stop being offered and then switches it off, so the apps and people relying on it must move to a newer one.
model hubAn online library where trained AI models are published so that others can find, download and reuse them instead of training their own from scratch.
model licenseThe legal terms under which an AI model may be downloaded and used — for example, whether businesses may use it, whether changes may be shared, and which uses are banned.
model monitoringContinuously checking an AI model after it goes live — what data it receives, what it predicts and whether it is still right — so that problems are caught before they cause harm.
model registryA central catalog where a team stores every version of its trained models, with records of how each was made and which version is approved for use.
normalized discounted cumulative gainA score for a ranked list of results that rewards putting the most relevant items near the top, scaled so that a perfect ordering scores 1.
NumPyThe basic free Python library for working quickly with large grids of numbers; nearly every data-science and machine-learning tool in Python is built on it or copies its style.
ONNXA shared file format for trained AI models, so that a model built with one toolkit can be moved to and run by another, much as a PDF opens on any computer.
OpenAI-compatible APIAn AI service that accepts the same request format as OpenAI's, so software written for OpenAI's models can use other companies' or self-hosted models by changing an address and key.
pandasA free Python library for loading, cleaning, combining and summarizing tables of data, like a programmable spreadsheet; it is a standard first tool for preparing data for machine learning.
product quantizationA compression trick for embeddings: chop each long list of numbers into short pieces and replace each piece with the ID of its closest typical piece, shrinking storage many times over while keeping search workable.
prompt managementTreating the written instructions given to AI models like valuable code: storing them in one place, tracking every change, testing new versions and rolling back bad ones.
prompt optimizationUsing software, often another AI model, to search automatically for the wording and examples that make an AI model perform best on a task, instead of tweaking prompts by hand.
PyTorchThe most widely used free software toolkit for building and training neural networks, especially in research; programmers write ordinary Python and it handles the calculus and the graphics chips.
query rewritingRephrasing a user's question before searching — fixing typos, adding missing details, splitting it into simpler questions — so the search engine finds better results.
rate limitA cap on how much a customer may use an online AI service in a given time — so many requests or words per minute — beyond which extra requests are refused until later.
RayFree software that spreads Python programs, including AI training and model serving, across many computers and graphics chips so that big jobs run in parallel.
recall@kA score for search systems: of all the right answers that exist, what fraction showed up in the top k results? Higher means fewer relevant items were missed.
reciprocal rank fusionA simple recipe for merging several ranked lists of search results: each result earns points according to how high it ranks in each list, and the totals decide the final order.
rerankingA second, more careful pass over search results: a fast method grabs a few hundred candidates, then a slower, smarter model reorders them so the best come first.
robotic process automationSoftware 'robots' that copy the clicks and typing a person would do in office computer programs — such as moving data between forms — to automate repetitive paperwork.
safetensorsA file format for storing an AI model's learned numbers that cannot hide harmful programs and loads very quickly, replacing older formats that could run code when opened.
scikit-learnA free, widely taught Python library of classic machine-learning methods — such as decision trees, regression and clustering — that all work in the same simple way.
semantic cachingSaving an AI system's past answers and reusing one whenever a new question means the same as an old one, even if worded differently, to save time and money.
semantic searchSearch that understands what you mean, not just the words you typed, so a search for 'cheap places to stay' also finds 'budget hotels'.
service-level objectiveA specific, measurable target for how well a service should perform — for example, 99.9 percent of requests answered within two seconds — used to decide when engineers must act.
shadow deploymentQuietly running a new AI model alongside the old one on real requests, recording its answers without showing them to anyone, to check it behaves well before it takes over.
sparse retrievalFinding documents by the words they contain, giving more weight to rare, telling words — the classic way search engines work.
spec-driven developmentBuilding software with AI coding agents by first writing a detailed description of what it should do, which the AI then follows to plan and write the code.
structured outputsGetting an AI model to answer in an exact, machine-readable format — like a form with fixed fields — so other software can use the answer without guesswork.
TensorFlowGoogle's free software toolkit for building, training and running machine-learning models; it led the field after 2015 and still runs many models inside phones, browsers and company systems.
text embeddingA list of numbers that captures what a piece of text means, so that texts with similar meanings get similar lists. Computers use them to search, sort and group text by meaning.
TF-IDFA simple score for picking out the words that characterize a document: a word counts for more the more often it appears there, and for less the more of the collection's other documents also use it.
TheanoOne of the first popular free software tools for deep learning, built at a Montreal university lab; development ended in 2017, but it shaped the toolkits that replaced it.
training–serving skewA mismatch between how data is prepared when a model is trained and when it is used for real, which makes the live model quietly worse than its tests promised.
TransformersA free software library from Hugging Face that lets programmers download and use thousands of ready-trained AI models for text, images and sound with a few lines of code.
vector databaseA database built to store embeddings — lists of numbers that represent meaning — and to quickly find the stored items most similar to a new one. It is the memory behind many AI search tools.
vector searchSearching by closeness instead of exact match: the query and all stored items are lists of numbers, and the search returns the items whose numbers are most similar to the query's.
vector space modelAn early, still-influential idea for search: treat every document and query as an arrow in a space with one direction per word, and rank documents by how closely their arrows point the same way as the query's.
vibe engineeringBuilding serious software with AI coding assistants while still taking full responsibility for its quality — the careful, professional opposite of vibe coding.
vibe portingUsing AI coding agents to translate a whole software project into another programming language, usually checking the result against the original project's tests.
Weights & BiasesA company and online service that machine-learning teams use to log, chart and compare their training experiments, models and AI-app test results in shared dashboards.

Also relevant

231

abliterationA technique for removing a chat model's ability to refuse requests by finding the internal direction responsible for refusals and permanently erasing it from the model's weights, with no retraining.
agent frameworkA software library that gives developers ready-made parts for building AI agents — model connections, tool handling, memory, multi-agent coordination and logging — so they need not write everything from scratch.
agent instruction fileA plain-text file kept in a software project that tells AI coding agents how the project works — how to build and test it, which conventions to follow, what to avoid — read at the start of every session.
agent skillA packaged how-to guide for an AI agent — written steps, sometimes with small programs and examples attached — that sits unread until a job of that kind comes up, when the agent opens it.
Agent2Agent protocolAn open standard that lets AI agents built by different companies find each other, say what they can do and work together on tasks, much as email lets different mail systems exchange messages.
agent–computer interfaceThe commands, tools and feedback designed for an AI agent to work with a computer — the agent's version of a user interface, shaped around how language models read and act.
Agentic AI FoundationA nonprofit group under the Linux Foundation that looks after shared, open technical standards for AI agents, such as the Model Context Protocol, so that no single company controls them.
agentic codingWriting software by directing AI agents that plan, edit and test code themselves, with the human acting more as a reviewer and manager than as the person typing every line.
agentic RAGRetrieval in which the AI runs its own research: it decides what to search for, looks at the results, searches again if needed, and only then answers.
agentic workflowA multi-step process in which AI models draft, check, use tools and revise their work over several rounds, instead of answering in one shot; often the steps are laid out by developers in advance.
AI applicationA program or product that uses AI to do something useful for people, such as a translation app, a medical-scan checker or a coding assistant.
AI compilerSoftware that translates an AI model into efficient instructions for a specific chip, rearranging and combining its calculations so the hardware is used as fully as possible.
AI regulatory sandboxA supervised testing ground where companies can try out new AI with regulators watching and advising, before the product faces the full weight of the rules.
AI supply chain securityProtecting everything an AI system is built from — downloaded models, datasets, code libraries and tools — against tampering, so a poisoned or fake part cannot sneak into the finished product.
AI systemA complete piece of technology that uses AI to do something, such as screening loan applications, driving a car or chatting with customers, including the AI model and the software around it.
AI-assisted vulnerability discoveryUsing AI to find security holes in software — and often to suggest fixes — faster than human experts alone, so that defenders, or attackers, can act on them.
AI-nativeDescribes a product, company or person built around AI from the start, rather than one that added AI features to older ways of working.
attention maskA grid of yes-or-no switches telling an attention layer which words each word may look at — for example, ignore blank padding, or do not peek at words that come later.
automatic differentiationA technique that lets a computer work out exactly how a program's output would change if its inputs changed slightly, by tracking every arithmetic step. It lets AI software compute training updates automatically.
AutoMLSoftware that automates the trial-and-error parts of building a machine-learning model, such as choosing the algorithm, its settings and the data-preparation steps, so less expert effort is needed.
background agentAn AI agent you hand a task to and then leave alone: it works by itself in the cloud, often for many minutes or hours, and reports back with finished work for you to review.
batch inferenceRunning an AI model over a big pile of inputs as one job rather than answering each request live, like processing a stack of forms overnight. Results come back later but cost less.
Bayesian optimizationA smart way to tune settings when each test is expensive: build a statistical guess of how settings affect results, then try whichever setting that guess says is most promising next.
BEIRA collection of 18 different search tests — on topics from medicine to finance to fact-checking — used to see whether a search model works on kinds of data it was never trained on.
CatBoostA free gradient-boosting library from the tech company Yandex, known for handling categories such as city names or product types directly, without first converting them into numbers by hand.
chat modelA language model trained to hold a back-and-forth conversation, keeping track of who said what and replying as a helpful assistant. It is the engine inside chat apps.
chat templateThe exact text layout a chat AI was trained on for marking who is speaking — system, user or assistant. Software must reproduce it precisely, or the model performs noticeably worse.
ChatGPT pluginsAdd-ons, launched in 2023 and retired in 2024, that let ChatGPT use outside services — web browsing, a code runner, travel booking or shopping sites — while answering a user.
checkpointA saved snapshot of a model partway through or at the end of training, like a save point in a video game: training can resume from it after a crash, and finished snapshots are what people download and run.
François CholletA French software engineer who created Keras, a popular tool for building neural networks, and the ARC benchmark, which tests whether AI can solve kinds of puzzles it has never seen.
chunked prefillA serving trick that splits the reading of a long prompt into smaller pieces and slots them between other users' replies, so nobody's answer freezes while a huge document is read.
Claude CodeA coding assistant from Anthropic that works as an agent: given a programming task, it explores the project, changes the code and runs programs to check its work, from the web, a desktop app, an editor or a terminal.
click-through rate predictionEstimating how likely a particular person is to click on a particular ad, search result or recommended item: a prediction made billions of times a day to decide what each user is shown.
code generationHaving an AI write computer code from a description in plain language, from finishing a single line as you type to building whole programs.
code modeA way for AI agents to use tools by writing a short program that calls them, instead of asking for each tool one step at a time.
CodexA name OpenAI has used twice: first for a 2021 AI model that wrote code and powered the original GitHub Copilot, then from 2025 for its family of coding agents.
coding agentAn AI agent that writes and fixes software by itself: it reads a codebase, edits files, runs commands and tests, and keeps working until a feature is built or a bug is fixed.
cold start problemThe trouble a recommendation system has with brand-new users or items: with no history to learn from, it has little basis for good suggestions.
collaborative filteringRecommending things based on what people with similar tastes liked, without needing to know anything about the things themselves, as in 'people who bought this also bought…'.
compactionLike replacing a stack of rough notes with a one-page digest: when an AI's conversation grows too long for its memory, the history is condensed into a summary and the work continues from there.
completionThe text an AI language model writes in response to a prompt — literally, its continuation of the text it was given.
computational graphA map of a calculation drawn as boxes and arrows: each box is a simple operation and each arrow carries a number to the next step. AI software uses it to trace errors backward.
compute budgetThe total amount of computing a team can spend on training (or running) a model, set in advance like a money budget and used to decide how big the model should be and how much data it sees.
conformity assessmentThe check, done by the maker or by an approved outside body, that a high-risk AI system meets the EU's legal requirements before it can be sold or used.
constrained decodingForcing an AI model's output to follow a required format — such as valid code or a fill-in form — by blocking, at every step, any word that would break the rules.
content-based filteringRecommending items that resemble what you already liked, judged by the items' own features, such as a film's genre, cast and plot, rather than by other people's choices.
context engineeringManaging an AI model's reading material: choosing, trimming and arranging the instructions, documents, tool results and memories it is given on every turn, so that it has what the task needs and little else.
continuous batchingA way for an AI server to handle many conversations together, letting new ones join and finished ones leave after every word, instead of waiting for a whole group to finish.
control vectorA steering signal saved as a small file that can be loaded alongside an open AI model to turn a trait — say, cheerfulness or laziness — up or down while it generates text.
copilotAn AI assistant built into software that works alongside a person — suggesting code, drafting text or answering questions — while the person stays in charge, like a co-pilot beside the captain.
CUDANvidia's software system for programming its graphics chips to do general calculations. Most AI software is built on it, which is a big reason Nvidia chips dominate AI.
data cleaningFinding and fixing errors in a dataset — typos, duplicates, missing values, impossible numbers, wrong labels — before using it, because flawed data produces flawed results.
data flywheelA cycle in which a product's users generate data that improves its AI, the better AI attracts more users, and those users generate still more data.
data labelingThe work of attaching correct answers to examples — often done by large paid workforces, sometimes by experts — so that AI systems can learn from them or be tested against them.
data pipelineThe chain of automated steps that collects raw data, cleans and transforms it, and delivers it in a usable form — the factory line that turns messy inputs into training material.
data provenanceThe documented history of a piece of data: where it came from, who created and changed it, under what permissions, and how it was processed before reaching an AI system.
data-centric AIAn approach to building AI that focuses on improving the data — fixing labels, adding missing cases, cleaning inconsistencies — rather than endlessly tweaking the model.
Jeff DeanAn American computer scientist who built much of Google's computing infrastructure, co-founded the Google Brain deep-learning team and was Google's chief scientist before leaving in 2026 to co-found a startup.
decision modelA kind of AI language model that reads text but answers only with numbers — a yes-or-no probability, a choice or a score — instead of writing words.
DeepSpeedFree software from Microsoft for training and running AI models too big for ordinary tools, spread over many chips; it is best known for a memory-saving technique called ZeRO.
DevinA commercial AI 'software engineer' from the startup Cognition, unveiled in 2024, that takes on programming tasks in its own cloud workspace, complete with a web browser, a code editor and a command line.
digital twinA detailed virtual copy of a real machine, factory, robot or city, kept in sync with live sensor data and used to monitor it, test changes safely and predict problems.
disaggregated servingRunning the two stages of an AI's reply, reading the prompt and writing the answer, on separate groups of chips, each tuned for its own job, and handing the work between them.
distributed trainingTraining one AI model on many computer chips at once, splitting up the data or the model itself among them and keeping them in sync, so that jobs that would take a single chip thousands of years finish in weeks.
distribution shiftWhen the data an AI system meets in real use differs from the data it was trained on — new users, places, times or habits — often making it perform worse than tests suggested.
document understandingUsing AI to read documents such as invoices, forms, contracts and scientific papers the way a person does, taking in layout, tables and charts as well as words, to pull out the information needed.
English is the hottest new programming languageA catchphrase meaning that people can now get computers to do things by describing what they want to an AI in ordinary language, instead of writing code.
evaluationTesting an AI system to find out what it can and cannot do, how well, and how safely — from exam-style quizzes to hands-on trials and expert reviews.
evaluation harnessSoftware that puts AI models through tests in a fixed, repeatable way — feeding in the questions, collecting the answers and scoring them — so that every model is judged identically.
extended thinkingA setting in some AI models, notably Anthropic's Claude, that lets the model write out private step-by-step reasoning before its final answer, improving results on hard problems.
externalizationMoving an AI agent's know-how and memory out of the model itself and into files, tools and rules around it, so the whole system can do more than the model alone.
feature engineeringTurning raw data into the inputs a model learns from, using knowledge of the problem; for example, converting a timestamp into 'hour of day' and 'is it a weekend'.
feature importanceA score for how much each input — age, income, location — matters to a model's predictions. It helps people understand, debug and simplify models, but can mislead if read as cause and effect.
feedback loopA cycle in which what a system produces comes back to shape its own future input, like a microphone howling near its loudspeaker. In AI, a model's predictions can shape the very data it later learns from.
few-shot promptingShowing an AI a handful of examples of a task — inputs with the answers you want — inside the prompt, so it can copy the pattern for a new case.
fill-in-the-middleA training trick that teaches a text-writing AI to fill a gap using what comes both before and after it — essential for coding assistants that insert code in the middle of a file.
fine-tuningTaking an AI model that has already learned a lot and training it a little more on a smaller, focused set of examples so it gets better at one job or behaves a certain way.
FLUXA family of image-generating AI models from the German start-up Black Forest Labs, founded by researchers behind Stable Diffusion; some versions can be downloaded and run freely.
formal verificationUsing mathematics, not just testing, to prove that a computer system always behaves as specified — for example, that a chip never adds wrongly or that a program can never crash in a particular way.
fully sharded data parallelA feature of the popular PyTorch software that lets many chips train a model too big for any one of them, by spreading the model and its training bookkeeping evenly across all the chips.
function callingA feature of AI model services that lets the model reply with a neatly formatted request to run one of a program's functions, rather than with ordinary text; the program, not the model, then runs it.
GemmaGoogle's family of free-to-download AI language models: smaller relatives of its Gemini systems, compact enough to run on a single computer and, in some versions, on a phone.
Gemma ScopeA free collection of ready-made concept-finding tools released by Google DeepMind for its open Gemma language models, so outside researchers can study their insides without training the tools themselves.
generative recommendationA newer approach to recommendation that uses language-model-style AI to predict the next item a person will want, one step at a time, much as a chatbot predicts the next word.
GGUFA file format that packs an AI language model, often in a shrunken low-precision version, into one file that ordinary computers can load and run quickly; used by llama.cpp and Ollama.
golden datasetA small, carefully checked set of example questions with known good answers, used as a trusted yardstick to test an AI system every time it changes.
goodputThe share of a big training job's time and computing that actually moves training forward, after subtracting time lost to crashes, restarts, waiting and repeated work. When serving users, it means the rate of requests answered fast enough.
gpt-ossTwo free-to-download reasoning models released by OpenAI in August 2025 — its first downloadable language models since 2019 — that anyone may run, modify and use commercially on their own hardware.
gradient accumulationA way to train with large batches on limited hardware: the computer processes several small batches one after another, adds up what each says, and only then makes a single update.
gradient boostingA method that builds a strong predictor by adding small decision trees one at a time, each trained to correct the errors that remain after all the trees before it.
grid searchTuning a model by trying every combination from a short list of candidate settings, like testing every pairing on a menu, and keeping whichever combination scores best.
guardrailsSafety checks built around an AI system — filters on what goes in and comes out, and limits on what it may do — to keep it from producing harmful or off-topic content or leaking data.
GymnasiumA free software library that gives reinforcement learning programs one standard way to plug into practice tasks, from simple games to simulated robots, so methods can be tried on identical problems. It succeeded OpenAI's Gym.
handoffThe moment one AI agent passes a conversation or task to another, better-suited agent — like a call-center operator transferring you to the right department.
harmonised standardsOfficial European technical standards, written at the EU's request, that companies can follow to show their AI systems meet the law, instead of proving it from scratch.
harnessThe software around an AI model that gives it instructions and tools, carries out its actions and decides when it stops. The model is the engine; the harness is the rest of the car.
hookA user-supplied script or check that runs automatically at set moments in an AI agent's work — before a command, after a file edit, when it tries to stop — enforcing rules the AI cannot skip.
human-centered AIAn approach to AI that puts people's needs, abilities and control at the center: systems designed to help humans do things better rather than to replace or overrule them.
HumanEvalA set of 164 small programming exercises, each described in plain English and checked by hidden tests, used to see whether an AI can write working Python code.
human-in-the-loopDescribes AI systems in which a person checks, corrects or approves the machine's work at key points, rather than letting it act entirely on its own.
HyperbandA tuning method that starts many candidate settings on small budgets, keeps only the most promising ones, and gives the survivors more training time, so effort isn't wasted on losers.
hyperparameter tuningTrying out different settings for a model's training, such as learning speed or model size, and keeping the combination that performs best on data held back for checking.
inference providerA company that runs AI models on its own computers and sells access to them over the internet, charging by use, so customers need not buy or operate the hardware themselves.
intended purposeThe use a maker says its AI system is meant for. In EU law, the rules that apply to a system depend largely on this declared purpose.
interactive theorem provingWriting a proof in a precise computer language so that software checks every step: the person supplies the ideas, the computer fills in small steps and guarantees that nothing is wrong.
IsabelleA widely used proof-checking program in which people write mathematics and verified software; it is known for automation that calls other theorem provers to finish routine steps.
ISO/IEC 23894An international guidance standard that explains how organizations can apply ordinary risk-management practice to the particular risks of AI systems.
ISO/IEC 42001An international standard for how an organization should run its AI work responsibly — policies, risk checks, roles and records — which outside auditors can certify, much as ISO 9001 does for quality.
Johnson–Lindenstrauss lemmaA surprising fact about high-dimensional data: a random projection can squash a large set of points into far fewer dimensions while distorting every distance between them only slightly.
just-in-time retrievalLetting an AI agent fetch information only at the moment it needs it — opening a file, running a search — instead of stuffing everything into its prompt in advance.
k-nearest neighborsPredicting something about a new example by finding the few most similar examples already known and going with their majority answer, or their average.
knowledge engineeringThe craft of collecting what human experts know — through interviews, observation and documents — and turning it into rules or structured data that a computer system can use.
latencyHow long you wait between asking an AI system for something and getting the answer. Lower latency means a snappier, more responsive system.
LeanA programming language and proof checker in which mathematicians write proofs that the computer verifies line by line. It has become the main language in which AI systems produce checkable proofs.
learning to rankTraining a model to put items in the best order — search results, products, recommendations — by learning from examples of which items people judged more relevant or chose more often.
LightGBMA widely used tool for making predictions from very large tables of data, quickly and in little memory, by combining many small decision trees. Microsoft released it as free software in 2016.
LispA programming language invented in 1958 for AI research, in which programs and data are written the same way, as nested lists in parentheses. It was the main language of AI for decades.
LlamaA family of AI language models from Meta, first released in 2023, whose trained settings can be downloaded and run by anyone, making them the foundation for thousands of other AI projects.
llama.cppA free, open-source program that runs AI language models on ordinary computers and phones, without needing a powerful graphics card, by using compact low-precision versions of the models.
llms.txtA short guide file that a website puts at a standard address to help AI assistants find and understand its most useful pages.
log probabilityThe logarithm of the probability a model gives to a token or a text. Computers use it because multiplying many tiny probabilities runs into rounding problems, whereas adding their logarithms does not.
logit biasA setting that lets developers make specific words more or less likely in an AI model's output — even ban a word outright — by adjusting the model's scores before it chooses.
low-rank adaptationA cheap way to customize a big AI model: instead of changing all its billions of numbers, you train a small add-on of a few million numbers that slightly adjusts how it behaves.
MCP serverA small program that makes some service — say a database, a calendar or a company's internal tools — available to any AI assistant that speaks the Model Context Protocol.
Megatron-LMNVIDIA's open-source software, first described in a 2019 research paper, for training very large language models across thousands of graphics chips by splitting each model in several ways at once.
mini-batchA small random handful of training examples that a model learns from in a single step. Training works through many such handfuls instead of studying the whole dataset at once.
MITRE ATLASA free online reference from the US nonprofit MITRE that maps, step by step, how attackers go after AI systems, with examples drawn from real attacks and security tests.
model cardA short document published with an AI model explaining what it is for, how it was trained and tested, how well it works for different groups of people, and its known limits.
Model Context ProtocolAn open standard that lets AI assistants plug into outside apps and data — files, calendars, databases, business software — through one common connector instead of a custom link for every pairing; often likened to a USB-C port for AI.
model risk managementThe rules banks and other firms follow to make sure the mathematical models and AI systems they rely on are built correctly, checked independently and used only for what they are fit for.
model routingAutomatically choosing which AI model should answer each request, a cheap quick one for easy questions and a bigger slower one for hard ones, to save money without hurting quality.
model servingRunning a trained AI model as a live service, so that apps and people can send it requests and get answers back quickly, reliably and at a sensible cost.
MS MARCOA large Microsoft dataset of real Bing search questions paired with web passages that answer them, used to train and test most modern AI search models.
MuJoCoA free physics simulator, now maintained by Google DeepMind, that AI researchers use to train and test simulated robots and bodies, especially with trial-and-error learning.
multi-armed banditA puzzle named after a gambler choosing between slot machines: you must keep picking among options whose payoffs are unknown, learning which is best while wasting as few tries as possible on the others.
NCCLNVIDIA's software library that lets its graphics chips exchange data with each other quickly — inside one computer and across networks — and that nearly all large-scale AI training on NVIDIA hardware relies on.
negative promptA second prompt listing what you do not want in a generated image — 'blurry, extra fingers, watermark' — which the generator then steers away from.
neocloudA newer kind of cloud company that mainly rents out powerful AI chips, rather than the full range of computing services that giants like Amazon or Microsoft sell.
neural architecture searchUsing an automated search, rather than human trial and error, to design the structure of a neural network: which layers to use, how many, and how to connect them.
NeuronpediaA free website where anyone can browse, search and test the internal concepts that researchers have found inside AI models, with examples of the text that switches each one on.
NIST AI Risk Management FrameworkA free, voluntary guide from the US government's standards agency that helps organizations spot, measure and reduce the risks of the AI systems they build or use.
notified bodyAn independent testing and certification organization, approved by an EU government, that checks whether certain risky products — including some AI systems — meet the law before they can be sold.
offloadingMoving part of an AI model's data or work out of a chip's scarce fast memory onto ordinary memory, a disk or another computer, so that models too big for the chip can still run.
OllamaA free app that makes it easy to download and run AI language models on your own computer with a single command, keeping your data on your machine.
Open Source InitiativeA nonprofit group that decides what counts as 'open source' software and, in 2024, published a definition of what makes an AI system genuinely open source.
orchestrationCoordinating several AI calls, tools or agents so each does its part in the right order and their results are combined into one finished piece of work, as a conductor directs an orchestra.
orchestrator–worker patternA way of organizing AI teamwork, like a newsroom: an editor AI decides which stories a big assignment needs, sends reporter AIs to cover them at the same time, and turns what they file into one report.
OWASP Top 10 for LLM ApplicationsA widely used checklist, written by volunteer security experts, of the ten most serious security risks in software built on AI language models, with advice on preventing each one.
parameter-efficient fine-tuningA family of tricks for customizing a large AI model by training only a tiny fraction of its numbers, or a small add-on, leaving the rest untouched to save memory and money.
physics simulatorSoftware that imitates how real objects move, collide, bend and fall, so robots and their AI can be designed, trained and tested on a computer before touching the real world.
post-market monitoringThe duty of an AI system's maker to keep watching how it performs after release, gathering evidence from real use so that problems are found and fixed.
program synthesisGetting a computer to write a program automatically from a description of what the program should do — a logical specification, a few input–output examples or a request in plain language.
progressive disclosureShowing only a short summary first and revealing details when they are actually needed. AI agents use it to avoid filling their limited memory with instructions they may never use.
PrologA programming language from 1972 in which you state facts and rules in logic and then ask questions; the computer finds answers by searching for proofs.
prompt cachingSaving the work a model did reading the start of a prompt, so later requests that begin the same way can skip that reading and come back faster and cheaper.
prompt chainingA way of getting a big job done with a relay of AI requests: each handles one stage and passes its result to the next — pull out the facts, write, polish — with automatic checks between stages.
prompt engineeringThe skill of writing requests to an AI — picking the words, examples, format and background information it receives — so that it reliably gives the result you want, without retraining it.
prompt templateA reusable prompt with blanks to fill in — for example, 'Summarize this email for a busy manager: {email}' — so a program can send the same tested instructions with different content each time.
provider and deployerThe two main roles in EU AI law: the provider is whoever makes an AI system and puts it out under its own name; the deployer is whoever uses it for work or official purposes.
PyTorch FoundationA nonprofit group, part of the Linux Foundation, that owns and steers the free PyTorch software for building AI so that no single company controls it. It now also looks after related projects.
QLoRAA memory-saving version of LoRA that squeezes the big model into a compact 4-bit form while training small add-ons, letting very large models be customized on a single graphics card.
QwenA family of AI language models from the Chinese company Alibaba, many free to download, which by 2025 had become the most popular starting point for other people's open AI models.
Ralph Wiggum loopA blunt way of using AI coding assistants: feed the same instructions to the AI again and again in an endless loop until the software gets built.
random forestA prediction method that builds hundreds of decision trees, each from a random sample of the data and features, and lets them vote. The crowd is far more reliable than any single tree.
random searchTuning a model by trying settings picked at random from sensible ranges and keeping the best; surprisingly, this usually beats checking a neat grid of values.
random seedA number that fixes the 'random' choices a computer makes; reuse the same seed and settings, and an image generator produces the same picture again.
recommender systemSoftware that guesses which songs, products, posts or jobs you are most likely to want and shows those first, as when a video app lines up the next clip or a shop suggests an add-on.
reinforcement fine-tuningImproving a trained AI model by trial and error instead of examples to copy: it tries answers, a grader scores them, and the model shifts toward whatever earned high scores.
reproducibilityWhether other people, or the same people later, can get the same results by rerunning an AI experiment. Many published AI results have proved hard to reproduce, which makes real progress harder to judge.
responsible AIAn umbrella name for the practices organizations use to keep their AI systems fair, safe, private and accountable, from the first design meeting to everyday use.
retrieval-augmented generationGiving an AI an open-book exam: before answering, it looks up relevant passages in a set of documents and bases its reply on them, often citing where each point came from.
Robot Operating SystemFree, widely used software that serves as the plumbing of many robots, letting separate programs for sensors, planning and motors pass messages to each other, with a large library of ready-made tools.
robots.txtA small public file in which a website posts its rules for automated visitors, such as search-engine and AI crawlers, listing areas they are asked to stay out of. Obeying it is voluntary.
ROCmAMD's free, open-source software for programming its graphics and AI chips — its answer to Nvidia's CUDA, letting AI software run on AMD hardware.
RocqA long-established computer system, known until 2025 as Coq, in which people write mathematical proofs and verified software that the computer checks step by step.
role promptingTelling an AI to take on a part — 'You are an experienced tax accountant' — so that its answers adopt that role's knowledge, vocabulary and tone.
roofline modelA simple chart showing the fastest a program can run on a chip: limited either by how quickly data arrives from memory or by the chip's top calculating speed.
routerIn a mixture-of-experts model, the small component that decides which specialist sub-networks should handle each word, like a dispatcher assigning jobs to workers.
rule-based systemA computer system that makes decisions by following a list of 'if… then…' rules written by people, rather than by learning from examples.
sandboxA sealed-off computer space where an AI agent can run programs and change files without being able to harm the real system, reach private data or contact the outside world unless allowed.
satisfiability modulo theoriesAn extension of logic-puzzle solving that also understands arithmetic, arrays and other data types, so it can check whether conditions in software, such as 'x + y > 10 and x < 3', can ever be met.
scale-up and scale-outTwo ways to get more computing power: make each machine bigger and more tightly linked (scale up), or add more machines connected by a network (scale out).
semantic parsingTranslating a sentence into a precise, machine-executable form — such as a database query or a logical formula — so that a computer can act on what was asked.
Sentence-BERTA way of turning a whole sentence into a list of numbers so that sentences with similar meanings get similar numbers, which makes it fast to search millions of texts by meaning rather than by exact words.
SentencePieceFree software from Google that learns how to split text into tokens straight from raw sentences in any language, treating spaces as ordinary characters, so no language-specific word splitting is needed.
SGLangA free, open-source engine for running AI language and image models on servers, built to be fast when many requests begin the same way, as in chat and agent workloads.
silent data corruptionWhen computer hardware quietly produces wrong numbers without crashing or raising any alarm. In huge AI training runs on thousands of chips, such hidden errors can damage a model before anyone notices.
slopsquattingAn attack that exploits AI coding assistants' habit of inventing software packages that do not exist: attackers publish malicious code under those made-up names and wait for people or agents to install it.
small language modelA language model built to be compact enough to run cheaply and quickly, even on a laptop or phone, trading some knowledge and skill for speed, privacy and lower cost.
Software 2.0Andrej Karpathy's 2017 name for software whose behavior is learned from examples by a neural network instead of being written line by line by programmers.
Software 3.0Andrej Karpathy's name for programming computers by writing plain-language instructions for AI models, following traditional code (Software 1.0) and trained neural networks (Software 2.0).
speaker diarizationWorking out who spoke when in a recording, splitting a meeting or phone call into turns labeled Speaker 1, Speaker 2 and so on, without necessarily knowing who the speakers are.
Stable DiffusionA widely used image-generating AI whose model files were released for anyone to download, so people could run it on their own computers and adapt it.
stackingCombining several different models by training one more model to decide how much to trust each of them, based on how they performed on data they had not seen.
staged deploymentReleasing a new AI system step by step — first to testers, then to limited groups, then widely — so that problems can be spotted and fixed before everyone has it.
stop conditionThe rule that tells an AI agent when to stop working — because the task is done, a time or cost limit has been reached, or something has gone wrong.
stop sequenceA piece of text that tells an AI model to stop writing as soon as it produces it — for example, stopping at a blank line or just before the next question.
stragglerOne slow computer or chip in a large group working in lockstep. Because everyone must wait for the slowest member at each step, a single straggler can slow down an entire AI training job.
streamingSending an AI's answer piece by piece while it is still being written, so the words appear on screen one after another instead of all at once at the end.
subagentA helper AI agent that a main agent starts to handle one piece of a job; it works in its own separate workspace and reports back a short summary when it is done.
substantial modificationA change to an AI system after release that is big enough, under EU law, to require a fresh check — and that can make whoever made the change legally responsible as its maker.
SWE-agentAn open-source research system from Princeton that turns a language model into a software-fixing agent by giving it a specially designed set of commands for browsing, editing and testing code.
SWE-benchA test that asks AI to fix real bugs and add features in real open-source Python projects, checked by running each project's own tests; the best-known yardstick for AI coding agents.
system promptBackground instructions given to a chatbot before the conversation starts, setting its role, rules, tone and tools. Users usually never see it, but it shapes every reply.
TensorRTNVIDIA's software for making a trained neural network run as fast as possible on NVIDIA graphics chips, by merging and simplifying its calculations and testing which way of doing each one is quickest.
TensorRT-LLMNVIDIA's free software library for running large language models as fast as possible on NVIDIA's own graphics chips, used by many companies that host AI models.
Terminal-BenchA test of AI agents working in a computer's command line — building software, analyzing data, securing or repairing systems — with each task checked automatically inside a sealed-off container.
The StackA large free collection of computer source code from public repositories, selected mainly for permissive licenses and released from 2022 onward by the BigCode project for training coding models.
thinking budgetA limit, set by whoever uses an AI model, on how much private step-by-step reasoning it may do before answering: more thinking usually means better answers but higher cost and slower replies.
Thompson samplingA way to choose among uncertain options by picking each one about as often as it is likely to be the best, based on everything learned so far.
threat modelA clear statement of who might attack a system, what they want, what they know and what they can do — the yardstick for judging whether a defense is good enough.
throughputHow much work an AI system gets done per second across all its users at once — for a chatbot service, roughly how much text it can produce in total each second.
tokenizerThe component that splits text into tokens and converts them to numbers before an AI model sees it, then turns the model's output numbers back into text.
tokenmaxxingUsing as much AI as possible — huge prompts, long sessions, many agents at once — mainly to rank high on workplace measures of how much AI people use.
toolAny outside capability an AI model is allowed to use — a web search, a calculator, a code runner, a database or an app — described to the model so it can ask for it by name.
training loopThe repeating cycle at the heart of training software: grab some examples, let the model guess, measure the error, adjust the model, and do it again — often millions of times.
training runOne complete attempt at training a model with a fixed recipe, from start to finish. For the largest AI models a single run can occupy tens of thousands of chips for months.
TransformerLensA free software library that lets researchers open up language models and record, inspect or alter what happens inside them at every step while the model runs.
TritonA free language from OpenAI for writing the fast inner routines of AI programs in Python-style code; its compiler handles hardware details that Nvidia's own tools leave to experts.
two-tower modelA design for recommendation and search in which one neural network turns users or queries into lists of numbers and another does the same for items, so good matches can be found by quick comparison.
value-sensitive designA way of designing technology that deliberately identifies the human values at stake, such as privacy, fairness or autonomy, and builds them in from the start instead of patching problems later.
vibe codingProgramming by conversation: you ask an AI for what you want, try out what it produces and ask again until it works, trusting the code without really reading it.
vLLMA free, open-source program for running large language models on servers efficiently, so one machine can answer many more users at once. Many companies use it to host AI models.
web crawlerA program that automatically travels the web, following links from page to page and downloading what it finds — used by search engines, archives and AI companies to gather data.
web scrapingUsing software to automatically copy information from websites — text, prices, images — in bulk, which is how much of the data used to train AI was gathered.
WhisperA free OpenAI model that writes down what is said in recordings in many languages, and can translate it into English; it learned from hundreds of thousands of hours of online audio.
wrapperA product built mostly on top of someone else's AI model, adding a simple interface or a few instructions; often used as a put-down implying the product is easy to copy.
XCONA rule-based program used from 1980 by the computer maker Digital Equipment Corporation to work out which parts each customer's computer order needed. It was the first big commercial success of expert systems.
XGBoostA fast, free software tool that makes predictions by adding up hundreds of small decision trees, each one fixing the mistakes of those before it. For years it was behind many winning entries in competitions on spreadsheet-style data.
XLAA compiler from Google that turns AI programs into fast code for chips such as TPUs and GPUs, mainly by merging many small steps into fewer, bigger ones.
ZeROA memory-saving technique for training big models on many chips: instead of every chip keeping its own complete copy of all the training bookkeeping, the chips divide it among themselves and share pieces only when needed.