DSPy
In plain wordsA free software framework in which programmers describe what each step of an AI pipeline should do, and the framework automatically works out good prompts and examples instead of hand-writing them.
An open-source Python framework for 'programming, not prompting' language models. Developers declare each step of a compound AI systemcompound AI system, n.: An AI product built from several working parts — models, search tools, code and checks — designed to work together, rather than a single model answering on its own. as a module with a signature — a typed specification of inputs and outputs such as 'question, context -> answer' — and compose modules (Predict, ChainOfThought, ReAct and custom ones) in ordinary Python; DSPy turns signatures into prompts for whichever model is configured. Optimizers, originally called teleprompters, then compile the program against a small training set and a metric: they bootstrap demonstrations from successful runs, search over instructions and example sets (MIPROv2), evolve prompts by reflecting on execution traces (GEPA), or fine-tune model weights. Separating program logic from prompt wording makes pipelines portable across models and turns prompt optimizationprompt optimization, n.: Using software, often another AI model, to search automatically for the wording and examples that make an AI model perform best on a task, instead of tweaking prompts by hand. into a repeatable, measurable step, although results depend on the quality of the metric and data.
We rewrote the extractor as a DSPy module and let the optimizer choose the few-shot examples.
Origin
Developed by Omar Khattab and colleagues in the Stanford NLP group, with Christopher Potts and Matei Zaharia, as the successor to the Demonstrate–Search–Predict framework (December 2022); the DSPy paper appeared in October 2023 and was published at ICLR 2024. The project glosses its name as 'Declarative Self-improving Python'.
Related
- prompt optimizationprompt optimization, n.: Using software, often another AI model, to search automatically for the wording and examples that make an AI model perform best on a task, instead of tweaking prompts by hand.
- compound AI systemcompound AI system, n.: An AI product built from several working parts — models, search tools, code and checks — designed to work together, rather than a single model answering on its own.
- LangChainLangChain, prop. n.: A popular free software toolkit, and the company behind it, for building applications that connect AI language models to documents, databases, tools and multi-step workflows.
- few-shot learningfew-shot learning, n.: Getting an AI system to handle a new task or recognize a new kind of thing from just a handful of examples, the way a person can learn a new word after hearing it once or twice.
- prompt engineeringprompt engineering, n.: The skill of writing requests to an AI — picking the words, examples, format and background information it receives — so that it reliably gives the result you want, without retraining it.
- evals-driven developmentevals-driven development, n.: Building an AI product by first writing tests that show what good answers look like, then accepting changes to prompts, models or code only when the test scores improve.
Sources
- Khattab, O. et al. (2023). DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines. arXiv:2310.03714 (ICLR 2024).
- Khattab, O. et al. (2022). Demonstrate-Search-Predict: Composing Retrieval and Language Models for Knowledge-Intensive NLP. arXiv:2212.14024. arxiv.org/abs/2212.14024
Mentioned in
agent frameworkagent framework, n.: A software library that gives developers ready-made parts for building AI agents — model connections, tool handling, memory, multi-agent coordination and logging — so they need not write everything from scratch., LangChainLangChain, prop. n.: A popular free software toolkit, and the company behind it, for building applications that connect AI language models to documents, databases, tools and multi-step workflows., prompt templateprompt template, n.: A reusable prompt with blanks to fill in — for example, 'Summarize this email for a busy manager: {email}' — so a program can send the same tested instructions with different content each time., ColBERTColBERT, prop. n.: A search model that compares a question and a document word by word — matching each question word to its best partner in the document — while still letting documents be processed ahead of time for speed., compound AI systemcompound AI system, n.: An AI product built from several working parts — models, search tools, code and checks — designed to work together, rather than a single model answering on its own., prompt optimizationprompt optimization, n.: Using software, often another AI model, to search automatically for the wording and examples that make an AI model perform best on a task, instead of tweaking prompts by hand.