dspy
DSPy is the programming framework that replaces hand-crafted prompts with compilable, optimizable modules for building LLM pipelines — developed at Stanford NLP, DSPy treats prompt engineering as a programming problem where modules declare what they need (signatures) and compilers automatically optimize prompts, few-shot examples, and fine-tuning to maximize pipeline performance on specified metrics.
What Is DSPy?
- Definition: A framework where LLM pipelines are built from declarative modules with typed signatures, then automatically optimized by compilers (teleprompters) that find optimal prompts and examples.
- Core Innovation: Separates the program logic (what to compute) from the LLM instructions (how to prompt), enabling automatic optimization.
- Key Concept: "Signatures" define input/output types; "Modules" implement reasoning patterns; "Teleprompters" compile and optimize.
- Creator: Omar Khattab and the Stanford NLP group.
Why DSPy Matters
- No Manual Prompting: Compilers automatically discover optimal prompts and few-shot examples — no prompt engineering required.
- Composability: Modules (ChainOfThought, ReAct, ProgramOfThought) compose into complex pipelines.
- Optimization: Teleprompters systematically search for configurations that maximize task-specific metrics.
- Reproducibility: Pipelines are programmatic and deterministic, unlike ad-hoc prompt engineering.
- Portability: Change the underlying LLM without rewriting prompts — DSPy recompiles automatically.
Core Abstractions
| Concept | Purpose | Example |
|---|---|---|
| Signature | Declare input/output types | ``question -> answer`` |
| Module | Implement reasoning patterns | ``dspy.ChainOfThought(signature)`` |
| Teleprompter | Optimize modules automatically | ``BootstrapFewShot``, ``MIPRO`` |
| Metric | Define success criteria | Accuracy, F1, custom functions |
| Program | Compose modules into pipelines | Class with ``forward()`` method |
How DSPy Compilation Works
1. Define: Write program using DSPy modules with signatures.
2. Provide: Supply training examples and evaluation metric.
3. Compile: Teleprompter searches prompt/example space to maximize metric.
4. Deploy: Use compiled program with optimized prompts for inference.
Built-In Modules
- Predict: Basic LLM call with signature.
- ChainOfThought: Adds reasoning before answering.
- ReAct: Interleave reasoning and tool actions.
- ProgramOfThought: Generate and execute code for answers.
- MultiChainComparison: Run multiple chains and select best.
DSPy is a paradigm shift from prompt engineering to prompt programming — proving that systematic optimization of LLM instructions through compilation produces more reliable, portable, and performant pipelines than manual prompt crafting.