MAGE: Understanding Stability-Performance Trade-offs in Multi-component Prompt Optimization
MAGE (Memory-Augmented Goal-directed Prompt Evolution) reveals the previously unreported Prompt Optimization Coupling Effect (POCE): when multiple stochastic optimization signals operate within a closed reflective loop, they jointly boost performance *and* amplify variance—a phenomenon unpredictable from isolated component analysis; failure-grounded reflection is essential, while score-only (OPRO) or abstract-critique-only (Self-Refine) methods fail to improve prompts.
Core Finding: The Prompt Optimization Coupling Effect (POCE)
MAGE (Memory-Augmented Goal-directed Prompt Evolution) is a controlled analysis framework designed to systematically study component interaction in prompt optimization. It integrates episodic memory, multi-objective Pareto selection, and adaptive evaluation—not as an absolute-state optimizer, but as a platform for precise, controlled ablation studies.
- The paper reports the first empirical identification and formalization of the ‘Prompt Optimization Coupling Effect’ (POCE): when multiple stochastic optimization signals (e.g., failure-driven reflection, memory retrieval, Pareto filtering) coexist within a closed reflective loop, their joint dynamics simultaneously increase task performance and output variance—an emergent behavior that cannot be predicted by analyzing any single component in isolation.
Key Empirical Results
- Failure-grounded reflection is necessary: OPRO (relying solely on scalar scores) and Self-Refine (generating only abstract textual critique) both fail to improve prompt quality on GSM8K-Hard; MAGE’s failure-case-driven reflection mechanism is empirically validated as the critical enabler of performance gain.
- Co-evolution of performance and variance: On GSM8K-Hard, MAGE achieves 46.4% accuracy versus GEPA’s 34.0% (+12.4%, P(MAGE>GEPA)=0.998, 5 seeds on gpt-4o-mini), with comparable variance (7.3% vs GEPA’s 7.1%), confirming that POCE-driven gains need not entail destabilized outputs.
Methodological Positioning
- MAGE is not an end-to-end optimization tool, but a foundational research platform for prompt engineering: all components—episodic memory, Pareto selection, adaptive evaluation—are modular and independently toggleable, enabling rigorous causal attribution.
- Published on arXiv:2607.11944v1, this work constitutes a mechanistic study in LLM prompt optimization, grounded in controlled experimentation rather than heuristic deployment.