Test-Time Learning with an Evolving Library
EvoLib is a test-time learning (TTL) framework that enables large language models to accumulate, reuse, and evolve knowledge across problem instances without parameter updates or external supervision—by maintaining and evolving a shared library of knowledge abstractions (including modular skills and reflective insights) automatically extracted from the model’s own inference trajectories—achieving substantial gains over state-of-the-art TTL and test-time scaling methods on mathematical reasoning, code generation, and multi-turn agentic benchmarks, all without ground-truth feedback.
Core Contribution: Parameter-Free, Self-Evolving Test-Time Learning via EvoLib
EvoLib introduces a novel test-time learning paradigm that eliminates both parameter adaptation and external supervision. Its core innovation is a dynamically maintained and evolved library of knowledge abstractions, comprising:
- Modular skills (e.g., subtask decomposition patterns or tool-use protocols);
- Reflective insights (e.g., failure root-cause analysis, hypothesis revision, or cross-instance analogical generalization).
All abstractions are automatically extracted from the LLM’s own inference trajectories—no human annotation or RLHF signal required.
Knowledge Evolution Mechanism: Joint Optimization for Immediate Utility and Long-Term Value
To enable continual knowledge refinement and generalization, EvoLib introduces a principled weighting and consolidation mechanism:
- Each newly generated abstraction is scored jointly on its immediate utility in the current instance and its long-term reusability across future instances;
- A learnable weighting function drives dynamic deduplication, merging, and generalization—for example, consolidating multiple concrete algebraic manipulations into a reusable symbolic transformation rule;
- Enables instance-specific, narrow abstractions to gradually evolve into broader, transferable knowledge units over time.
Empirical Results: SOTA Performance Without Ground-Truth Feedback
EvoLib is rigorously evaluated on three challenging test-time learning benchmarks:
- Mathematical reasoning (GSM8K, MATH);
- Code generation (HumanEval, MBPP);
- Multi-turn agentic environments (ALFWorld, WebShop).
Results show EvoLib consistently outperforms state-of-the-art test-time scaling (e.g., ToT, Self-Refine) and test-time learning methods (e.g., STaR), with all gains achieved without access to ground-truth feedback.