Critic Experience Bank: Self-Evolving Step-Level Confidence Estimation for LLM Agents
Critic Experience Bank (CEB) introduces a self-evolving, execution-feedback-driven step-level confidence estimation framework that enables LLM agents to predict—before execution—whether each proposed action is productive, thereby improving robustness against irreversible errors.
Core Contribution: Pre-execution discrimination of irreversible failures in LLM agents
LLM agents operate in external environments where each action modifies the state upon which subsequent decisions depend; a single erroneous step can rapidly exhaust interaction budgets or trigger irreversible side effects long before final failure manifests. Reliable deployment thus demands step-level confidence estimation: a calibrated probability—available before action execution—that each proposed action will productively advance the task.
Fundamental limitation of existing approaches
Current LLM confidence estimators (e.g., response scoring or self-evaluation) score static prompt-response pairs and ignore execution consequences: whether similar actions in similar states actually advanced the task after environmental feedback was observed. This causal gap decouples confidence from real-world utility.
Critic Experience Bank (CEB): A hindsight-feedback-driven self-evolving critic
- CEB is an LLM-based self-evolving critic framework, centered on building and continuously updating a structured Experience Bank;
- After each full trajectory, a hindsight LLM—fed the complete execution trace and outcome feedback—votes on whether each step was productive;
- These hindsight judgments generate high-quality pseudo-labels, stored in the Experience Bank alongside context (state, action, environment feedback);
- At inference time, the critic retrieves semantically similar productive/unproductive experiences from the Bank and dynamically injects them into its prompt for context-aware confidence calibration.
Key technical characteristics and value
- 【Preserve information value · Critical】Method name strictly retained as Critic Experience Bank (CEB); paper ID: arXiv:2607.12397v1;
- Fully inference-time—no fine-tuning or additional training required; compatible with any LLM agent architecture;
- Enables cross-task generalization via semantic retrieval—not hardcoded rules;
- Empirical results show CEB significantly reduces wasteful calls (e.g., redundant API invocations, dead-end subgoals), improving success rates and resource efficiency in long-horizon tasks.