What matters in this issue
Agent reliability is becoming a set of concrete systems questions: how an agent proceeds after bad tool information, whether its harness faithfully conveys errors, and whether recovery may repeat an external action whose outcome remains uncertain. Start with MCP-Injector and TraceTrial-Bench. One injects faults to measure resilience; the other uses evidence across boundaries to check harness obligations. Both show why the final answer cannot evaluate the whole execution.
The other selected papers examine environmental checks before execution, correction during multi-agent discussion, recovery after authority changes, and KV-cache retrieval. Community developments add AKV’s cache-operation semantics and Resin’s runtime boundary. These offer distinct intervention points: expose errors earlier, ground recovery in evidence, and retain explicit operating conditions for performance improvements.
01 · MCP-Injector: model scale cannot substitute for recovery mechanisms
Tool-Calling LLM Agents Under Realistic MCP Faults: Scale Does Not Purchase Fault Tolerance
Antonio Ken Iannillo, Joshua Segun Owotogbe, Roberto Natella, Francesco Avallone, Kristina Kudryavtseva, Indika Kumara · 2026-10-23
Put faults at the tool-call boundary
An agent that succeeds with healthy tools has not necessarily learned to handle incorrect values, type mismatches, or calls that never return. MCP-Injector makes these external failures controllable: middleware between the MCP client and server observes a call, selects an injection, applies it, and records the event. It can mutate a real response or introduce delays, explicit errors, and silent hangs without rewriting the agent.
The contribution centers on the fault profile and persistence schedules. The authors analyze 57,240 issue/PR threads from 1,774 repositories and combine six failure manifestations with permanent, transient, and intermittent behavior. Seeds control injection schedules, while a pass-through proxy checks whether interposition itself changes results. These proportions describe reported incidents, however, rather than the probability that a production call fails. Approximately 91.3% of the injection profile is permanent, making it best understood as a stress test.
Read left to right through observe, decide, act, and record; the return path can carry a mutated tool result.
Finishing a run is different from accomplishing the task
The experiment crosses 25 single-server tasks, gpt-oss-20B and 120B, three agent stacks, three proxy conditions, and three seeds, producing 1,350 experimental cells. Both models run at MXFP4 precision through LM Studio on Apple Silicon with a shared 16K context window. Tasks have a 3,000-second timeout plus framework-specific loop bounds. An LLM judge scores repeatedly, with median aggregation.
Under injected faults, the pooled task-completion quality score falls from 8.08 to 3.23, approximately 60%. Even among runs that terminate normally in both conditions, the decline remains 57%. Monitoring whether a process survived and an answer appeared would therefore miss considerable quality loss. Some agents bypass tools and answer from prior knowledge, further separating apparent completion from grounded execution.
The 120B model shows no detectable advantage over 20B in retaining its own fault-free quality score (p=0.96). Its completion rate under faults is 91.1%, compared with 95.6% for 20B, but the confidence interval for the difference crosses zero. This does not establish that scaling reduces reliability. The stacks also behave differently, although error handling and call volume change together, leaving the framework effect suggestive.
Seeds also need careful interpretation. They make injection decisions reproducible for a given call history, not the entire agent execution bit-for-bit deterministic. Generation and judging remain stochastic, and changes in tool-call order can change actual fault exposure. Separating controlled scheduling from end-to-end determinism matters when designing follow-up comparisons.
Prioritize the fault profile and the decomposition of results. The AgentOps contribution is an evaluation that connects response content, timeouts, retries, and answer grounding instead of comparing only healthy-task scores. The study covers one model family and 25 single-server tasks; very few transient or intermittent events actually fire. It does not establish a universal scaling law across models and failure regimes.
02 · TraceTrial-Bench: successful answers can conceal harness failures
Trial by Trace: Benchmarking AI Agent Harnesses for Reliability and Failure Attribution
Zhiyong Cao, Hengjie Zhu, Yongxin Wang, Xuchen Li, Jintao Li, Shuxia Liu, Dunqiang Liu, Bo Wang, Kairan Zhang, Cong Lin, JiangDi Yu, Siyuan Liu, Sijia Yao, Yi-Liang Wei, Lingxu Ma, Aijia Wang, Anxin Chen, Zixian Wei, Xiang Ruan, Ruizhi Xu, Ruqian Shi, Baihua Xiao, Sicheng Zhou · 2026-10-23
Score task outcomes and harness obligations separately
When an agent fails, changing the model is often easier than diagnosing its execution harness. Yet the final answer cannot tell us whether an exit code survived, a long output was truncated, or a child task terminated correctly. TraceTrial-Bench treats the native harness as the system under test. Its 95 tasks cover seven dimensions: context, execution, governance, lifecycle, observability, tools, and verification, with explicit triggers and external truth anchors.
Adapters collect traces while preserving native prompt construction, permissions, and tool behavior. Each task separately checks the agent’s actions and deliverables, and the harness’s execution and information-delivery obligations. An actual tool response, the payload visible to the model, a session archive, and a final file need not agree. A process might time out without that status reaching the model. Comparing these boundaries makes information loss observable.
Tasks exercise native harnesses; multiple evidence sources support separate scores, and candidate mechanism chains still require review.
Attribution needs intermediate evidence
The system normalizes native records into a shared event sequence while retaining their provenance. Calls and results that cannot be correlated reliably remain unlinked. Declared selectors build criterion-specific evidence packets: an environment ledger establishes external effects, model-facing records establish what was visible, and workspace differences establish persisted deliverables. Harness evaluation can thus look beyond an agent’s own assertion of success.
The experiment retains 3,420 executions across four harnesses, nine model groups, and 95 tasks. All Agent criteria pass in 39.5% of runs; all criteria in both layers pass in only 8.4%. The denominator is the same, but the questions differ. The latter does not mean that every remaining run crashed. The harness with the highest task-success rate also differs from the one with the highest harness-criterion satisfaction, exposing trade-offs hidden by a single leaderboard.
Joint failure still does not prove causation. The paper calls a match between predefined endpoints and activation conditions a candidate mechanism link. Validating propagation additionally requires checking both endpoint verdicts, event order, observable intermediate distortion, and competing explanations. Likewise, a failed scoring criterion is not automatically a confirmed software defect.
Another 31.1% of runs contain failed Harness criteria while satisfying every Agent criterion. Application behavior can absorb some problems, and retaining only final successes would overlook these harness weaknesses. This observation does not establish that the same problems would remain harmless on other tasks.
The reusable contribution is the evidence contract, rather than the ranking alone. Each combination has one selected execution, sometimes replacing an earlier run; task and judge-output budgets are not fully uniform, and within-cell repeatability is not estimated. Platform teams can adapt this approach into harness regression tests: preserve external truth and model-visible evidence independently, check task completion and runtime obligations separately, and build reviewable explanations for failures.
03 · StateMirror: validate environmental prerequisites before tool execution
StateMirror: Pre-Execution Detection of Hallucinated Tool Calls in Coding Agents
Shiwen Shan, Yintong Huo, Zhiying Wu, Yuxin Su · 2026-10-23
A syntactically valid tool call can still name a nonexistent file, module, or exported symbol. StateMirror normalizes proposed actions, extracts prerequisites through command analysis and Python ASTs, and probes the current workspace read-only. Statically unknown values are omitted, and uncertain checks pass conservatively. It checks action feasibility, not whether a patch is semantically correct.
Normalization feeds predicate extraction; environment probes report prerequisites confirmed to be false.
Across 2,508 actions from four agent sources, precision is 0.940, recall 0.722, and F1 0.817, without LLM tokens for detection. On an Apple M4 Max with 36 GB RAM, median latency is 1.50 ms, but p95 reaches 306.66 ms; the median alone cannot budget every action. Most misses arise in prerequisite extraction and state reconstruction, and recall is lower on the resolved-trajectory subset. This is a concrete execution-gate design for coding agents, bounded by the environmental facts it can extract and check.
04 · Multi-agent communication: how correct minority answers disappear
Understanding Communication Failures in Multi-Agent LLM Systems
Priya Pitre, Jun Hou, Yi Fang, Naren Ramakrishnan, Xuan Wang · 2026-10-23
The study selects 3,211 failed multi-round trajectories from 7,300 task executions. It compresses behavior into evidence-backed cards, then maps, clusters, and expands labels into a communication-failure taxonomy. Human annotation checks 200 failed trajectories, while 500 successful trajectories provide a contrast set.
Map existing categories, cluster uncovered labels, expand or subdivide categories, then remap.
The temporal pattern is useful: consensus problems increase after agents first see peer responses, while later rounds more often involve failed correction. The resulting recommendation is to preserve disagreement and corrective signals rather than merely add agents. In the HotpotQA and MuSiQue comparisons, retrieval reduces overall failure from 74.6% to 36.4%, a result specific to those QA settings. The taxonomy is primarily failure-derived and uses LLM annotation. It is a diagnostic vocabulary, not a way to infer production failure rates from category shares, and it does not thoroughly cover collaboration through files or long-horizon tool workflows.
05 · Permission to Retry: preserve uncertainty about external effects
Permission to Retry: A Study of Authorization and Recovery in AI Agent Workflows
Deekshith Marla, Kushagra Bhatnagar · 2026-10-22
A request may take effect even when its acknowledgment is lost, and permission may be revoked before recovery. The paper separates checkpoints, current dispatch authority, evidence of external effects, and reconciliation authority. Crossing 14 schedules, three provider contracts, and four application policies on LangGraph and Temporal yields 336 finite test histories.
Ledger reconciliation satisfies the declared contract in all 42 histories per host, but only 21 are checked completions: 15 are denials and six remain unknown with reservations retained. That decomposition matters more than “100% passing.” Safe recovery can require neither resending nor assuming the old request failed. Matching outcomes across the two hosts do not establish equivalence of all framework behavior. Scripted providers and a specific authority contract also cannot establish atomicity between real IAM and external services. The paper has no standalone method architecture figure; this section explains the recovery boundary in text.
06 · KVStream: use spare storage bandwidth for KV retrieval
KVStream: Adaptive Parallel Storage Management for KV Caching in Large-Scale LLM Serving
Avinash Maurya, Jaiaid Mobin, Jie Ye, M. Mustafa Rafique, Franck Cappello, Bogdan Nicolae · 2026-10-22
When reusable prefixes exceed GPU and host memory, storage retrieval can block inference. KVStream pools a step’s reads, replicates each KV chunk across storage tiers, and lets whichever tier finishes first claim more work. Writes are deferred into the forward pass. Completion-driven read assignment buys bandwidth flexibility at the cost of additional copies and writes.
The shared filesystem is on the left, independent node storage paths in the center, and pre-optimization read/write contention on the right.
Across four models and six workloads, single-node H100 offline throughput reaches 1.49× LMCache. For a SWE-bench agent request stream on one A100 node, mean time to first token improves 6.07× at an arrival coefficient of variation of 4. This includes reduced queueing, not a sixfold disk-read speedup. At only 4% prefix reuse, the speedup falls to 0.81×, a regression. Deployment decisions should start with cache misses and the read/write balance: replication offers little when the working set stays in memory or new writes dominate.
Community and project developments
AKV: cache reuse must also match operation history
AKV’s operation-history constraints on cache reuse are covered in the October 22 paper discussion. That discussion complements this issue’s KVStream storage-read mechanism.
Resin: separate scheduling from model execution
Resin project · Eric Zhang · Architecture
Resin gives its Rust core ownership of requests, batching, and caches. Model runners receive frozen execution plans and manage weights and kernels. Its failure boundary deserves attention: when execution may have modified state, resources are quarantined; successful device synchronization alone does not make a retry safe. Runtime reuse and reliability contracts become one design problem. The project is explicitly experimental, and the current architecture lacks multi-node serving and prefill/decode disaggregation. It is useful research infrastructure to follow, without inferring production performance advantages from social-post line counts or charts.