In this issue
This issue focuses on failures hidden by apparently successful agent execution. A fixed RCA workflow may skip decisive evidence. State may survive a container restart while unfinished work never resumes. Prompt caching may silently fail while answers remain correct. Three studies examine those boundaries through shared evaluation, platform events, and real fixes.
Other readings separate the value of independent reviewers, constructed memory, and retrospectively generated rules from their implementation alone. On the serving side, ZipFusion fuses lossless KV reconstruction into attention, while InferScale exposes trade-offs among prediction, cold starts, and resource use. Across these results, distinguish aggregate accuracy, local improvements, and complete system behavior.
01 · RCA: letting the model choose its investigation
Free to Look, Free to Reason: Simple LLM Harnesses Outdiagnose Hand-Crafted Root Cause Analysis Agents
Bi, Tingzhu, Zhang, Xun, Li, Jinglin, Wang, Ping, Ma, Meng · 2026-10-26
Microservice RCA agents often encode expertise as fixed procedures, stages, or roles. This study asks whether those prescriptions still help a capable model that can investigate telemetry directly. Its useful contribution is a shared comparison of three simple harnesses, five published LLM RCA methods, and statistical or causal approaches.
Compare control over the investigation
RCAtlas harmonizes eight public data sources and selects 90 validated cases across 23 scenarios. Methods receive incident and reference windows, an entity inventory, and available telemetry. Entity-level scoring maps fine-grained answers to the dataset’s annotation level.
The simple alternatives differ: one-shot receives a rule-selected evidence package; a SQL agent chooses queries; Codex writes analysis code. Only the latter two dynamically control evidence collection. None imposes a domain-specific sequence of model calls.
The two flexible approaches inspect logs and downstream traffic; fixed workflows stop at a symptomatic service. Red dots mark decisive steps.
The illustrated failure makes the distinction concrete. Cart emits errors because inventory stops receiving requests. Flexible investigation follows connection-refusal logs and collapsing downstream traffic; a fixed procedure omits logs, while a call-chain walk misses the service receiving no calls. This is a mechanism to examine when evaluating an RCA workflow: does its control flow make decisive evidence unreachable?
Interpret the comparison carefully
The main table reports PR@1 of 65.3–77.2% for simple harnesses, 15.6–53.3% for published LLM methods, and 38.9% for the strongest non-LLM baseline, NSigma. These summarize the table’s mid-tier model settings; some method–model combinations were not run. They are not results from a fully populated matrix. Simple harnesses also lead on models where both families were evaluated.
Repeated runs separate two useful properties. Stronger models mainly improve consistency; multi-turn harnesses mainly increase the set of cases solved at least once. Higher reasoning effort can add substantial time for relatively small accuracy changes. For incident response, that distinction matters: an occasional correct investigation is different from a dependable diagnosis.
Read the shared evaluation and investigation traces first. Some baselines were adapted or reimplemented, only one general coding harness was tested, and the case study does not isolate causality through an ablation. The prescribed-call ratio measures control structure, not proof that prescriptions cause failure. A productive implication is to benchmark mandatory procedures against optional guidance with access to raw evidence. Domain knowledge may still help; the question is how the harness exposes it without closing off useful investigation paths.
02 · Containerized agents: persistence, resumption, and shutdown are separate guarantees
Built to Remember, Shipped to Forget: Operational Contracts, Failures, and Repairs in Containerized LLM Agent Deployments
Mastouri, Meriem, Ksontini, Emna, Barrak, Amine · 2026-10-25
Agent frameworks retain conversations, pause for human input, and resume saved work. Container platforms restart, replace, and terminate processes. Both components can behave as documented while their composition breaks an agent’s promise. This study expresses the missing obligations as operational contracts.
Turn documentation into deployment checks
The authors combine first-party LangGraph and Docker, Compose, and Kubernetes documentation, then validate and consolidate 14 contracts. They cover saved state, running work, clients, and external resources. One example pairs a 180-second framework drain window with a 10-second container shutdown grace period: a shutdown handler cannot finish work if the platform kills it first.
The four rows derive contracts, inspect repositories, induce platform events, and examine repairs. Static predictions remain distinct from runtime evidence.
Repository inspection covers 433 systems and distinguishes inapplicable, satisfied, violated, and undecidable contracts. Among 265 systems needing persistent state across requests, 99 fail the process-restart obligation and 110 fail the container-replacement obligation: 37.4% and 41.5%. These are proportions within the applicable subset, not production incident rates.
A checkpoint does not restart a job
Persistence answers whether data survive. It does not establish that a worker will reclaim an unfinished job, that replayed steps have harmless side effects, or that a streaming client can recover missing output. Of 61 relevant background systems, 54 provide no resumption mechanism in the repository; 28 of those already preserve state across process lifetimes. A database is therefore insufficient evidence of recovery.
The runtime study selects 12 runnable systems and induces seven classes of platform event. Fourteen of 16 predicted violations manifest as corresponding failures; all seven scorable predictions of satisfaction pass. Two predicted violations do not reproduce because the deployed image supplies safeguards absent from repository evidence. That discrepancy is valuable: static deployment checks cannot fully describe behavior hidden inside an image.
Repairs must cross the same boundary
A separate history study identifies 122 repairs. Seventy-eight percent change application code only, and roughly two-thirds fully restore the relevant obligation. Adding a shutdown handler or marking work interrupted can still leave the platform grace period or resumption mechanism unresolved.
The transferable contribution is a better set of assertions for agent chaos testing. Check state retention, eventual completion, repeated side effects, and client delivery separately. The evidence is centered on LangGraph; runtime subjects were selected for contract coverage, with few systems per contract. It supports a structured deployment-review and fault-injection starting point, not an ecosystem-wide failure-rate estimate.
03 · CACHELINT: correct answers can hide broken caching
Silent and Costly: Characterizing and Detecting Prompt-Caching Mis-Integration in LLM Agents
Zhang, Ying, Guo, Ruoqi, Liu, Yi, Deng, Gelei, Li, Yuekang, Liu, Yepang, Zhang, Leo · 2026-10-26
An agent can answer correctly while prompt caching silently stops saving money. A timestamp may enter the cached prefix, a renamed model may fail a hard-coded capability check, or returned cache-token counts may be mishandled. This paper treats these as integration defects with their own correctness conditions.
Separate request construction from accounting
The authors identify 120 defects from merged fixes in ten agent systems: 97 on the request side and 23 in usage handling. Of the 120, 108 produce no explicit error. Some waste computation; others leave caching intact but corrupt the telemetry used to judge it. Neither a successful request nor a plausible dashboard proves correct integration.
CACHELINT models a defect as an invocation site plus an unmet precondition. CodeQL locates cache directives and usage fields. Checks then concern prefix stability, provider capabilities, or accounting safety. Context matters: a clock reference is harmless if it never reaches the cached span or has already been normalized appropriately.
Provider support is a different kind of fact. Cache markers and TTLs vary across APIs and over time. The detector compares code assumptions against a dated, sourced capability table. It uses an LLM for bounded code-context guard checks, such as whether sorting or a null check resolves a suspected problem, rather than asking the model to recall current provider capabilities.
Keep the evaluation denominators visible
On the ten projects used to derive the rules, precision is 0.71 and recall 0.78. The paper explicitly treats that recall as an in-sample upper bound. Across 571 additional projects, manual review finds 32 true positives among 37 warnings, giving 0.86 precision. Exhaustive defect labels are unavailable there, so recall is unknown. The body reports 21 developer confirmations and 18 fixes.
The single-site model misses behavior requiring multiple snapshots, particularly cross-turn mutation and prefix reassembly. Together these account for 24 of the 120 labeled defects and most false negatives. LLM guard judgments can also fail, and the external capability table requires maintenance.
The operational contribution is to extend correctness to cost and measurement semantics. A useful cache check should explain agreement among constructed prefixes, provider capabilities, response counters, and accounting. CACHELINT makes part of that reasoning actionable at code locations; runtime evidence remains necessary for behavior spanning calls.
04 · Verification harnesses: a separate critic can offer better value
Rethinking Verification in Agent Harnesses
Zhang, Zhengxin, Zhu, Erkang, Lin, Yin, Cardie, Claire, Ding, Bolin, Zhou, Jingren · 2026-10-25
The study compares 17 designs across initial instructions, execution reminders, completion gates, and independent review. A critic starts with fresh context, reads the task and changes, and tests the live sandbox. It cannot edit the deliverable; the main agent receives findings and repairs it.
Across seven coding models on selected Terminal-Bench and SWE-bench Verified subsets, independent review improves average accuracy by 5.9–9.5 and 3.9–6.6 percentage points. Total costs, including review and revision, are 1.13–1.25× and 1.00–1.04× baseline. Repeatedly rejecting completion can help but costs more. Full trajectories do not consistently beat task-and-change context and increase critic cost.
The useful contribution is a comparison of verification placement and economics. These are cross-model averages on subsets; individual combinations regress. Read the cases where critics construct counterexamples missed by the original agent.
Follow the central workflow: the first three layers prompt self-checking; the last returns findings from a separate critic.
05 · Memory Tax: preserved information can become harder to retrieve
Memory Tax: Dissecting the Net Value of Agent Memory
He, Xuan, Zhoubian, Sining, Yang, Ruozhen, Wen, Yuhao, Wang, Zhineng, Hu, Yuyang, Yang, Ke, Zhang, Yuyu, Nie, Ping, Han, Jiawei, Zhai, ChengXiang · 2026-10-25
Using retrieval from raw history as the reference, this study separates questions a memory system newly solves from questions it newly loses. Comparisons of eleven representative systems, three backbones, and three main benchmarks show that additional processing often fails to offset information loss and retrieval difficulty.
The diagnosis goes beyond aggregate accuracy. Supplying gold memory units bypasses retrieval to expose construction losses; repeated retrieval probes accessibility; swapping construction and answering models tests sensitivity. Retrieving raw history alongside memory is one evaluated recovery path.
For long-lived agents, the contribution is a demand to measure memory’s net value against direct retrieval. The evaluation is mainly static, offline, and English, with stratified subsets for some benchmarks. Retrieval-friendly tasks leave less headroom for elaborate memory. The findings do not establish that memory is universally ineffective.
Light extensions show gains from bypassing retrieval; remaining red-box gaps indicate that construction can also lose useful information.
06 · Harness self-evolution: retained advice must earn its place
How Do Coding-Agent Harnesses Self-Evolve? An Empirical Study of Retrospective Adaptation
Treude, Christoph, Baltes, Sebastian · 2026-10-26
Claude Code produced 125 retrospective proposals after 30 SWE-bench Verified tasks. Human selection led to 24 implementations and evaluation of 12 adaptations on later issues in the same repositories, using fresh sessions and containers.
Among seven informative evaluations, two changes eliminated the targeted action, one reduced it, and four did not. Naming the correct test runner prevented attempts to use unavailable pytest. Incomplete known-failure lists could add checking on unrelated tasks. One change removed a failed command but bundled lint instructions, raising median time from 88 to 156 seconds.
This is a small behavioral study with human selection, not an autonomous self-improvement success rate. The operational lesson is to distinguish exposure to the problem, loading the retained advice, changed behavior, task outcomes, and total cost.
07 · ZipFusion: keep KV compressed until attention consumes it
ZipFusion: Fusing Lossless KV Decompression with Attention for Efficient LLM Decoding
Chen, Yuetao, Wang, Peng, Xu, Hong · 2026-10-24
Earlier lossless KV codecs often reconstruct the full cache before attention, saving transfers but leaving repeated HBM reads intact. ZipFusion co-designs BF16 exponent encoding and attention layouts. Eligible blocks retain the original bits in 12 rather than 16 bits per value; other blocks remain raw. Threads reconstruct values in registers without a full decompression buffer.
On an H100 80GB with two 8B models, decode attention improves by approximately 1.23–1.25×. At 8K input length, equal memory budgets, and each system’s maximum batch, decode throughput improves by up to 26.3% over uncompressed SGLang. Timing excludes prefill and initial compression; newly generated KV remains uncompressed. This is not an end-to-end serving speedup.
Read the interaction among encoding, operand layout, and cross-layer memory reclamation, rather than compression ratio alone.
Top to bottom: eligible blocks shrink, attention reconstructs in registers, and freed chunks can move between layers.
08 · ToolContractGate: a policy decision still needs enforcement
ToolContractGate: Versioned Trusted Contracts for Drift-Safe Agent Tool Invocation
Gupta, Sachin · 2026-10-22
ToolContractGate combines an approved OpenAPI artifact digest with version, scope, schema, and behavioral declarations. It verifies SHA-256 at startup and applies ten invocation checks, returning ALLOW, WARN, REVIEW, or BLOCK. The addition over schema validation is explicit binding to an approved artifact and policy.
All 35 unsafe cases in a 45-case synthetic benchmark produce non-ALLOW decisions: 29 BLOCK, three REVIEW, and three WARN. The 900 records repeat these cases across configurations, rather than representing independent production workloads.
The experimental endpoint returns decisions; the host must enforce blocking and review. Post-startup file changes require reloading, and a backend can change behavior while preserving its name and schema. This is an inspectable contract-evaluation prototype, not evidence of complete production drift prevention.
09 · InferScale: prediction buys time for cold starts
InferScale: Kubernetes-Native Predictive Autoscaling for SLO-Aware AI Inference
Huang, Wen · 2026-10-25
InferScale combines short-horizon prediction, SLO states, spike handling, and tenant priority at Kubernetes’ replica-control layer. Every five seconds it reads Prometheus signals; capacity estimates use healthy observations so overload does not masquerade as normal per-replica capacity. It leaves the scheduler unchanged.
On AKS with CPU-based ResNet-50 inference and a roughly 21-second cold start, three ramp-burst runs improve SLO attainment from external-metric HPA’s 47.8%±1.5% to 65.3%±9.0%. Average replica use also rises from 2.1 to 5.5. A separate controlled multi-tenant experiment protects Gold more consistently, sometimes worsening lower-priority latency.
The useful evaluation dimensions are startup delay, resource use, and who bears contention. These are not GPU or LLM measurements; prefill/decode behavior, KV pressure, and dynamic batches require a different capacity model.