What matters in this issue
Start with the KV residency study and SchemaShield-Lite. The former extends agent resource management from a request to an entire session: memory can remain occupied by context long after the model stops computing. The latter asks why a previously capable tool caller fails when descriptions, fields, or candidate tools change, and how much validation feedback can repair. Production traces and controlled experiments give these performance and reliability questions concrete evidence.
The other papers contribute at different levels. BaGuaLu-2 addresses fluctuating activation memory in MoE training; an optical-network demonstration connects diagnostic statements to actual data fragments; MLOps-Bench tests whether code changes satisfy new behavior while preserving existing contracts. Their maturity differs: training throughput on thousands of GPUs, a prototype demonstration, and synthesized-task scores do not establish the same kind of operational readiness.
01 · KV residency: should every live session keep its cache?
Understanding KV Residency in Production-Scale Agentic LLM Workloads
Mayank Arya, Agrim Bari, Anjaly Parayil, Yogesh Simmhan, Renee St. Amant, Victor Rühle, Saravan Rajmohan · arXiv v1 · 2026-10-16
From request speed to state lifetime
After using a tool, an agent often returns to its model with a longer context. Prefix caching avoids repeated prefill, but retained KV still occupies memory during tool execution and user pauses. The paper analyzes four Microsoft M365 production workloads covering more than 121 million model calls. Coding Agent records are primarily events; the authors distinguish them from per-call traces and exclude them from byte-valued capacity and placement analyses. Observation windows span late June to early August 2026.
In Deep Research, 89.3% of calls are internally initiated and about 80.3% of context tokens hit the prefix cache. The contribution is the joint view of sessions, context, and serving deployments: how large state becomes, how long it should remain resident, and whether the next call can reach it.
Reading the diagram: the shorter chat path appears above; the agent repeatedly invokes models and tools before returning control. Means are calls per user turn, not session depth.
How long, in what representation, and where?
To protect production-model details, the authors apply the FP16 GQA layout of Llama-2-70B uniformly: 327,680 bytes per token. Assuming every live session retains its lifetime peak from first to last call gives a Deep Research full-trace peak of 501.8 TB. This deliberately pessimistic capacity ceiling excludes weights and other memory; it is not observed physical allocation.
A one-hour idle cap lowers that ceiling to 140.8 TB, while the replayed fraction of calls requiring reconstruction rises from 4.07% to 4.53%. Long pauses retain substantial capacity but precede relatively few calls. Cutting the cap to one minute, however, raises reconstruction to 14.1%, affecting shorter pauses inside tool loops.
For constrained capacity, the paper replays 176,007 calls from the busiest hour in timestamp order, treating a session’s current KV as a variable-sized object. At 10% of that hour’s 47.0 TB no-eviction peak, a 4.7 TB budget, LRU reconstructs 62.9% of calls; clairvoyant furthest-next-use, FNU, reconstructs 31.1%. MRR demotes sessions in LRU order to int4 copies one quarter of the size, charged to the same HBM budget, and promotes them on return. Reconstruction falls to 13.6%. The benefit primarily comes from retaining more state. FNU is not a strict optimum for variable-sized objects either.
Placement is a separate constraint. Mean same-model reuse is 88.9% when calls stay on one deployment and 29.0% after a move within the same region, an observational association. Changing replay keys from sessions to session–deployment pairs raises the compulsory first-touch floor from 5.4% to 20.2%, even with unlimited memory. Capacity alone cannot make remotely stranded state accessible.
What still needs validation
Read the capacity replay and placement analysis first. MRR does not charge promotion, dequantization, or transfer latency, nor validate response quality on these production sessions. Converting int4 back to FP16 does not restore lost precision. Including quantization metadata raises reconstruction to 14.99%–19.25%. The budget is also idealized as a pool shared across replicas. These results motivate tiered retention and affinity routing, but do not demonstrate end-to-end speedups. Production traces cannot be released; AgenticTraceSim remains a planned release in the paper.
02 · SchemaShield-Lite: repairing tool calls with validation feedback
Robust Function Calling for LLM Agents Under Interface Drift
Aniruddha Chattopadhyay, Kanishka Halder, Partha Sarathi Roy · arXiv v1 · 2026-10-16
Unchanged semantics can still leave a tool caller confused
Tool updates rename parameters, leave stale examples, and add tools with overlapping descriptions. The interface text changes even when the underlying operation does not. SchemaShield-Lite studies this drift through a runtime repair layer, without retraining. Its perturbations are author-designed, not a complete empirical distribution of API evolution.
The input is user context and the changed visible tools. A model produces a call; a validator checks the tool name, required fields, and argument structure. Valid calls pass through. Invalid calls return to the model with the original call, structured errors, and tool information for constrained repair. If no tool name was supplied, a retry first resolves a target from visible candidates before repairing the selected tool. Thus “one-shot repair” can involve an additional candidate-selection call.
Reading the diagram: valid calls follow the upper pass-through branch; invalid calls take the feedback-and-repair branch. Missing names require visible-candidate resolution first, a step not separately expanded in the figure.
Feedback helps, at an additional inference cost
The main evidence uses a non-oracle protocol: the repairer is not given the gold target tool. Qwen3.5-9B is evaluated on 300 single-function, round-one DICE dialogues with mild drift and one distractor, and 200 BFCL examples equally divided among four categories with medium drift and three distractors. Temperature is 0.
| Main experiment | Original interface | Drifted | Repaired | Drift failures recovered on originally correct examples |
|---|---|---|---|---|
| DICE-300 | 75.7% | 71.7% | 81.3% | 19/22 |
| BFCL-200 | 79.0% | 58.0% | 80.5% | 33/45 |
Repaired scores exceed original scores partly because repair also fixes pre-existing baseline mistakes. The entire gain cannot be attributed to drift recovery. In separate baseline runs, naive retry reaches only 72.7% and 61.0%, compared with structured repair at 81.3% and 79.5%. These are different runs; the two BFCL repair figures should not be conflated. Main-run repair triggers are 16% and 32%, with additional tokens equal to 32.6% and 47.1% of drifted-call tokens.
In the DICE ablation, expanding the candidate set causes more damage than changing descriptions or parameter names alone. Tool-catalog growth deserves monitoring alongside schema changes. Some mechanism ablations, however, provide the target tool through an oracle protocol and should be treated as upper-bound context.
Repair depends on what validation can detect
Structurally valid but semantically wrong values can pass unchecked. No observed repair harms on the originally correct subset is not a safety guarantee. Portability is uneven: Llama-4-Scout improves only from 63.5% to 66.5% on BFCL, with omitted tool names leaving many targets unresolved. In a separate BFCL latency-instrumented rerun, a 27% trigger rate implies approximately 3.5 seconds of expected additional latency per example. That rate must not be substituted for the main table’s 32%.
This controlled reliability study merits close reading. Its most useful contribution is the relationship between validation, candidate resolution, and repair. Broader models, real changes, complex multi-tool refactors, and latency budgets remain to be tested. The layer does not authenticate tools; provenance checks for newly added tools are still necessary.
03 · BaGuaLu-2: adapting activation memory to MoE routing
BaGuaLu-2: Taming Dynamic Memory Fluctuation for Efficient MoE Model Training over 10,000 GPUs
Mingshu Zhai, Minkun Guo, Hongrui Lian, Shuangyu Li, Qi Zhang, Chen Hu, Xiawei Du, Yuyang Jin, Zan Zong, Jidong Zhai · arXiv v1 · 2026-10-16
MoE routing skew changes activation memory across microbatches. At pipeline scheduling points, BaGuaLu-2 estimates memory demand, chooses recomputation, offloading, or quantization per operator, then overlaps adjacent forward and backward microbatches to hide mitigation overhead.
Reading the diagram: pipeline state and expert imbalance feed the memory model; adaptive activation retention and overlapping execution advance the next state.
On vendor GPUs with 64 GB HBM and a 480 TFLOPS BF16 peak per device, the authors report 2.012 EFLOPs across 12,288 GPUs. Average gains across configurations are 1.14× over Megatron-Kuai and 1.13× over AdaPipe. Baselines receive MoE adaptations or component restrictions. Throughput includes machine warm-up after OOM restarts but excludes human intervention time. Dual-batch execution requires pipeline parallelism greater than two; evaluated configurations use at least four. The abstract and figure differ in maximum-scale MFU rounding, so that percentage is not used as a conclusion here. The work is relevant to large-scale training-memory scheduling; transfer depends on hardware cost models and configuration.
04 · Optical troubleshooting: grounding displayed evidence in data references
Preventing Data Hallucinations in Agentic AI for Optical Network Troubleshooting
Jeremias Dötterl, Martin Kuipers · arXiv v1 · 2026-10-16
This ECOC Demo Zone-oriented manuscript makes tools return URI data references identifying time-series windows, log lines, or JSON fragments. The agent cites these locations; a backend retrieves and renders the original telemetry rather than letting the model rewrite its values.
Reading the diagram: detectors and inspection tools provide references, the agent organizes its explanation, and the lower query-and-lookup path replaces references with actual fragments before display.
The paper describes commercial OLT/ONU equipment in Adtran labs, fiber-cut and power-loss demonstrations, and a deepagents implementation. It provides no sample count, root-cause accuracy, or numerical controlled improvement. A nonexistent reference can trigger a warning; an existing one does not establish correct causal reasoning or prove telemetry is unpoisoned. The contribution is an evidence interface and display constraint, supported at prototype-demonstration level, not elimination of every diagnostic hallucination.
05 · MLOps-Bench: turning operational ML contracts into executable tests
MLOps-Bench: Benchmarking AI Agents on Production ML Engineering in Real Repositories
Quoc Co Tran, Hitesh Laxmichand Patel, Diwakar Mahajan, Kshitij Bakliwal, Avi Sil, Katrin Kirchhoff · arXiv v1 · 2026-10-16
MLOps-Bench synthesizes tasks from real repository code and configuration, covering data, features, training, serving, monitoring, governance, and cross-stage integration. Each task has an oracle patch, a plausible buggy patch, and new-behavior and regression tests. Its 212 tasks span 31 repositories; 25 tasks support development, while reported evaluation uses 187 tasks from the remaining 29 repositories.
Reading the diagram: the left freezes repository evidence and behavioral gaps; the middle constructs requirements, correct and buggy patches, and tests; evaluated agents on the right receive only the task and workspace before hidden-test scoring.
Across six agent configurations, the authors report the best Codex/GPT-5.5 completion rate as 31.2%, versus a 68.7% test pass rate. Partial implementation remains far from satisfying the full contract. These are not 212 real production incidents: model-assisted task and test construction introduces artifacts, finite tests cannot capture all operational risks, and interactive coding systems versus base wrappers also compare workflows. This is useful evidence for operational ML evaluation, not an estimate of industrial deployment success.
Two community updates
CNCF highlights telemetry poisoning. Its October 16 post promotes a zero-trust telemetry session at KubeCon + CloudNativeCon North America 2026. The official agenda lists Chad M. Crowell and Diana Todea and covers collection-path authentication, query permissions, and tamper detection. This is a session announcement, not a published attack-success evaluation.
Push versus pull changes when preparation happens. Navaneeth Krishnan’s post revisits the October 14 vLLM article. Push prepares decode memory early and moves transfer setup to a prefill background thread. For Qwen3-32B on P5en/EFA in a 1P1D setup, decode prefix caching disabled and 128 output tokens, the authors report TTFT falling from approximately 1,744 ms to 540 ms at 8 QPS and 16K input. Short prompts at low load differ little; selecting the decoder early also sacrifices some late-routing flexibility.