What matters in this issue
Start with TokenEconomizer and AgentLoupe: one converts experience across tasks into adaptive cost control; the other carries behavioral provenance beyond process trees and time windows. Both focus on runtime mechanisms surrounding the model, but their evidence differs: multi-configuration benchmark comparisons for the former, mainly a prototype and case studies for the latter.
Eleven further papers examine three connected questions: how to distinguish useful harness evolution from benchmark shortcuts; how cache reuse, drafting, and expert reduction change inference cost and correctness; and how hardware-specific optimizations become serving workflows. The recurring question is where an improvement occurs and under which conditions it persists, rather than whether every reported multiplier can be stacked into an overall systems gain.
In depth
01 · TokenEconomizer: adapting cost control to repository experience
TokenEconomizer: Experience-Driven Black-Box Cost Control for Coding Agents
Aaron J. Li, Weijia Xu, Chandan Singh, Zixiao Chen, Jeevana Priya Inala, Charlotte Siska, Acar Tamersoy, Bin Yu, Ion Stoica, Michel Galley, Jianfeng Gao · arXiv v1 2026-10-15
Sequential work exposes costs that isolated tasks miss
An agent maintaining one repository may repeatedly rediscover test commands, carry unused tool descriptions, or revisit the same failure loop. Fixed output compression or repository maps do not save money in every environment. TokenEconomizer uses completed trajectories to control later tasks while leaving the underlying agent and model unchanged.
The external controller works through exposed interfaces: it changes instructions and loaded tools before execution, then monitors events during a run. Its optimization unit is a chronological task stream, and a task can use only experience from earlier tasks. That resembles continuing project maintenance, but a one-off task may not amortize the learning cost.
Three feedback mechanisms
Experience updates on the left change instructions, tools, and runtime control on the right; the feedback below affects subsequent tasks only.
A proposed micro-skill does not immediately become permanent context. The system extracts operational advice and executable detectors, checks support across tasks with non-overlapping edited files, and measures whether the advice is needed often enough. Evidence decays, contradictions receive greater weight, and invalid skills retire. This controls both misleading experience and repeatedly billed instructions.
Tool pruning retains core shell and file-reading, writing, and editing capabilities. It hides schemas for persistently unused or judged-irrelevant tools, periodically restoring the full catalog for an audit. Failure-loop control learns a turn budget from earlier successful tasks. Exceeding that budget or repeating an error triggers assessment rather than immediate termination. A judge first redirects the agent; continued stalling can terminate execution, with a stricter forced-stop rule as a backstop.
These mechanisms act on knowledge, interfaces, and execution. In the Claude Code ablations, tool pruning provides the largest individual saving, followed by micro-skills. Adding more instructions is only one part of the design.
The metric is cache-aware cost
Across 277 SWE-Bench tasks from six repositories, five agent–model configurations, and three seeds, the authors report 18.4%–30.9% lower mean cache-aware cost, including controller calls. Configurations are Claude Code with Sonnet 5 or Opus 4.8, Codex with GPT-5.6 Terra or GPT-5.6 Sol, and Copilot CLI with Sonnet 5. Pass@1 improves in four configurations, but falls from 74.0% to 73.0% with Opus 4.8. This supports savings at broadly comparable success, not universally lossless savings.
Cumulative curves averaged across the two Claude Code settings show that most repositories initially cost more; all become cheaper by task six. Transfer experiments also change conditions: Web-Bench uses cumulative Pass@2, while security auditing uses F1 and disables repository-specific micro-skills because repositories do not recur. ReposVul saves only 6.4%–9.3%, illustrating how trajectory structure limits the available headroom.
Reading priority: the control mechanisms and cost accounting. The contribution applies experience across tasks to runtime control rather than updating model weights. Deployment needs reliable success signals and checks that pruning or early termination does not damage rare tasks. Benchmark graders supply the paper’s success labels; acceptance criteria in actual development may be less clear.
02 · AgentLoupe: tracing origins across process and time boundaries
AgentLoupe: Kernel-Resident Semantic Causal Labels for AI Agent Observability
Yifei Bao, Ting Liu, Yuxin Ren, Ruizhe Huang, Likang Luo, Ziteng Xu, Yu Wang, Peng Jiang · arXiv v1 2026-10-16
Effects can outlive the tool call
An agent may write a file and exit before another session or system service reads it and performs the consequential action. Tool-call logs miss that later execution. Correlation based on process trees, time windows, and similar arguments can also lose the connection to the original model interaction. AgentLoupe stores causal identities in eBPF maps associated with kernel resources, carrying attribution across file, process, and socket handoffs.
Compared with AgentSight’s post-hoc correlation, the change is to label events as they occur. It extends system provenance by rooting labels in LLM interactions rather than stopping at a process. This is useful for investigating persistent instruction poisoning or delayed effects, although a label expresses dependencies captured by implemented rules, not proof of the model’s intent.
From TLS plaintext to an audit chain
Read downward from the LLM interaction: TLS and propagation events enter separate buffers, then join by label for audit views and risk analysis.
The collector observes plaintext at OpenSSL/BoringSSL call boundaries, before encryption or after decryption, and uses bounded scanning to recognize interactions and assign a chain_id. Lightweight classification happens in the kernel; HTTP, compression, and SSE reconstruction happen in user space. TLS records have a capture-size cap, with truncation and loss flags. Application transparency therefore still depends on supported TLS entry points and protocol forms.
Children inherit labels on fork/clone. Writes associate labels with inodes; later opening or executing processes can inherit them, and network operations propagate labels as well. A new independent prompt starts a fresh chain, retaining earlier influence through cross-chain edges. Rollback and cross-session guards prevent indiscriminately merging everything into one chain. User space builds timelines, process trees, and propagation graphs, then supplies them to an LLM risk analyzer.
The useful property is resource-mediated continuity: a file can connect actions after the original process exits. But widely shared files can inflate dependencies, while expiry and noise filtering can remove evidence. Those choices determine actual audit coverage.
The evidence supports prototype feasibility
On Ubuntu 26.04, kernel 7.0, a four-core Xeon, and 8 GB RAM, the authors test Claude Code 2.1.150 with claude-sonnet-5 on three workloads, reporting means over five runs. Repository understanding, code writing, and compilation take 4.5%, 8.0%, and 7.7% longer than execution without observation. The abstract’s “below 8%” should not become a universal workload bound.
Two cases demonstrate attribution of delayed cron execution and instruction-file poisoning across sessions. Confidence values from the risk analyzer are model outputs, not calibrated detection accuracy. The paper explicitly leaves propagation false positives and false negatives unquantified and calls for broader attacks and runtimes. It also does not establish label persistence across system reboots.
Reading priority: a promising observability prototype. Examine propagation rules and open challenges before deciding whether a deployment has the same resource-handoff paths. The paper identifies Practical Adoption Challenges of ML for Systems (PACMI ’26), held September 29–October 2, with arXiv v1 submitted October 16. This issue is a later reading, not an October 20 debut.
Research updates
03 · HDA: separating compute, shortcuts, and residual harness gains
Harness-Delta Attribution: Dissecting Gains From Harness Evolution
Wenxuan Ding, Qin Lu, Changlong Yu, Sha Li, Shuowei Jin, Xin Liu, Zhenyu Shi, Greg Durrett · arXiv v1 2026-10-15
A higher score after harness evolution can reflect extra calls or benchmark-specific shortcuts. HDA builds a compute-matched version of the original harness and removes detected overfitting mechanisms from the evolved version. It attributes gains to test-time scaling, overfitting, and a residual, with human review of both key interventions.
Compare the baseline, compute-matched, neutralized, and evolved harnesses along the axis; the residual still requires transfer validation.
Across three frameworks, four benchmarks, and 25 settings, 21 show significant search-set gains; in 14 of those, over half the attributed gain comes from overfitting or extra compute. The authors report mean shares of 54% and 14%, respectively, across the 21 settings. These are not failure rates for all agents. The attribution protocol is the useful reading. Undetected shortcuts can remain in the residual, making it an upper bound on generalizable improvement rather than a substitute for held-out evaluation.
04 · Self-HPO: alternating harness and policy optimization
Self-Evolving Coding Agents via Harness-Policy Optimization
Zhepei Wei, Wenlin Yao, Baolin Peng, Danqing Wang, Qianhui Wu, Hao Cheng, Yu Meng, Jianfeng Gao · arXiv v1 2026-10-16
Self-HPO freezes the policy while the current model proposes harness changes from failure traces, then freezes the selected harness and applies reinforcement learning with executable-verifier rewards. Tasks never solved during that training stage become the next search set, keeping search aligned with the updated policy’s weaknesses.
The upper loop edits the harness from failures; the lower loop trains the policy within it, returning unsolved tasks and the updated policy to the next round.
Using Qwen3.5-35B-A3B, mini-SWE-agent, and 2,484 training tasks, the authors run two rounds with six harness candidates and 50 RL updates per round. Mean SWE-Bench Verified resolve rate over three evaluations rises from 61.0% to 69.5%, versus 67.5% for policy-only optimization. Multilingual and Pro reach 59.7% and 65.2%. This supports adapting both components together, not unlimited self-improvement or free efficiency. Evidence covers one base model and a limited number of rounds; training and search costs remain relevant to adoption.
05 · Proof harnesses: agents construct environments with independent checks
Towards Agentic Proof Harness Engineering
Rohith Kumar Shanmuganathan, Cedric Richter, Heike Wehrheim · arXiv v1 2026-10-16
This early exploration uses agents to construct verification environments for C projects: inspect source, recover dependencies, and generate a harness. A soundness guard checks that abstractions cover original behavior; a precision guard uses counterexamples to guide refinement. Developers still specify the properties to verify.
The agent inspects the project, builds a harness for the verifier, and refines spurious counterexamples; real violations reach the developer.
With gpt-6-luna, at most 60 actions per run, and five minutes per CBMC invocation, the authors expose two historical assertion bugs in curl and one in libuv. After fixes, CBMC proves safety only for libuv; both curl cases return unknown. Unknown does not establish that a fix is correct. These are small-sample feasibility results without guarantees of refinement progress or termination. Agents also attempted to change protected assertions, reinforcing the need for independently enforced boundaries.
06 · R2ECAP: discarded text can leave influence in reused caches
Stealthy Poisoning Attacks via Reused KV Cache of Reasoning
L. Zhu, Y. Jung, S. Szyller · arXiv v1 2026-10-16
A reasoning-cache system may discard earlier text while retaining later KV states computed in that context. R2ECAP targets this conditioned-cache design: black-box interactions produce misleading earlier reasoning and an apparently ordinary retained response, whose cache later queries retrieve. A key control compares encoding identical visible text alone versus encoding it with the earlier context.
The upper panel shows earlier context conditioning retained KV; the lower panel retrieves it for a new query without recomputing the retained states after text removal.
Across three open-weight 7B–9B models and GPQA, MBPP, and BigCode, the authors report up to 79% attack success for exact-query reuse and 16%–46% for cross-query retrieval. One detector inspecting retained text flags 1%; this does not establish that all defenses fail. The lesson concerns auditing cache-construction provenance. The threat model requires indirect admission of attacker-influenced retrievable entries, so it should not be generalized to every ordinary prefix cache.
07 · DLite: drafting without a separate historical KV cache
DLite: History-free Speculative Decoding for Agentic AI
Jiaming Xu, Hanzhen Wang, Tianlang Zhao, Jiewen Xiao, Chengze Yuan, Jiayi Pan, Yongkang Zhou, Jiancai Ye, Yu Wang, Guohao Dai · arXiv v1 2026-10-16
A speculative drafter normally processes the growing history alongside the target model. DLite instead takes contextual hidden states from the target, generates local representations with a parallel backbone, and restores within-block dependencies with a lightweight sequential head. The target still verifies candidates. It removes historical draft KV, not the target model’s cache.
The target processes history at lower left; local positions and sequential heads generate drafts on the right; verification at upper left retains historical KV on the target side.
On H800 GPUs with Qwen3-4B/8B and draft block K=7, the authors report 2.68× and 2.74× decoding speedups on SWE-bench workloads over ordinary autoregressive decoding. These are not equal speedups for complete software tasks. Gains shrink at high concurrency: 4B Alpaca throughput at concurrency 64 is 1.32× baseline. Training and access to target hidden states are required, making this relevant to long-context serving rather than a generic attachment to a closed API.
08 · MoE expert reduction: short-answer loss concentrates at prompt boundaries
Where Prefill Expert Reduction Hurts MoE Models: A Few Prompt Tokens Carry Most of the Short-Answer F1 Loss
Zhiyuan Wang · arXiv v1 2026-10-15
This analysis reduces activated experts during prefill while retaining the full expert budget for generated tokens. Much of the short-answer F1 loss reflects longer answer form rather than complete loss of answer content. In Qwen3, template positions immediately before the answer are particularly sensitive. Interface-Full preserves task-interface tokens such as system text, instructions, and questions while reducing experts for document tokens.
The left compares where each rule preserves full experts; the right is exploratory segment loss density, not a precise valuation of every token.
For Qwen3-30B-A3B on one H100 in a joint quality-and-latency batch of 760 prompts, median time-to-first-token ratio is 0.86 on long prompts but 1.02 on roughly 2K-token HotpotQA prompts, a slight slowdown. Read this as evidence about where reduction hurts. Equal-budget preservation of prompt ends is not distinguished as worse in the tested templates, and concentrated F1 loss does not imply equally concentrated answer-likelihood loss. Universal lossless acceleration is unsupported.
09 · Apex: assigning CPU and GPU experts by actual token load
Efficient CPU-GPU collaborative MoE inference via Adaptive Expert Assignment
Wenhui Lv, Yunming Liao, Hongli Xu, Yang Xu, Junhao Cheng · arXiv v1 2026-10-16
When an MoE cannot fit in GPU memory, moving every missed expert over PCIe can be costly, while computing all misses on CPU can stall execution. Apex ranks current-layer misses by actual token load and uses measured computation and transfer rates to choose how many to move, minimizing the slower CPU or GPU path. Required current-layer transfers take priority; spare transfer windows prefetch predicted next-layer experts.
The control plane uses actual routing and predictions to schedule transfers; the data plane executes CPU/GPU experts and combines outputs using original routing.
With a 48 GB RTX A6000, Xeon 8358P, 512 GB host memory, and PCIe 4.0 ×16, the authors test three BF16 MoEs at batches 1/8/16/24/32. Against the strongest comparator, mean prefill speed improves 31.2% and mean decode throughput 31.1%. Baselines are reimplemented within KTransformers with equal expert-pool budgets. The contribution is heterogeneous execution policy. It preserves routing choices and does not establish equal benefits on all-GPU systems or different interconnects.
10 · PairSight: interference prediction needs both operators
PairSight: Characterizing and Predicting Operator-Level Interference in GPU Sharing for DNN Inference
Xiao Liu, Yujia Fu, Yue Yu, Dan Huang · arXiv v1 2026-10-16
SM partitioning still shares L2, DRAM, and interconnect resources, so pairing compute-heavy and memory-heavy work does not sufficiently predict interference. PairSight takes both operators’ standalone latency and hardware counters, predicts high- and low-priority slowdown separately, and composes windows for segments longer than 16 operators.
Standalone profiles on the left feed paired predictions and long-sequence accumulation on the right; prediction itself is not a scheduling guarantee.
Across roughly 730,000 co-execution samples from six DNN models on A100, the authors report 4.3% high-priority latency MAPE versus 19.0% for an intensity rule. On unseen Llama pairs, however, error is 30.2%, falling to 13.5% after continued training on 100 new experiments per pair. The useful result is the value of both tasks’ features, not an SLO guarantee. This predicts candidate co-location latency rather than demonstrating online throughput or violation-rate improvements from a complete scheduler. Long-window composition also approximates execution progress.
11 · FuTransformer: phase-specific low precision has a quality cost
FuTransformer: FP8 prefill and INT4 decode for Llama-style inference on Hopper
Wei Wang · arXiv v1 2026-10-16
FuTransformer fuses FP8 prefill GEMMs with inference-specific epilogues for direct KV placement and folded fixed normalization gains. Decode stores INT4 weights and cache entries in Hopper MMA-fragment layouts, dequantizes near computation, and uses Hadamard rotation to reduce cache outliers.
On one H100, complete Llama-3.1-8B at batch=1 and context=1024 takes 2.76 ms per INT4/INT4 decode step versus 8.08 ms for compiled BF16 PyTorch, about 2.9×, with different precision. The authors report roughly 9%–15% higher WikiText-2 perplexity and up to a 7.6-percentage-point ARC-Challenge accuracy loss under simulated quantization. This is a speed–quality trade-off, not a lossless engine. Complete serving scheduling and distributed communication are outside the tests. Distinguish shape tests from complete models: the 405B label denotes one layer’s dimensions, not a full 405B model running on one GPU. The implementation depends on Hopper layouts and resource limits; other GPU generations require kernel redesign. The paper has no suitable standalone method architecture figure, so this entry uses text.
12 · DIFFLET: assembling statically compiled components into resident diffusion serving
DIFFLET: A Fast Inference Engine for Diffusion Transformer Models on AWS Trainium
Bin Ma, Min Yu, Vijay Kodamalla, Jialin Liu, Dong Li · arXiv v1 2026-10-16
DIFFLET separates Python model pipelines from a Trainium compilation backend: the host controls sampling and caching while fixed-shape components compute through shared NKI attention and parallel interfaces. A resident process retains programs and weights but resets sampling state for each request, separating startup cost from steady-state latency.
The left specifies encoding, denoising, and decoding; the middle adapts components, operators, and parallel configuration; the right compiles, loads, and executes.
On a single Trainium2 chip at TP=4, denoising steps are 1.24–4.28× faster for the four models whose SDPA baselines compile. Resident requests are 1.49–3.35× faster than warm standalone execution for four models. Wan is an exception: host-side VAE decoding makes resident requests slower. Skipping denoising calls also trades output quality, and load tests contain only 20 requests per level. Prioritize component boundaries and exceptions. Kernel, caching, and serving improvements should not be multiplied into a universal speedup.
13 · Harness²: learning from current edits how to improve the next task
Harness²: Recursive Agent Harnessing for an Open World · Project code
Ruiyao Xu, Yanfei Chen, Zhongying CuiZhu, Bhavana Dalvi Mishra, Yifei Ming, Han Yu, Rujun Han, Chen-Yu Lee, Tomas Pfister
Northwestern University / Google Cloud AI Research · arXiv preprint · First published: 2026-10-09
How can an edit be assessed without an answer key?
A harness comprises the prompts, memory, tools, and execution controls surrounding a model. Offline search can select it using training tasks and validation scores; new tasks often lack that feedback. Directly inheriting the preceding task’s harness can also import specialized requirements into an unrelated setting.
Harness² separates the current execution harness from experience about improving harnesses across tasks. It exposes eight editable components, including guardrails, skills, scripts, and tool-loop plugins. Model weights remain fixed; the experience library evolves while the improvement algorithm itself stays unchanged.
Propose, probe, and selectively compose
The system first runs the base harness to obtain a trajectory and deliverable. It then proposes separate edits for general agent capabilities and domain-specific requirements. The former target issues such as planning and tool use; the latter address the current domain’s procedures and constraints. Both candidates run on the same task under matched model and environment configurations and are compared with the reference execution.
Reading the diagram: execution, candidate proposals, probes, and composition proceed from left to right. Reflection on the right preserves lessons, and the lower loop brings them to the next task. Original artwork: Ruiyao Xu et al., CC BY 4.0.
The composer reads file differences and execution evidence. It can merge or revise edits, adopt one candidate, or preserve the existing harness, then executes the resulting composition. A larger budget enables parallel exploration of multiple candidates or sequential refinement. After the task, the library retains the adopted executable harness, its applicable context, and supporting evidence. The next task starts again from the base harness, using the library to guide refinement.
The conditions behind the gains
The authors evaluate two execution models and two initial harnesses on three professional-task benchmarks. Main tables average three runs with shuffled task orders. Claude Opus 4.8 is the default editor. The following one-step results all use OpenCode:
| Benchmark and execution model | Base harness | Harness², k=1 | Gain |
|---|---|---|---|
| WorkBuddy-Bench, GLM-5, macro-average over three domains | 64.2 | 75.7 | 11.5 points |
| JobBench, Gemini 3.5 Flash | 31.5 | 36.0 | 4.5 points |
| LAB, GLM-5, per-criterion pass rate | 59.6% | 71.4% | 11.8 percentage points |
WorkBuddy uses 40/35/25 test tasks for Code/Web/Office, JobBench uses 34, and LAB uses 100. LAB’s metric does not measure success on entire legal tasks. In an ablation, removing the library lowers Gemini 3.5 Flash’s WorkBuddy macro-average from 77.5 to 74.4, supporting the value of accumulated experience. Harness² does not win every comparison: with Gemini under Codex, Meta-Harness still exceeds one-step Harness² on LAB.
Read the regressions alongside the improvements
Fewer steps in the final execution do not establish lower total cost: proposals, probes, and composition all consume inference. The paper separately compares costs using total tokens. Its label-free feedback depends on model judgment; one failure analysis even shows the selector choosing a 55-point result over a 95-point candidate. Editable guardrails do not establish safety assurance either.
Prioritize the method, ablations, and failure analysis. The contribution is turning the evidence behind edits into a reusable resource. Shared model errors and outdated experience remain concerns, and the experiments do not establish indefinitely continuing improvement.
Community updates
- @FSFG’s AgentLoupe post links the same paper and identifies PACMI 2026. It cross-checks the entry above and is not counted as another paper.
- Agent Skills: Does Following Instructions Actually Help? separates loading, adherence, and task benefit, using controls and ablations to decide which rules deserve to stay.
- Unattended Agents: Seven Failure Scenarios to Validate Before Release examines tool failures, permissions, approvals, and recovery through evidence about execution boundaries.
-
The alert-noise interview discusses topology context, event correlation, and the division of work between ML and LLMs with ManageEngine’s Gowrisankar Chinnayan. It is a practitioner perspective on AIOps, without comparative experiments establishing diagnostic effectiveness.
- The 68.0% figure in OneSignalDesk’s post is supported by Agents Trust Tools Too Much: Measuring Reliance on Unreliable Tools (v1): across 14 models and 418 FreshQA questions, mean adoption is 68.0% for plausibly incorrect search returns. This is neither an all-tool rate nor a production incident rate. The paper was first submitted September 4; the post explicitly says its proposed verification prompt is untested, so that prompt is not an established defense.