Highlights
Two approaches bring evidence closer to execution. Interaction-as-a-Judge operates websites and games to detect behavior missed by static code inspection. RankLens preserves one rank’s real computation and uses communication records to reconstruct its distributed context. The former strengthens evaluation and repair evidence; the latter lowers the resource barrier to repeated diagnosis. Both require a clear account of what their evidence covers.
Several systems contributions reduce repeated work: ATTUNE guards against mistaking incorrect configurations for throughput improvements, FastEEP retains expensive state during resizing, and optical KV networking and CacheSmith optimize cache movement and selective recomputation. Tremulant and CSNB show that warm starts and local numerical checks still leave residual risks. We first examine the two lead papers, then compare the remaining mechanisms and their operating conditions.
01 · Interaction-as-a-Judge: Let the judge operate the application
Interaction-as-a-Judge
Wayne Chi, I-Hung Hsu, Li Chen, Wei-Lin Chiang, Ion Stoica, Anastasios Nikolas Angelopoulos, Ameet Talwalkar, Chris Donahue, Cho-Jui Hsieh · 2026-10-13
An application may build successfully and look plausible in code while its buttons do nothing or a game freezes during a scene transition. Interaction-as-a-Judge adds observable behavior to the evidence available to a judge, then studies human-preference prediction, requirement verification, and repair. Its most useful contribution is separating the ability to operate an application from the ability to evaluate it, rather than reporting only an end-to-end score.
From interaction traces to judgments and feedback
Inputs include the user request, candidate applications, and optionally code and static screenshots. A computer-use agent operates each application, producing screenshots, actions, and reasoning. A judge then compares the candidates or labels explicit requirements as working, broken, or unverifiable. Preference comparisons are run in both candidate orders to reduce position bias. Traces can also train a reward model or feed a generator’s repair loop.
The authors compare iterative observation and action, a single plan for the whole action sequence, and heuristic control selection. Iterative interaction adapts to changing screens but requires a model call at every step; one-shot planning costs less but cannot respond to unexpected states. The main preference experiments allow 30 tool calls per application and use GPT 5.6 Luna for computer use. This supplies behavioral evidence without guaranteeing coverage of every functional path.
Reading the diagram: preference and requirement labels are on the left; interaction produces traces in the middle; the right routes them into judges, reward models, and generation feedback.
Read each number with its judge and task
ArenaPreferences contains 6,858 application pairs across seven web-development categories. ArenaCorrectness contains 222 game versions and 412 human-verified requirements, including 323 broken and 89 working ones. The former excludes build failures and tied preferences; the latter derives requirements from complaints and repair histories. They measure different aspects of quality.
For Gemini 3 Flash, preference accuracy rises from 65.6% with code alone to 69.1% with iterative interaction; code plus a static screenshot reaches 67.6%. GPT 5.6 Luna as judge, however, shows no significant improvement. GPT 5.6 Sol’s requirement-verification score rises from 77.0% to 81.6%. This is the unweighted mean of accuracy on working and broken requirements, not overall accuracy across all labeled requirements.
In the repair experiment, GPT 5.6 Luna performs five repair iterations and the stronger GPT 5.6 Sol checks the outcomes. Under interaction-based verification, repair success is 75.5%, versus 55.4% with code-only feedback. This supports the value of interaction feedback, but the final score still depends on a model judge rather than item-by-item human acceptance.
What this adds to coding-agent evaluation
The training experiment uses Qwen-3.5-9B, DAPO, and an interaction reward model across six categories excluding games. Relative to code rewards, generated code is 35.7% shorter and build failures fall from 18.7% to 11.5%. The 6.72-percentage-point win-rate gain over the base model comes from an automated rubric judge on applications that successfully build and render; it is not an unconditional human win rate.
The interaction-strategy comparison and repair experiments deserve close reading because they show how evaluation evidence becomes an improvement signal. The limits are concrete: cost restricts experiments with stronger computer-use models, requirement discovery remains difficult, and a finite trace may never trigger a hidden fault. Interaction-based evaluation can complement code and test evidence, with benefits depending on both evidence collection and the judge’s ability to use it.
02 · RankLens: Preserve a distributed training context on one GPU
A Distributed View from a Single GPU: RankLens for Distributed Training
Yufan Wang, Yihao Zhang, Yi Liu · 2026-10-13
A distributed-training anomaly may require inspecting only one rank, yet reproducing it repeatedly can occupy an entire cluster. Converting the job to ordinary single-GPU training changes sharding, process groups, and communication paths. RankLens instead keeps real execution on the target rank and represents its peers’ effects at communication boundaries, allowing repeated debugging and local tuning on one GPU.
Preserve remote effects, execute local computation
Inputs are the original training script, parallel configuration, and execution records. The system collects traces at three levels: training state, process-group communication, and collective-library communication. During single-GPU replay it virtualizes world size, rank identity, topology, and process groups so the framework follows its original distributed control flow. It neither shrinks the model into a different training job nor replaces all local computation with predicted durations.
At each communication call, it matches records by training position, process group, operation, message size, and submission order. Path inspection needs structure alone. Wait and overlap analysis adds recorded or estimated completion times. Numerical debugging additionally restores post-communication tensors. Substituted Work objects retain submission, completion, and wait dependencies, rather than incorrectly blocking at the submission of an asynchronous operation.
Reading the diagram: real execution supplies traces above; the middle runs the target rank on one GPU; the right restores structure, timing, or tensor values for the chosen analysis and connects to debugging tools.
The separation matters because communication metadata is relatively cheap to record, whereas capturing every tensor can substantially perturb collection. Numerical replay also requires local state and inputs to correspond to the original execution. It recreates the local symptoms of recorded remote results without recomputing all remote processes. Existing GDB, debugpy, and Nsight Systems remain usable for inspection and analysis.
Where fidelity is established
On A100 PCIe 40GB GPUs with 100 Gb/s RoCEv2 between nodes, the authors compare DeepSpeed and Megatron-LM configurations spanning 8–64 GPUs against single-GPU replay. Training-tensor communication-signature counts match at 100%, with relative step-time error below 5%. The structural metric excludes some management interfaces and does not establish equivalence of every runtime behavior.
A controlled anomaly experiment reproduces communicated NaNs and their effect on parameter updates. The remote fault location, however, is specified by the injection procedure; this does not demonstrate automatic remote root-cause localization. A scheduling experiment shows a particularly useful boundary: deferring a wait reduces replay step time by 23.60%, versus 14.75% in real two-GPU execution, an overestimate of 8.85 percentage points. Resource contention between actual communication and computation is not fully reproduced.
The value is inexpensive repeated observation
Timing-trace collection adds less than 4% overhead for the evaluated 355M–1.5B models on 8–64 GPUs. Full tensor recording is substantially more expensive and should be selected according to the diagnostic task. When target timing traces are absent, historical records of similar communication can estimate durations; this does not reconstruct unknown communication outputs.
Readers working on distributed diagnosis and training tools should prioritize the three replay modes, asynchronous semantic substitution, and scheduling comparison. The first two explain why single-GPU execution remains useful; the last shows how to interpret its predictions. Local changes that break correspondence with recorded control flow exceed the replay’s evidence. Predicted improvement directions can help screen optimizations, while their eventual magnitude still requires validation in the target environment.
Research updates
03 · ATTUNE: Gate throughput tuning with independent correctness checks
ATTUNE: Agentic Throughput Tuning for LLM Training
Leo Fan, Hongyang Wang, Wenbo Chen, Jingguo Liu, Xian Li, Xuan Wang, Ruochen Wen, Yiming Yan, Hao Wu, Zheshen Wang, Benoit Dumoulin · 2026-10-13
A faster training recipe may have silently changed the training objective. ATTUNE uses one coding agent to diagnose bottlenecks and adjust configurations, with independent scripts enforcing budgets and acceptance. A configuration check rejects objective-changing arguments and known defects, followed by a comparison of candidate and baseline loss curves over 20 steps.
Against public recipes run on the same software stack, mean throughput gains across three campaigns on 128 B200s are 6.7% for Llama-3.1-405B and 33.9% for GLM-4.5. On 128 H100s, no Mixtral-8x22B improvement clears the noise band. These are gains from adapting recipes to this environment, not improvements over their authors’ best deployments.
Reading the diagram: hypothesis, execution, and diagnosis are at upper left; independent configuration and loss checks are at upper right; experience, execution controls, and budget records sit below.
The gate accepts all 133 legitimate runs and rejects 16 of 24 faulty runs, missing eight with subtle short-term loss effects. The useful design is the separation of search and acceptance; a short loss curve still cannot guarantee full-training convergence.
04 · Optical KV networking: Reserve circuits during prefill
An Optical KV Cache Network for Disaggregated LLM Serving
Seongwon Yoon, Shimeng Yu · 2026-10-13
When a long-context agent moves to another decode instance, the full historical KV state may need transfer even though prefill computes only a small addition. This paper routes KV through an optical circuit-switched network. It reserves the prefix circuit at prefill start and the new-KV circuit at the final prefill chunk, aiming to overlap mechanical reconfiguration with computation.
Reading the diagram: the packet network still carries requests on the left; the middle shows migration between turns; the measurement-driven timing example on the right illustrates overlap of circuit setup and KV transfer with prefill.
The authors drive flow-level simulations of 128/256 GPUs with Llama-3.1-70B prefill times measured on four H100s. With WDM optics, the cost model for a single-stage 256–3,072-GPU network is 0.55–0.78 times that of a 2:1 electrical packet fabric; 50 ms reconfiguration is tolerable at 60% load. Performance gains are mainly against ECMP, while strong adaptive routing performs similarly.
This is a network-and-scheduling feasibility study. Decode is modeled as a KV sink without decode computation. Cost also depends on optics, port prices, and link margin, so these results do not establish benefits for a deployed end-to-end service.
05 · FastEEP: Retain expensive runtime state when resizing expert groups
FastEEP: Fast Intra-Instance Elastic Scaling for Expert-Parallel MoE Serving
Yunfei Feng, Jingkai He, Shiqi Liu, Cheng Zhang, Dong Du, Erhu Feng · 2026-10-13
Resizing expert parallelism in MoE serving can pause service while workers, communicators, and CUDA Graphs are rebuilt. FastEEP initializes a maximum topology, preserves execution contexts and virtual addresses, and changes active membership and physical GPU-memory residency. Request ownership, expert mappings, NCCL subgroups, and NIXL reachability switch consistently at a brief quiescence point.
Reading the diagram: the three columns retain worker contexts, virtual addresses, and the parent communication topology; prepare, commit, and finalize coordinate the resource transition below.
Across six models on two nodes with 16 H20-3e GPUs, standard FastEEP completes resizing in 4.2–13.4 seconds. Speedups over vLLM Elastic EP are 4.5–11.7× for model-transition pairs where that baseline runs successfully. The baseline fails on two models, which are excluded from that comparison.
The useful mechanism is reclaiming physical GPU memory while retaining logical runtime state. Elasticity remains bounded by the maximum topology initialized at startup; growth needs available GPU memory, and larger topology changes require reconstruction. These are resizing-latency improvements, not steady-state inference-throughput multipliers.
06 · Tremulant: Warm starts also produce extreme latency tails
Tremulant: Latency in Serverless Platforms beyond the 99th Percentile
Linus Gustafsson, Trever Schirmer, David Bermbach · 2026-10-13
Reducing cold starts does not eliminate all FaaS latency tails. Tremulant reanalyzes millions of calls across seven platforms. Its primary dataset contains empty-function requests over 53 days, using client-observed time to first byte to capture platform and network-path effects; four additional public datasets test whether the phenomenon persists.
Reading the plot: the vertical axis is percentile latency divided by the median; orange p99.99 curves expose the extreme tail. The FEC panel includes all calls. The paper has no method architecture diagram; this result plot directly supports the finding.
For Deno Deploy, p95/p50 and p99/p50 are about 5.43 and 5.85, rising to 138.46 and 424.05 at p99.9 and p99.99. Reporting p99 alone misses the steep end of the distribution. Fastly does not preserve global state and labels every invocation cold, so its plotted data includes all calls rather than a clean warm-only subset.
This measurement study does not provide a general mitigation mechanism. Empty functions magnify relative latency, and the VM baseline was collected in a different period. For workflows chaining functions, the immediate lesson is to report higher percentiles and sample counts separately for cold and warm calls, rather than attributing every rare slowdown to cold starts.
07 · CSNB: Choose low-cost numerical checks by computation type
Revisiting Cauchy-Schwarz Norm Bounds for SDC Detection in LLM Inference
Anton Altenbernd, Dylan Everingham, Philipp Wiesner, Odej Kao · 2026-10-13 · SoCC ’26
CSNB uses input-vector norms to bound matrix-multiplication outputs and flags violations, reducing checking work relative to checksum-based V-ABFT. On L40S GPUs with BF16 inference, the authors inject 1, 2, 64, or 256 bit flips into linear-layer outputs using GSM8K and SQuAD v2 prompts, checking whether generated token sequences diverge from fault-free references.
Reading the plot: left and right enable and disable KV caching; colors distinguish detection and output changes. Orange denotes undetected output changes, and darker/lighter bars distinguish the two checks.
For the evaluated dense-model linear layers, CSNB retains an average 99% of V-ABFT’s detection capability while reducing verification overhead by 87% and separately measured threshold-computation overhead by 67%. For Mixtral-8x7B router gates, it retains only 55% of detection capability. The 99% is relative to the baseline, not the fraction of all faults detected; the two overhead reductions cannot be added into an end-to-end speedup.
The paper has no method architecture diagram. Its result plot shows that locally undetected errors can propagate through KV caching. Matrix multiplications inside FlashAttention are outside the protected scope, and a changed token sequence is not synonymous with semantic error. The contribution is identifying which computations suit the check, rather than guaranteeing correctness of the whole inference system.
08 · CacheSmith: Schedule for both answer quality and first-token deadlines
CacheSmith: Accurate RAG Serving under Latency SLOs with Adaptive KV Reuse
Xinyue Ma, Damin Lee, Aashaka Shah, Myeongjae Jeon · 2026-10-13
Reusing KV for independently cached RAG chunks loses cross-chunk attention, while a fixed recomputation fraction cannot adapt to queues and heterogeneous requests. CacheSmith divides work into WarmUp, Probe, and Patch: restore some complete layers, extract token-selection signals, then repair important tokens within budget. Its scheduler jointly chooses batches and budgets, and runtime monitoring can trigger replanning.
Reading the diagram: offline profiling supplies costs; the scheduler assigns a batch and budget, and the planner selects recomputation. On the right, KV loading and execution proceed while monitoring can trigger replanning.
On RTX PRO 6000 GPUs, the authors evaluate four task types with Llama-3.1-8B on one GPU and Qwen3-32B on two. In the main experiment, CacheSmith beats the retrospectively selected BestStatic in 93 of 96 task, arrival-pattern, and deadline settings, with reported quality-gated SLO-attainment gains of 2–27.5%. The metric requires both timeliness and sufficient quality: exact match for retrieval/question answering, and F1 above 0.5 for the other two tasks.
The useful contribution is coordinating recomputation with queueing. Online planning relies on offline cost and quality distributions and cannot guarantee each answer’s quality. Some baselines share the authors’ reimplemented, untuned Probe kernels, an important condition on the performance comparison.
Project update: ReplaySSM in vLLM
Official vLLM announcement · @vllm_project · 2026-10-13
vLLM announces support for Mamba2 standard decoding and a FlashInfer-backed multi-token prediction (MTP) path. ReplaySSM caches SSM inputs and updates the checkpoint only when needed, reducing recurrent-state memory traffic. For state-space-model serving, the interesting change is the implementation of state updates and memory access. The post does not provide complete benchmark conditions for this vLLM configuration, so peak speedups in the quoted post should not be treated as integration guarantees. The post has no suitable mechanism illustration.