Prioritize ByteSplat and Coda. ByteSplat, with Minyi Guo among its authors, reduces both off-chip accesses and inter-GPU transfers in 3D scene training. Coda brings coding-agent KV restoration costs and within-batch context differences into serving decisions. Both explain when local optimization improves the complete execution path, while their peak results remain workload- and configuration-dependent.
The other four papers address different layers: WakeKV makes cache demotion reversible; ServeTwin preserves queue and KV feedback in simulation; VenusRL schedules around trajectory-group completion; and BitIR compares the consequences of semantically aligned faults across GPU backends. These are not a single integrated solution. Distinguish hardware measurements, trace simulation, and fault-model experiments when choosing what to read next.
01 · ByteSplat: Optimize Data Movement Together with Scene Representation
ByteSplat: Efficient Distributed 3D Gaussian Splatting Training via Intra- and Inter-GPU communication reduction
Shuo Wu, He Zhu, Han Zhao, Xiaohui Zhang, Yaqian Zhao, Hui Wei, Ruyang Li, Hongzhi Shi, Lihua Lu, Jingwen Leng, Yu Feng, Minyi Guo · 2026-10-02 (UTC) · arXiv v1 · With Minyi Guo
Less movement, with a different working set
3D Gaussian Splatting represents scenes with many Gaussians. In distributed training, the GPUs owning Gaussian parameters send attributes to GPUs responsible for image regions; those GPUs render, compute loss, and return partial gradients. Distribution expands memory capacity, but local DRAM and inter-GPU links can both become bottlenecks. ByteSplat addresses both paths, connecting kernel design with scene representation.
The conventional pipeline separates forward rasterization, loss computation, and backward rasterization. Intermediate pixels, gradients, and Gaussian attributes repeatedly leave and re-enter on-chip storage. ByteSplat fuses the three stages to keep data on chip. This requires tile-local loss evaluation: ordinary SSIM crosses tile boundaries, so the paper uses S3IM within each tile to remove that dependency. The design therefore changes the training objective as well as kernel execution; it is more than an equivalent kernel merger.
Reading the diagram: the three stages on the left exchange intermediates through DRAM; on the right, pixels, loss, and Gaussian state pass directly through fused computation. Final gradients still leave the chip.
Fusion also extends shared-memory lifetimes. A tile with too many intersecting Gaussians may not fit, and large allocations can reduce concurrent thread blocks even when they do fit. The system first performs conventional training and densification, removes Gaussians that never become a pixel’s dominant contributor, and fine-tunes. Hardware-aware pruning then targets overloaded tiles using the destination GPU’s shared-memory budget. Tiles exceeding that budget retain the conventional path, while training lets the remaining representation compensate for removed primitives.
The inter-GPU optimization exploits a separate fact: a Gaussian selected as intersecting an image region need not actually contribute to a pixel. The system compacts and sends only nonzero partial gradients, then decodes and aggregates them on their owner GPUs. It omits records only when every component is exactly zero. This should be distinguished from Gaussian pruning, which can affect reconstruction quality.
Pruning makes the representation depend on the hardware budget. Moving to another platform therefore requires checking both reconstruction quality and fusion coverage. Porting only the fused kernel while retaining the original representation may not deliver the full system’s gains.
What the 6.1× result measures
Across six datasets against Grendel-GS, the authors report average and peak per-iteration speedups of 3.4× and 6.1× on eight A6000 GPUs, and 2.7× and 3.6× on eight A100 GPUs. The full system reduces backward inter-GPU traffic by 65.8% on average: zero elision contributes 46.2% and pruning another 19.6%, both relative to original traffic. The largest PSNR reduction is 0.1 dB and the largest SSIM reduction is 0.01; quality is not identical in every case.
The evidence supports joint optimization of movement and working-set structure. Its key boundary is that per-iteration latency is measured using the pruned, fine-tuned checkpoints. Initial training takes about 200,000 iterations on average, followed by 30,000 fine-tuning iterations after pruning. The 6.1× peak therefore cannot directly describe total time from scratch to equivalent quality. Gains also depend on scene density, shared memory, and interconnects. Prioritize the fused dataflow, hardware-aware training, and ablation sections: they explain which improvements might transfer to another GPU workload better than the peak figure alone.
02 · Coda: Admission Decisions That Account for Restoration Cost and Context Length
Coda: Exploiting Admission Flexibility for Coding-Agent Serving
Youhe Jiang, Fangcheng Fu, Binhang Yuan, Krishna Malladi, Ehsan K. Ardestani, Zhan Shu, Adnan Aziz, Yi Xu · 2026-10-02 (UTC) · arXiv v1
A tool return does not guarantee efficient execution
Coding agents alternate between model calls and tools. A completed tool call makes a continuation logically ready, but its historical KV may still reside on the CPU. After restoration, its context may also differ sharply from those of other active requests. Coda places both costs in an admission layer: which request should be prepared next, and how should attention be organized in the current iteration?
Compared with systems primarily scheduling for cache reuse and load, it adds preparation cost and execution compatibility. Preparation includes KV restoration and prefill for new tokens. Compatibility uses current materialized context lengths, rather than predicting final answer length. Both views change as sessions resume, state moves, and requests finish.
Tiered-Aging initially favors inexpensive preparation among requests that fit available HBM, then increases older requests’ competitiveness. The evaluation uses a five-second aging timescale; at 20 seconds of ready waiting, the oldest feasible request takes priority. This limits repeated bypass by the efficiency preference, subject to memory feasibility. It does not promise that every request executes or completes within 20 seconds.
Separate attention while preserving the iteration’s participants
During eligible pure-decode iterations, the system sorts requests by context length and considers a shorter-context and a longer-context cohort. It splits only when conservative predicted savings reach at least 5% and both cohorts are large enough. Each layer still performs one full-batch QKV projection and KV update. Attention runs separately for the two cohorts, followed by full-batch downstream computation. The request set is unchanged, and the complete model is not executed twice.
Reading the diagram: sort by context length, write KV once, then run attention in two cohorts. The bottom returns to full-batch projection and dense/MoE computation.
With multiple workers, routing first preserves useful KV affinity within load constraints, then considers context compatibility. Cache location and batch execution shape are distinct signals: the GPU with the most reuse may not be the best destination, while discarding useful locality merely to match lengths can also be expensive.
Throughput evidence and implementation boundaries
The authors replay TraceLab-derived workloads at three, five, and seven requests per second against CacheWise, SMetric, and vLLM. For Qwen3-Coder-30B-A3B-FP8 on four A100-80GB GPUs, output-token throughput improves by 4.2–12.1% and SLO-compliant throughput by 15.9–44.3% relative to the strongest competing baseline for each metric and trace. The latter depends on the paper’s latency-target definition; it is not an improvement in task correctness. Single-GPU ablations and larger-model experiments add evidence, but aggregated peaks across settings should not be assigned to every deployment.
An easily missed limitation is that the motivating hierarchy replay includes SSD and the design has an SSD-loading term, whereas the appendix explicitly says the current prototype and reported experiments have no SSD tier, making that term zero. A disk-cache deployment needs a separately measured loading curve; this path has not been validated by those experiments. Read the admission mechanism and appendix calibration procedure first. This is an improvement to scheduling and execution organization whose gains depend on restoration costs, context heterogeneity, and cost-model accuracy.
03 · WakeKV: Keep a Return Path for Displaced KV State
WakeKV: Reactive, Reversible KV Residency for Heads That Change Their Minds
Utkarsh Ranjan · 2026-10-02 (UTC) · arXiv v1
Attention heads change what they read, making a one-time prefill classification stale. WakeKV maintains a budgeted GPU LRU set for each head, preserves demoted full KV pages on the CPU, and fetches them when needed. Permanent loss becomes a recoverable transfer stall.
Reading the diagram: the left gives the authors’ method classification. On the right, follow demotion from GPU to CPU and retrieval after a miss; the dashed path represents irreversible deletion in the comparison policy.
On an A30 with Mistral-7B through a FlexiCache/vLLM integration, the authors report 1.34× stock FlexiCache throughput and 97.4% relative LongBench quality when reranking every step. Reranking every 16 steps reaches 2.33× with 95.0% quality. Reduced attention work also contributes, so the gains cannot all be attributed to movement policy. Simulation compares miss rates using some simplified baseline reimplementations, rather than measuring end-to-end stalls. A useful direction for reversible caching, but hardware evidence remains limited to one A30 and one model; CPU capacity and PCIe waiting still matter.
04 · ServeTwin: Let Simulated Time Reshape Serving State
ServeTwin: A Benchmark-Validated Simulator for Distributed LLM Architecture Exploration
Sungjoon Park, Changue Jung, Kyungno Joo, Mincheol Kang, Jaehyung Ahn, Sehwan Lee, Sangjoon Kim · 2026-10-02 (UTC) · arXiv v1
Adding operator timings can miss feedback from full KV pools, queues, and batch changes after request completion. ServeTwin couples a stateful serving loop, iSTAGE analytical execution traces, and ASTRA-sim. Predicted execution time advances the clock and updates admission, batching, and KV lifecycles.
Reading the diagram: serving state determines batch shape, iSTAGE generates traces, and ASTRA-sim estimates execution. The bottom feedback path returns timing to the serving loop.
The authors report 3.6% mean error across 27 InferenceX operating points and 3.7% on held-out points. Aggregate multi-turn LMBenchmark error is 9.9%, although individual metrics can deviate by more than 20%. It avoids operator-by-operator profiling on target hardware but still calibrates component-owned throughput, scheduling, and runtime costs; it is not measurement-free. Useful for architecture exploration and screening bottleneck hypotheses. Simulated bandwidth–capacity trade-offs are not promised gains on an actual chip; prioritize the calibration-transfer rules and multi-turn state validation.
05 · VenusRL: Finish the Trajectory Groups Needed by the Next Training Step
VenusRL: A Fully Disaggregated Agentic RL System with Priority Scheduling and Scalable Interaction
Mingjun Zhang, Yucheng Li, Menghao Zhang, Shuyong Zhu, Ping Zhang, Xiaohe Hu, Jun Chen, Zhixin Wang, Xutong Wang, He Liu, Yanmin Jia, Shengrong Zhu, Peng Sun, Mingjie Zhang, Liming Liu, Jinlong Hou, Yuan Cheng, Yujun Zhang · 2026-10-02 (UTC) · arXiv v1
Agent RL can use a trajectory group only once all its members finish. Maximizing GPU occupancy alone can leave many groups one trajectory short. VenusRL separates training, generation, and environment interaction, predicts remaining generation length to prioritize groups likely to finish soon, and coordinates admission, KV residency, and request migration. Environments share read-only snapshot pages keyed by template, with copy-on-write on modification.
Reading the diagram: priorities enter from the left, generation overlaps environment actions in the middle, completed groups reach training on the right, and new weights return along the bottom.
With 32 Hopper GPUs—16 for generation and 16 for training—Qwen3-4B/32B, and OpenSWE, the authors report up to 4.24× training speedup over RollFlash. That peak depends on larger batches and asynchronous settings; strict on-policy gains are smaller. Higher sandbox density also does not guarantee freedom from memory overcommitment: dynamic admission and migration handle bursts. The group-completion objective is worth following; length prediction, weight staleness, and shared-page isolation are central boundaries for transferring the design.
06 · BitIR: Compare GPU Fault Outcomes at Aligned Semantic Sites
BitIR: Cross-Architecture Fault Injection for Resilience Analysis of Heterogeneous GPU Applications
Maisy Dunlavy, Michael Papka, Zhiling Lan · 2026-10-02 (UTC) · arXiv v1
Different GPU instruction sets make fault-injection results difficult to compare directly. BitIR defines semantically aligned sites and bit indices at LLVM IR level, then compiles through each vendor’s toolchain for A100, Intel GPU Max 1550, and AMD MI250X GPUs. Platform-specific fault-free outputs support a shared classification of masked results, silent data corruption, and failures.
Reading the diagram: follow source code downward through a single-bit XOR in IR and then backend lowering. This example shows the NVIDIA path, not identical machine instructions across all three backends.
Across the retained aligned injection space, the authors report that only 10.4% of injections produce the same outcome category on all three backends. Functional portability alone therefore cannot establish equivalent resilience. This percentage applies to selected HeCBench workloads and a single-bit IR fault model, including compiler and runtime effects. It is neither a natural hardware error probability nor a vendor reliability ranking. Read the fault-equivalence definition and output classification first: they establish what this cross-architecture comparison actually compares.
A reading lead
The original @learnk8s post points to production-agent trace replay for inference evaluation, emphasizing session-level metrics and call dependencies through OpenTelemetry Trace Replay for Inference Perf. Its connection to ServeTwin is that relationships between turns shape the workload. This records a lead from the post, not a validated performance result. The post displays October 2 while the input lists October 3; the page’s display timezone was not established.