Submitted:
18 November 2026
Posted:
19 November 2026
You are already at the latest version
Abstract
As test-time scaling (TTS) emerges as a new scaling axis, a model's sample diversity (pass@\(k\) at large \(k\)) has become as important as its single-sample accuracy (pass@1) in the many applications that combine a verifier with multiple reasoning traces. Reinforcement learning with verifiable rewards (RLVR), however, improves pass@1 while narrowing this diversity (Yue et al., 2025), which makes test-time scaling less sample-efficient. We apply WiSE-FT-style weight interpolation (Wortsman et al., 2022), \(W(\alpha) = W_{\mathrm{base}} + \alpha(W_{\mathrm{rl}} - W_{\mathrm{base}})\), as a single training-free knob that trades pass@1 for pass@\(k\). Under the experimental setup of Yue et al. (2025) (Qwen2.5-7B and SimpleRL-Zoo-7B on six math benchmarks), \(\alpha = 0.25\) recovers the base model's large-\(k\) coverage (pass@128 of 70.4 vs. 70.3 for the base model and 63.8 for RL, averaged over the two non-saturated benchmarks) while keeping 64–80% of the RL pass@1 gain, and it reaches the RL model's pass@128 with 2.3–7.1\(\times\) fewer samples. \(\alpha = 0.5\) strikes a balance (pass@1 \(-1.2\) and large-\(k\) pass@\(k\) \(+1.6\) points relative to RL), and \(\alpha = 0.75\) is nearly a free lunch (RL-level pass@1 with OlympiadBench pass@128 \(+2.5\)). The coverage reached at \(\alpha = 0.25\) exceeds what the RL model reaches at any sampling temperature reported for this pair.
Keywords:
reinforcement learning with verifiable rewards
; weight interpolation
; model merging
; pass@k
; test-time scaling
; sample diversity
; large language models
; mathematical reasoning
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.