Preprint
Article

This version is not peer-reviewed.

Mix Your Base and RLVR Weights: A Training-Free Knob for Diversity

Submitted:

18 November 2026

Posted:

19 November 2026

You are already at the latest version

Abstract
As test-time scaling (TTS) emerges as a new scaling axis, a model's sample diversity (pass@\(k\) at large \(k\)) has become as important as its single-sample accuracy (pass@1) in the many applications that combine a verifier with multiple reasoning traces. Reinforcement learning with verifiable rewards (RLVR), however, improves pass@1 while narrowing this diversity (Yue et al., 2025), which makes test-time scaling less sample-efficient. We apply WiSE-FT-style weight interpolation (Wortsman et al., 2022), \(W(\alpha) = W_{\mathrm{base}} + \alpha(W_{\mathrm{rl}} - W_{\mathrm{base}})\), as a single training-free knob that trades pass@1 for pass@\(k\). Under the experimental setup of Yue et al. (2025) (Qwen2.5-7B and SimpleRL-Zoo-7B on six math benchmarks), \(\alpha = 0.25\) recovers the base model's large-\(k\) coverage (pass@128 of 70.4 vs. 70.3 for the base model and 63.8 for RL, averaged over the two non-saturated benchmarks) while keeping 64–80% of the RL pass@1 gain, and it reaches the RL model's pass@128 with 2.3–7.1\(\times\) fewer samples. \(\alpha = 0.5\) strikes a balance (pass@1 \(-1.2\) and large-\(k\) pass@\(k\) \(+1.6\) points relative to RL), and \(\alpha = 0.75\) is nearly a free lunch (RL-level pass@1 with OlympiadBench pass@128 \(+2.5\)). The coverage reached at \(\alpha = 0.25\) exceeds what the RL model reaches at any sampling temperature reported for this pair.
Keywords: 
;  ;  ;  ;  ;  ;  ;  
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.