Although working memory (WM) is crucial for intellectual abilities, not much is known about its brain underpinnings, especially the structural connectivity. We used diffusion tensor imaging (DTI) to look across the whole brain for the white matter integrity correlates of the individual differences in the reading span (verbal WM capacity during reading) in healthy adults. Right-handed healthy native Russian speakers (N = 67) underwent DTI on a 3T Philips Ingenia scanner. Verbal WM was assessed with the Daneman-Carpenter reading span test (Russian version). Fractional anisotropy maps from each participant were entered into the group tract-based spatial statistics analysis with the reading span as a covariate; the results were TFCE-corrected. After taking into account effects of age, sex, education and handedness, reading span positively correlated with the white matter integrity in multiple sites: the body, the genu and the splenium of corpus callosum; bilateral corona radiata (anterior, posterior, and superior); bilateral superior longitudinal fasciculus; several tracts in the right hemisphere only, including the internal and external capsule; bilateral superior parietal and frontal white matter. Although the left hemisphere is central for verbal processing, we revealed the important role of the right hemisphere white matter for the verbal WM capacity. Our finding indicates that larger verbal working memory span may originate from additional processing resources of the right hemisphere.