Near-Infrared Optical Constants and Guided-Mode Benchmarkng of High-Index MoSe2 for Nanophotonics

The integration density of photonic integrated circuits is fundamentally limited by evanescent field overlap and subsequent inter-channel crosstalk. Layered transition metal dichalcogenides (TMDCs) bypass these confinement constraints through intrinsic optical birefringence and high refractive indices. Here, we report the near-infrared optical constants and waveguide dispersion of molybdenum diselenide (MoSe2). Ellipsometry performed on centimeter-scale crystals yields an in-plane refractive index of 4.1–4.7 over 1000–2000 nm, with an extinction coefficient close to the sensitivity limit of the fit away from strong excitonic resonances. To validate the anisotropic dielectric tensor at the device scale, scattering-type scanning near-field optical microscopy (s-SNOM) was utilized to map the propagation of transverse-magnetic modes in 235-nm-thick exfoliated flakes. Spatial Fourier analysis of the edge-scattered near-field interference yields effective mode indices that precisely match the modeled dispersion. Using the verified dielectric tensor, finite-element simulations demonstrate that single-mode MoSe2 waveguides optically outperform equivalent tungsten disulfide (WS2) benchmarks. The enhanced evanescent field suppression in the claddings of MoSe2 waveguide increases the coupling length by a factor of 3.5, reducing the required routing pitch and enabling a 12.5% direct increase in on-chip integration density. The results identify MoSe2 as a high-index anisotropic platform for compact waveguiding in the near-infrared.