Silicon-based photodetectors, as low-cost and environmentally friendly optical sensors, are attractive. Also, their compatibility with complementary metal-oxide-semiconductor (CMOS) technology is advantageous for the development of silicon photonics systems. However, extending optical responsivity of silicon-based photodetectors to the mid-infrared (mid-IR) wavelength range remains challenging. In developing mid-IR infrared Schottky detectors, nanoscale metals are critical. Nonetheless, one key factor is the Fermi level pinning effect at the metal/silicon interface and the presence of metal-induced gap states (MIGS). Here, we demonstrate the utilization of the passivated surface layer on semiconductor materials as an insulating material in metal-insulator-semiconductor (MIS) contacts to mitigate the Fermi level pinning effect. The removal of Fermi level pinning effectively reduces the Schottky barrier height by 12.5% to 16%. The demonstrated devices exhibit a high responsivity of up to 234 μA/W at a wavelength of 2 μm, 48.2 μA/W at 3 μm, and 1.75 μA/W at 6 μm. The corresponding detectivities at 2 and 3 μm are 1.17×10^8 cm Hz^1/2 W^-1 and 2.41×10^7 cm Hz^1/2 W^-1, respectively. The expanded sensing wavelength range contributes to the application development of future silicon photonics integration platforms.