This study presents an innovative approach named Optimal Adaptive Continuous Barrier Function Terminal Sliding Mode Control (OACBF-TSMC) approach with a novel switching manifold, to control mathematical model of nonlinear systems under model uncertainties and external disturbances. The proposed OACBF-TSMC approach combines the advantages of adaptive control, continuous barrier functions, and terminal sliding mode control to achieve robust stability of the mathematical model of nonlinear systems, and by using a genetic algorithm that tries to optimize the controller parameters as much as possible so that the proposed controller is the most to have efficiency and performance. The proposed method is designed to address the challenges of systems and provide superior performance in terms of stabilization accuracy, disturbance rejection, and robustness against uncertainties. However, the critical challenge facing the proposed controller is the complete elimination of the chattering phenomenon, which is the most important problem of ordinary sliding mode controllers. The simulation results show that the system trajectories in the proposed approach have converged to the origin with minimal overshoot and under-shoot in a finite time; In addition, the chattering problem in the control inputs has been well eliminated, which makes the proposed method suitable for various systems, including chaotic systems, energy sources, and distributed production systems. Furthermore, the OACBF-TSMC approach has the potential to enhance the stability and reliability of control systems, contributing to the integration of the system states.