A Deterministic Global Optimization Framework via Monte Carlo Region Integration: Rigorous Convergence, KKT Equivalence, and Multi-Objective Extension

1. Introduction

Optimization constitutes a foundational pillar across a wide spectrum of disciplines, including engineering design, scientific computing, machine learning, control theory, and operations research. Classical deterministic optimization methods—such as gradient descent, quasi-Newton schemes, trust-region frameworks, and interior-point methods—have achieved remarkable success in smooth and convex settings, where convergence properties can be rigorously established [1,2,3,4]. However, their performance deteriorates significantly when confronted with nonconvexity, multimodality, discontinuities, or nonsmooth constraint structures, which are ubiquitous in real-world applications [5,6,7].

To address these limitations, a broad class of stochastic and population-based methods has been developed, including Genetic Algorithms (GA) [8], Differential Evolution (DE) [9], Particle Swarm Optimization (PSO) [10], and Covariance Matrix Adaptation Evolution Strategy (CMA-ES) [11]. These methods exhibit strong empirical robustness and flexibility in navigating complex landscapes. Nevertheless, their stochastic nature fundamentally limits their theoretical characterization: convergence is typically asymptotic or probabilistic, and rigorous guarantees—particularly for constrained or multi-objective problems—remain incomplete or highly problem-dependent [12,13,14].

In parallel, surrogate-based and probabilistic approaches, such as Bayesian optimization [15,16], have emerged as powerful tools for expensive black-box optimization. While these methods provide probabilistic guarantees under specific assumptions, they rely on model fidelity and often struggle with high-dimensional spaces or highly irregular feasible regions [17]. Similarly, deterministic global optimization methods, including branch-and-bound and Lipschitz-based algorithms [18,19,20], offer rigorous guarantees but frequently suffer from exponential complexity and limited scalability.

Multi-objective optimization further amplifies these challenges. Evolutionary algorithms such as NSGA-II [21] and SPEA2 [22] have become standard tools for approximating Pareto fronts, yet they produce discrete approximations without deterministic convergence guarantees. Moreover, the theoretical connection between such algorithms and the Karush–Kuhn–Tucker (KKT) optimality conditions remain indirect and largely heuristic [23,24]. Weighted-sum scalarization provides a classical bridge between multi-objective and single-objective optimization; however, it fails to recover non-supported Pareto solutions in nonconvex settings and lacks a unified convergence theory [25].

Against this backdrop, Inage and Hebishima introduced a fundamentally different paradigm known as the Monte Carlo Stochastic Optimization Technique (MOST) [26,27]. Unlike conventional pointwise evaluation methods, MOST operates on a region-based principle: the search domain is recursively partitioned, and each subregion is evaluated via Monte Carlo integration. The region with the smallest integral value is selected for further refinement. This seemingly simple mechanism leads to a striking property: the search region shrinks deterministically at each iteration, yielding geometric convergence of the form

$$\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{m}\left(R_n\right)=O\left(2^{-n}\right).$$

This deterministic region-shrinking mechanism distinguishes MOST sharply from both stochastic and gradient-based methods. In particular, the use of integral-based evaluation introduces an intrinsic smoothing effect over the objective landscape, enabling the method to avoid narrow local minima that would trap pointwise algorithms. Initial studies demonstrated that MOST outperforms conventional evolutionary methods in benchmark problems such as the Ackley and Schwefel functions, while maintaining stable convergence behavior [26].

Subsequently, MOST was extended to multi-objective optimization through a weighted scalarization framework that preserves its deterministic convergence properties [27]. This extension showed that, for each weight vector, MOST converges to a Pareto-consistent solution without the oscillatory behavior typical of evolutionary approaches. However, despite these promising developments, several fundamental theoretical gaps remained unresolved.

First, existing MOST formulations lacked a rigorous treatment of constrained optimization. In particular, no formal connection had been established between MOST and the KKT conditions, which constitute the cornerstone of constrained nonlinear optimization theory [3,23]. Second, the role of Lagrange multipliers—central to both theoretical analysis and practical interpretation—had not been incorporated into the MOST framework. Third, while multi-objective MOST demonstrated empirical success, its relationship to Pareto–KKT optimality conditions remained unproven. Finally, the behavior of MOST near nonlinear (curved) constraint boundaries, which are prevalent in engineering applications, had not been analyzed.

The present work addresses these gaps by developing a unified and mathematically rigorous optimization framework based on MOST. The core idea is to embed the KKT system directly into an augmented Lagrangian functional and to apply the deterministic region-shrinking mechanism of MOST in this extended variable space. This approach enables simultaneous enforcement of stationarity, primal feasibility, dual feasibility, and complementary slackness within a single optimization process.

A key contribution of this paper is the establishment of a non-circular global convergence theory based on integral comparison over shrinking regions. Under mild regularity assumptions, including Lipschitz continuity, we show that the region containing the global KKT point is selected infinitely often, leading to deterministic convergence. In addition, we provide a probabilistic analysis of Monte Carlo integration errors using concentration inequalities, demonstrating that sampling noise does not compromise convergence. The resulting framework thus combines deterministic geometric convergence with probabilistic robustness.

Furthermore, we rigorously establish the equivalence between the minimization of the extended Lagrangian and the satisfaction of the full KKT system. Under the Linear Independence Constraint Qualification (LICQ), we prove the uniqueness and continuity of Lagrange multipliers and show that they converge at the same geometric rate as the primal variables. This result places MOST in a fundamentally different category from evolutionary algorithms, which do not provide access to multiplier information.

In the multi-objective setting, we demonstrate that the proposed framework converges to Pareto–KKT stationary points for each weight vector, thereby establishing a direct and rigorous connection between scalarization-based optimization and KKT theory. Unlike classical weighted-sum methods, the proposed approach retains deterministic convergence properties even in highly nonconvex settings.

Finally, we develop a geometric theory explaining the behavior of MOST near curved constraint boundaries. We show that, due to exponential region shrinking, nonlinear constraint surfaces become asymptotically indistinguishable from their tangent hyperplanes. This result provides a natural explanation for the emergence of KKT normality conditions without explicit constraint linearization.

The contributions of this study can be summarized as follows:

• A rigorous formulation of constrained MOST based on an extended Lagrangian embedding the full KKT system.
• A non-circular proof of global convergence via integral-based region selection.
• A probabilistic analysis of Monte Carlo errors ensuring robust region selection.
• A proof of uniqueness and geometric convergence of Lagrange multipliers under LICQ.
• A unified Pareto–KKT convergence theory for multi-objective constrained optimization.
• A geometric explanation of constraint handling via automatic tangent-plane approximation.

Collectively, these results establish MOST as a new class of deterministic, derivative-free optimization methods with rigorous guarantees for constrained and multi-objective problems.

2. Mathematical Preliminaries

This chapter establishes the mathematical foundations required for the rigorous analysis of the Monte Carlo Stochastic Optimization Technique (MOST). We introduce the notation, define constrained and multi-objective optimality conditions, clarify the limitations of classical scalarization methods, and explicitly state the regularity assumptions—particularly Lipschitz continuity, measure-theoretic structure, and probabilistic conditions—that are essential for the convergence theory developed in subsequent chapters.

2.1. Notation and Problem Setting

Let $\mathsf{\Omega }\subset {\mathbb{R}}^d$ be a bounded hyperrectangle, which serves as the search domain:

$\begin{array}{cccc}& \mathsf{\Omega }=\prod _{i=1}^d[a_i,b_i].& & \left(1\right)\end{array}$

A decision variable is denoted by

$\begin{array}{cccc}& x\in \mathsf{\Omega }.& & \left(2\right)\end{array}$

We consider a general constrained nonlinear optimization problem:

$\begin{array}{cccc}& \underset{x\in \mathsf{\Omega }}{\mathrm{m}\mathrm{i}\mathrm{n}}f\left(x\right)& & \left(3\right)\end{array}$

subject to equality and inequality constraints:

$\begin{array}{cccc}& h_i\left(x\right)=0,i=1,\dots ,m,& & \left(4\right)\end{array}$

$\begin{array}{cccc}& g_j\left(x\right)\le 0,j=1,\dots ,p.& & \left(5\right)\end{array}$

The feasible set is defined as

$\begin{array}{cccc}& \mathcal{F}=\{x\in \mathsf{\Omega }\mid h_i(x)=0,\hspace{0.25em} g_j(x)\le 0\}.& & \left(6\right)\end{array}$

We denote the diameter of a set $R\subset {\mathbb{R}}^d$ by

$\begin{array}{cccc}& \mathrm{d}\mathrm{i}\mathrm{a}\mathrm{m}\left(R\right)=\underset{x,y\in R}{\mathrm{s}\mathrm{u}\mathrm{p}}\parallel x-y\parallel .& & \left(7\right)\end{array}$

2.2. Karush–Kuhn–Tucker (KKT) Conditions

The Lagrangian associated with problem (3)–(5) is defined as

$\begin{array}{cccc}& \mathcal{L}(x,\lambda ,\mu )=f\left(x\right)+\sum _{i=1}^m{\lambda }_i{h}_i\left(x\right)+\sum _{j=1}^p{\mu }_j{g}_j\left(x\right),& & \left(8\right)\end{array}$

where $\lambda \in {\mathbb{R}}^m$ and $\mu \in {\mathbb{R}}^p$ are Lagrange multipliers. A point $x^{*}\in \mathcal{F}$ is said to satisfy the KKT conditions if there exist multipliers $\left({\lambda }^{*},{\mu }^{*}\right)$ such that:

(i) Stationarity

$\begin{array}{cccc}& {\nabla }_x\mathcal{L}(x^{*},{\lambda }^{*},{\mu }^{*})=0,& & \left(9\right)\end{array}$

(ii) Primal feasibility

$\begin{array}{cccc}& h_i\left(x^{*}\right)=0,g_j\left(x^{*}\right)\le 0,& & \left(10\right)\end{array}$

(iii) Dual feasibility

$\begin{array}{cccc}& {\mu }_j^{*}\ge 0,& & \left(11\right)\end{array}$

(iv) Complementary slackness

$\begin{array}{cccc}& {\mu }_j^{*}{g}_j\left(x^{*}\right)=0.& & \left(12\right)\end{array}$

A fundamental regularity condition is the Linear Independence Constraint Qualification (LICQ), which requires that the gradients of active constraints are linearly independent:

$\begin{array}{cccc}& \{\nabla h_i(x^{*}){\}}_{i=1}^m\cup \{\nabla g_j\left(x^{*}\right)\mid g_j\left(x^{*}\right)=0\} \mathrm{are} \mathrm{linearly} \mathrm{independent}.& & \left(13\right)\end{array}$

Under LICQ, the multipliers $\left({\lambda }^{*},{\mu }^{*}\right)$ are unique [28,29,30].

2.3. Multi-Objective Optimization and Pareto Optimality

We consider a vector-valued objective function

$\begin{array}{cccc}& F\left(x\right)=\left(f_1\right(x),\dots ,f_k(x\left)\right).& & \left(14\right)\end{array}$

A point $x^{*}\in \mathcal{F}$ is Pareto optimal if there is no $x\in \mathcal{F}$ such that

$\begin{array}{cccc}& f_i\left(x\right)\le f_i\left(x^{*}\right)\hspace{0.25em} \hspace{0.25em} \forall i,\mathrm{and}{f}_j\left(x\right)<f_j\left(x^{*}\right) \mathrm{for} \mathrm{some} j.& & \left(15\right)\end{array}$

Under standard regularity conditions, a Pareto-optimal point satisfies the Pareto–KKT condition:

$\begin{array}{cccc}& \sum _{i=1}^k{w}_i\nabla f_i\left(x^{*}\right)+\sum _{i=1}^m{\lambda }_i^{*}\nabla h_i\left(x^{*}\right)+\sum _{j=1}^p{\mu }_j^{*}\nabla g_j\left(x^{*}\right)=0,& & \left(16\right)\end{array}$

for some weight vector

$\begin{array}{cccc}& w\in {\mathbb{R}}^k,w_i\ge 0,\sum _{i=1}^k{w}_i=1.& & \left(17\right)\end{array}$

2.4 Weighted-Sum Scalarization and Its Limitations

A classical approach to multi-objective optimization is weighted-sum scalarization:

$\begin{array}{cccc}& \underset{x\in \mathcal{F}}{\mathrm{m}\mathrm{i}\mathrm{n}}\sum _{i=1}^k{w}_i{f}_i\left(x\right).& & \left(18\right)\end{array}$

While (18) recovers all Pareto-optimal solutions when the objective functions are convex, it suffers from fundamental limitations:

• In nonconvex problems, certain Pareto-optimal points (non-supported points) cannot be obtained [31].
• Multiple distinct solutions may correspond to the same weight vector.
• The mapping from weights to Pareto solutions may be discontinuous.

These limitations motivate the need for a more robust framework capable of preserving optimality conditions beyond convex settings.

2.5. Regularity Assumptions

To establish rigorous convergence results, we impose the following assumptions.

Assumption A1 (Lipschitz Continuity)

The objective and constraint functions satisfy

$\begin{array}{cccc}& \mid f\left(x\right)-f\left(y\right)\mid \le L\parallel x-y\parallel ,\forall x,y\in \mathsf{\Omega },& & \left(19\right)\end{array}$

and similarly, for $h_i,g_j$.

Assumption A2 (Boundedness)

The domain $\mathsf{\Omega }$ is compact:

$\begin{array}{cccc}& \mathsf{\Omega }\subset {\mathbb{R}}^d \mathrm{is} \mathrm{bounded} \mathrm{and} \mathrm{closed}.& & \left(20\right)\end{array}$

Assumption A3 (Measurability)

All functions are Lebesgue measurable:

$\begin{array}{cccc}& f,h_i,g_j\in L^{\mathrm{\infty }}\left(\mathsf{\Omega }\right).& & \left(21\right)\end{array}$

Assumption A4 (Existence of KKT Point)

There exists at least one KKT point $x^{*}\in \mathcal{F}$. (22)

Assumption A5 (Regularity of Constraints)

LICQ holds at the optimal point $x^{*}$. (23)

2.6. Measure-Theoretic Framework

For any measurable subset $R\subset \mathsf{\Omega }$, we define its integral value:

$\begin{array}{cccc}& I\left(R\right)={\int }_{R}f\left(x\right)\hspace{0.17em}dx.& & \left(24\right)\end{array}$

Under Assumption A1, the following approximation holds for sufficiently small regions:

$\begin{array}{cccc}& I\left(R\right)=f\left(x^{*}\right)\mid R\mid +O\left(\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{m}\right(R\left)\right),& & \left(25\right)\end{array}$

where $x^{*}\in R$ is a minimizer. This relation is fundamental to the integral-based selection mechanism of MOST.

2.7. Probabilistic Framework for Monte Carlo Estimation

Let $X_1,\dots ,X_n\sim \mathrm{Uniform}\left(R\right)$. The Monte Carlo estimator is defined as

$\begin{array}{cccc}& {\widehat{I}}_n\left(R\right)={\displaystyle \frac{\mid R\mid }{n}}\sum _{i=1}^{n}f(X_i).& & \left(26\right)\end{array}$

By the law of large numbers:

$\begin{array}{cccc}& {\widehat{I}}_n\left(R\right)\to I\left(R\right)\mathrm{almost} \mathrm{surely}.& & \left(27\right)\end{array}$

Moreover, concentration inequalities provide finite-sample guarantees:

$\begin{array}{cccc}& \mathbb{P}(\mid {\widehat{I}}_n(R)-I(R)\mid >ϵ)\le 2\mathrm{e}\mathrm{x}\mathrm{p}(-Cn{ϵ}^2),& & \left(28\right)\end{array}$

for some constant $C>0$ depending on the boundedness of $f$ [32].

2.8. Summary of Assumptions and Their Role

The assumptions introduced above serve distinct purposes:

• Lipschitz continuity (A1) ensures stability of integral comparisons.
• Compactness (A2) guarantees existence of minimizers.
• Measurability (A3) enables integration-based evaluation.
• LICQ (A5) ensures uniqueness of multipliers.
• Probabilistic bounds (28) control Monte Carlo error.

Together, these conditions establish a mathematically rigorous foundation upon which the deterministic and probabilistic convergence properties of MOST will be built.