A Measure-Theoretic Formulation of Monte Carlo Stochastic Optimization: Unifying Continuous and Discrete Domains

1. Introduction

Optimization constitutes a fundamental component across a wide range of scientific and engineering disciplines, including mechanical design, control systems, machine learning, and operations research. Classical deterministic optimization methods—such as gradient descent, quasi-Newton methods, trust-region frameworks, and interior-point techniques—have demonstrated strong theoretical guarantees in convex and smooth settings [1,2,3,4]. However, their applicability becomes limited when addressing nonconvex, multimodal, discontinuous, or nonsmooth optimization problems, which are frequently encountered in real-world applications [5,6,7].

To overcome these limitations, numerous stochastic and population-based algorithms have been developed, including Genetic Algorithms (GA) [8], Differential Evolution (DE) [9], Particle Swarm Optimization (PSO) [10], and Covariance Matrix Adaptation Evolution Strategy (CMA-ES) [11]. These methods exhibit robustness against multimodality and do not require gradient information. Nevertheless, they rely heavily on stochastic exploration, and their convergence properties are typically probabilistic or asymptotic, lacking deterministic guarantees in general [12,13,14].

In parallel, surrogate-based optimization methods such as Bayesian optimization have gained attention for expensive black-box problems [15,16]. Although these approaches provide probabilistic convergence guarantees under specific assumptions, their performance depends strongly on model accuracy and deteriorates in high-dimensional or irregular search spaces [17]. Deterministic global optimization methods, including branch-and-bound and Lipschitz-based algorithms [18,19,20], offer rigorous guarantees but often suffer from exponential computational complexity and scalability issues.

Multi-objective optimization further increases complexity. Evolutionary algorithms such as NSGA-II [21] and SPEA2 [22] are widely used to approximate Pareto fronts, yet they provide only discrete approximations and lack deterministic convergence guarantees. Moreover, classical scalarization approaches fail to capture non-supported Pareto solutions in nonconvex problems [23,24,25].

Against this backdrop, Inage and Hebishima introduced a novel optimization paradigm known as the Monte Carlo Stochastic Optimization Technique (MOST) [26,27]. Unlike conventional pointwise optimization methods, MOST is based on a region-wise evaluation principle: the search domain is recursively partitioned, and each subregion is evaluated via Monte Carlo integration. The subregion with the smallest integral value is selected, and this process is repeated iteratively. This region-based strategy introduces an intrinsic smoothing effect over the objective landscape, enabling robust global exploration and deterministic geometric contraction of the search domain. Numerical studies have demonstrated that MOST achieves higher accuracy and faster convergence than genetic algorithms and gradient-based methods on benchmark problems such as the Ackley and Schwefel functions [26].

Subsequent extensions have generalized MOST to multi-objective optimization problems, where convergence toward Pareto-consistent solutions has been demonstrated [27]. More recently, a rigorous deterministic–probabilistic framework has been established, including convergence proofs, connections to Karush–Kuhn–Tucker (KKT) conditions, and extensions to constrained optimization [28]. These developments position MOST as a new class of derivative-free global optimization methods combining deterministic structure and stochastic sampling.

Despite these advances, an important theoretical gap remains. Existing formulations of MOST have been developed primarily for continuous domains, implicitly relying on Lebesgue integration. However, many practical optimization problems—particularly in engineering design, combinatorial optimization, and digital control—are inherently discrete. Extending MOST to discrete domains is not a trivial matter, as the concept of integration must be carefully reinterpreted, and the algorithmic structure must be adapted accordingly.

In particular, the core mechanism of MOST—region selection based on integral comparison—raises a fundamental question: Can the integral-based framework be rigorously extended to discrete optimization problems in a mathematically consistent manner? This question naturally leads to a measure-theoretic perspective, in which continuous and discrete optimization can be treated within a unified framework.

The key idea of this study is to reinterpret MOST as a measure-based optimization method, where region evaluation is defined through a normalized integral with respect to an underlying measure. In continuous domains, this corresponds to the Lebesgue measure, whereas in discrete domains, it corresponds to the counting measure. Furthermore, in the case of odd-number partitioning in discrete spaces, we introduce a weighted counting measure to consistently treat shared boundary points.

Based on this perspective, we develop a unified theoretical framework that encompasses both continuous and discrete MOST. In the discrete setting, we establish a rigorous algorithmic construction, including a novel bisection strategy for even and odd cardinalities, and a consistent Monte Carlo sampling scheme based on discrete probability measures. This formulation enables a direct extension of the integral-based selection principle to discrete spaces without loss of theoretical consistency.

The contributions of this study are summarized as follows:

• A measure-theoretic reformulation of MOST that provides a unified mathematical framework for region-based optimization.
• A rigorous construction of discrete MOST using counting measures and weighted measures.
• A theoretical formulation of bisection strategies for both even and odd cardinalities in discrete domains.
• A unified theory showing that continuous and discrete MOST are special cases of a single measure-based optimization framework.
• A clarification of the theoretical limitations of MOST, particularly in the presence of highly localized optima.
• Numerical validation through benchmark functions (e.g., Ackley function), comparing theoretical solutions and discrete MOST solutions under various discretization resolutions.

Through these developments, this study establishes MOST as a measure-based optimization paradigm that bridges continuous and discrete domains within a single rigorous framework. This unified perspective not only deepens the theoretical understanding of MOST but also expands its applicability to a broader class of optimization problems.

Chapter 2. Fundamental Concept of MOST in Continuous Domains

2.1. Problem Setting

We consider the unconstrained minimization problem defined over a bounded continuous domain:

$$\begin{array}{cccc}& \underset{x\in \mathsf{\Omega }}{\mathrm{m}\mathrm{i}\mathrm{n}}f\left(x\right),& & \end{array}$$

(1)

where $\mathsf{\Omega }\subset {\mathbb{R}}^n$ is a compact hyper-rectangular domain, and $f:\mathsf{\Omega }\to \mathbb{R}$ is a measurable objective function. Classical optimization methods evaluate the objective function at specific points and iteratively update candidate solutions based on gradient information or stochastic sampling [1,2,3,4,7]. In contrast, the Monte Carlo Stochastic Optimization Technique (MOST) adopts a fundamentally different viewpoint: it evaluates regions rather than individual points.

2.2. Core Idea: Region-Based Optimization via Integral Comparison

The central idea of MOST is to replace pointwise evaluation with integral-based evaluation over subdomains.

For any measurable subset $A\subset \mathsf{\Omega }$ with nonzero measure, define the regional evaluation functional:

$$\begin{array}{cccc}& J\left(A\right)={\displaystyle \frac{1}{|A|}}{\int }_{A}f\left(x\right)\hspace{0.17em}dx,& & \end{array}$$

(2)

where $|A|$ denotes the Lebesgue measure of $A$. This quantity represents the average value of the objective function over the region $A$.

The key principle of MOST is: Regions containing lower objective values tend to exhibit smaller average values. Thus, instead of directly searching for a point minimizing $f\left(x\right)$, MOST iteratively identifies subregions with smaller values of $J\left(A\right)$.

2.3. Binary Partitioning of the Search Domain

Let the initial search domain be denoted by

$$\begin{array}{cccc}& {\mathsf{\Omega }}^{\left(0\right)}=\mathsf{\Omega }.& & \end{array}$$

(3)

At iteration $k$, the current region ${\mathsf{\Omega }}^{\left(k\right)}$ is partitioned into two subregions along a selected coordinate direction. Without loss of generality, suppose that the partition is performed along the $j$-th coordinate:

$$\begin{array}{cccc}& {\mathsf{\Omega }}^{\left(k\right)}={\mathsf{\Omega }}_L^{\left(k\right)}\cup {\mathsf{\Omega }}_R^{\left(k\right)},{\mathsf{\Omega }}_L^{\left(k\right)}\cap {\mathsf{\Omega }}_R^{\left(k\right)}=\mathrm{\varnothing }.& & \end{array}$$

(4)

Each subregion satisfies

$$\begin{array}{cccc}& |{\mathsf{\Omega }}_L^{\left(k\right)}|=|{\mathsf{\Omega }}_R^{\left(k\right)}|={\displaystyle \frac{1}{2}}|{\mathsf{\Omega }}^{\left(k\right)}|.& & \end{array}$$

(5)

his binary partitioning ensures a systematic reduction of the search domain.

2.4. Monte Carlo Approximation of Regional Integrals

In practical implementations, the integral in (2) is evaluated using Monte Carlo sampling. Let $\left\{x_i{\}}_{i=1}^M\right.$ be independent and identically distributed samples drawn uniformly from the region $A$:

$$\begin{array}{cccc}& x_i\sim \mathrm{Uniform}\left(A\right).& & \end{array}$$

(6)

Then the Monte Carlo estimator of $J\left(A\right)$ is given by:

$$\begin{array}{cccc}& {\widehat{J}}_M\left(A\right)={\displaystyle \frac{1}{M}}{\sum }_{i=1}^{M}f(x_i).& & \end{array}$$

(7)

By the law of large numbers [13,14], we have:

$$\begin{array}{cccc}& {\widehat{J}}_M\left(A\right)\to J\left(A\right)\mathrm{as} M\to \mathrm{\infty }.& & \end{array}$$

(8)

Moreover, concentration inequalities provide probabilistic error bounds:

$$\begin{array}{cccc}& \mathbb{P}\left(|{\widehat{J}}_M(A)-J(A)|>\epsilon \right)\le C\mathrm{e}\mathrm{x}\mathrm{p}(-cM{\epsilon }^2),& & \end{array}$$

(9)

for some constants $C,c>0$ depending on the boundedness of $f$ [14].

2.5. Region Selection Rule

At each iteration, MOST compares the estimated average values of the two subregions:

$$\begin{array}{cccc}& {\widehat{J}}_M\left({\mathsf{\Omega }}_L^{\left(k\right)}\right),{\widehat{J}}_M\left({\mathsf{\Omega }}_R^{\left(k\right)}\right).& & \end{array}$$

(10)

The next search region is selected as:

$$\begin{array}{cccc}& {\mathsf{\Omega }}^{\left(k+1\right)}=\left\{\begin{array}{cc}{\mathsf{\Omega }}_L^{\left(k\right)},& \mathrm{if} {\widehat{J}}_M\left({\mathsf{\Omega }}_L^{\left(k\right)}\right)\le {\widehat{J}}_M\left({\mathsf{\Omega }}_R^{\left(k\right)}\right),\\ {\mathsf{\Omega }}_R^{\left(k\right)},& \mathrm{otherwise}.\end{array}\right.& & \end{array}$$

(11)

This deterministic selection rule forms the core of MOST.

2.6. Deterministic Shrinking of the Search Region

Since each iteration halves the domain along one coordinate, the diameter of the search region satisfies:

$$\begin{array}{cccc}& \mathrm{d}\mathrm{i}\mathrm{a}\mathrm{m}\left({\mathsf{\Omega }}^{\left(k+1\right)}\right)\le {\displaystyle \frac{1}{2}}\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{m}\left({\mathsf{\Omega }}^{\left(k\right)}\right).& & \end{array}$$

(12)

Recursively,

$$\begin{array}{cccc}& \mathrm{d}\mathrm{i}\mathrm{a}\mathrm{m}\left({\mathsf{\Omega }}^{\left(k\right)}\right)\le 2^{-k}\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{m}\left(\mathsf{\Omega }\right).& & \end{array}$$

(13)

Thus, the search region shrinks geometrically, independent of the objective function. This property distinguishes MOST from both gradient-based and stochastic optimization methods, which do not guarantee deterministic contraction [1,7,8,9,10,11].

2.7. Integral Averaging Effect

A crucial feature of MOST is the smoothing effect of integration. Let $x^{*}\in \mathsf{\Omega }$ be a global minimizer of $f$.

For a sufficiently small region $A$containing $x^{*}$, we have:

$$\begin{array}{cccc}& J\left(A\right)=f\left(x^{\mathrm{*}}\right)+O\left(\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{m}\right(A\left)\right).& & \end{array}$$

(14)

Thus, as the region shrinks,

$$\begin{array}{cccc}& J\left(A\right)\to f\left(x^{\mathrm{*}}\right).& & \end{array}$$

(15)

This implies:

·

Narrow local minima contribute negligibly to the integral

·

Regions containing the global minimum dominate asymptotically

This mechanism explains the robustness of MOST against multimodal objective functions, as observed in benchmark studies [26,27].

2.8. Comparison with Classical Optimization Methods

MOST differs fundamentally from existing optimization frameworks:

·

Gradient-based methods [1,2,3,4]

Depend on differentiability and are sensitive to local minima

·

Evolutionary algorithms [8,9,10,11,12]

Provide stochastic exploration but lack deterministic convergence

·

Deterministic global optimization methods [18,19,20]

Require Lipschitz constants or bounding functions

·

Bayesian optimization [15,16,17]

Depends on surrogate models and suffers in high dimensions

In contrast, MOST:

·

Does not require gradients

·

Does not rely on surrogate models

·

Provides deterministic region shrinking

·

Uses integral averaging to mitigate local irregularities

2.9. Summary

In this chapter, we have revisited the fundamental concept of MOST in continuous domains.

The key elements of the method are:

• Region-based evaluation using average integrals (2)
• Recursive binary partitioning of the search domain (4)–(5)
• Monte Carlo approximation of regional integrals (7)–(9)
• Deterministic region selection based on integral comparison (11)
• Geometric shrinking of the search domain (12)–(13)

These properties establish MOST as a deterministic, derivative-free optimization framework that leverages measure-based averaging to achieve robustness against multimodality.

This continuous formulation serves as the foundation for the subsequent development of a measure-theoretic framework and its extension to discrete optimization domains in the following chapters.

Chapter 3. Measure-Theoretic Reformulation of MOST

This chapter establishes a measure-theoretic foundation for the Monte Carlo Stochastic Optimization Technique (MOST). By reformulating MOST within the framework of finite measure spaces, we provide a unified mathematical structure that encompasses both continuous and discrete optimization problems. This perspective reveals that MOST is not a pointwise optimization method, but rather a measure-based region selection mechanism.

3.1. Measure Space Formulation

To generalize the MOST framework, we introduce a finite measure space

$$\begin{array}{cccc}& (X,\mathcal{F},\mu ),& & \end{array}$$

(16)

where $X$ denotes the search domain, $\mathcal{F}$ is a σ-algebra of measurable subsets of $X$, and $\mu$ is a finite measure on $\left(X,\mathcal{F}\right)$.

Let $f:X\to \mathbb{R}$ be a measurable objective function.

This formulation enables a unified treatment of optimization problems:

·

Continuous domains: $\mu$ is the Lebesgue measure

·

Discrete domains: $\mu$ is the counting measure

Thus, both continuous and discrete optimization problems are embedded into a single mathematical framework.

3.2. Measure-Based Evaluation Functional

Within this setting, we define the MOST evaluation functional as the normalized integral over a measurable region $A\in \mathcal{F}$ with $\mu \left(A\right)>0$:

$$\begin{array}{cccc}& J_f\left(A\right)={\displaystyle \frac{1}{\mu \left(A\right)}}{\int }_{A}f\left(x\right)\hspace{0.17em}d\mu \left(x\right).& & \end{array}$$

(17)

This quantity represents the measure-weighted average of the objective function over the region $A$. A fundamental reinterpretation of MOST follows immediately: MOST does not minimize the objective function pointwise; instead, it selects regions that minimize the measure-based average of the objective function.

This shift from pointwise evaluation to regional averaging constitutes the conceptual core of the method.

3.3. Correspondence to Probability Measures

The normalized measure naturally induces a probability measure supported on $A$. Specifically, for any $A\in \mathcal{F}$ with $\mu \left(A\right)>0$, define

$$\begin{array}{cccc}& P_A\left(B\right)={\displaystyle \frac{\mu (B\cap A)}{\mu \left(A\right)}},B\in \mathcal{F}.& & \end{array}$$

(18)

Then $P_A$ is a probability measure on $\left(X,\mathcal{F}\right)$, and the functional $J_f\left(A\right)$ admits the probabilistic representation

$$\begin{array}{cccc}& J_f\left(A\right)={\int }_{X}f\left(x\right)\hspace{0.17em}d{P}_A\left(x\right)={\mathbb{E}}_{P_A}\left[f\left(x\right)\right].& & \end{array}$$

(19)

Thus, the regional evaluation in MOST is equivalent to computing an expectation under a probability measure induced by the underlying measure $\mu$.

This interpretation provides a direct connection between MOST and stochastic approximation theory [13,14].

3.4. Monte Carlo Approximation

In practical implementations, the expectation (19) is approximated via Monte Carlo sampling. Let $X_1,X_2,\dots ,X_M$ be independent and identically distributed random variables drawn according to $P_A$:

$$\begin{array}{cccc}& X_i\sim P_A.& & \end{array}$$

(20)

The Monte Carlo estimator of $J_f\left(A\right)$ is given by

$$\begin{array}{cccc}& {\widehat{J}}_M\left(A\right)={\displaystyle \frac{1}{M}}{\sum }_{i=1}^{M}f(X_i).& & \end{array}$$

(21)

By the strong law of large numbers [13], we have

$$\begin{array}{cccc}& {\widehat{J}}_M\left(A\right)\to J_f\left(A\right) \mathrm{almost} \mathrm{surely} \mathrm{as} M\to \mathrm{\infty }.& & \end{array}$$

(22)

Moreover, concentration inequalities provide finite-sample guarantees:

$$\begin{array}{cccc}& \mathbb{P}\left(|{\widehat{J}}_M(A)-J_f(A)|>\epsilon \right)\le C\mathrm{e}\mathrm{x}\mathrm{p}(-cM{\epsilon }^2),& & \end{array}$$

(23)

for some constants $C,c>0$ depending on the boundedness of $f$[14].

3.5. Fundamental Theoretical Properties

Theorem 3.1 (Consistency of Monte Carlo Estimation)

Let $f\in L^1(A,\mu )$. Then the estimator ${\widehat{J}}_M\left(A\right)$ satisfies

$$\begin{array}{cccc}& {\widehat{J}}_M\left(A\right)\to J_f\left(A\right)\mathrm{almost} \mathrm{surely} \mathrm{as} M\to \mathrm{\infty }.& & \end{array}$$

(24)

Proof.

This follows directly from the strong law of large numbers applied to the i.i.d (independent and identically distributed). sequence $\left\{f\left(X_i\right){\}}_{i=1}^M\right.$[13].

Theorem 3.2 (Measure Dependence of MOST Evaluation)

The functional $J_f\left(A\right)$ depends explicitly on the underlying measure $\mu$. In particular, if ${\mu }_1\ne {\mu }_2$, then in general

$$\begin{array}{cccc}& J_f^{\left({\mu }_1\right)}\left(A\right)\ne J_f^{\left({\mu }_2\right)}\left(A\right).& & \end{array}$$

(25)

Proof.

From definition (17), $J_f\left(A\right)$ depends on both the integral and the normalization factor $\mu \left(A\right)$. Since both quantities are determined by the measure, different measures yield different evaluations in general.

3.6. Interpretation and Structural Insights

The measure-theoretic formulation provides several fundamental insights into the nature of MOST:

1.

Measure-Based Optimization

MOST optimizes regions with respect to a measure-weighted objective, rather than optimizing points directly.

2.

Expectation-Based Evaluation

The regional evaluation is equivalent to an expectation under an induced probability measure.

3.

Intrinsic Smoothing Effect

The integral-based formulation suppresses narrow local fluctuations of the objective function, emphasizing global structure.

4.

Unified Framework

The same formulation applies to:

• o Continuous spaces (Lebesgue measure)
• o Discrete spaces (counting measure)
• o Hybrid spaces (product measures)

This unified viewpoint reveals that the distinction between continuous and discrete optimization is not structural, but purely a consequence of the choice of measure.

3.7. Summary

In this chapter, we have established a rigorous measure-theoretic formulation of MOST. The principal results are:

• The definition of the MOST evaluation functional as a normalized integral (17).
• The equivalence between regional evaluation and expectation under a probability measure (19).
• The Monte Carlo approximation framework and its almost sure convergence (21)–(24).
• The explicit dependence of MOST on the underlying measure (25).

These results demonstrate that MOST is fundamentally a measure-based optimization framework, providing a unified theoretical foundation for both continuous and discrete optimization.

This formulation serves as the basis for the construction of discrete MOST in the next chapter.

Chapter 4. Discrete MOST: Construction and Algorithm

This chapter develops a rigorous formulation of the Monte Carlo Stochastic Optimization Technique (MOST) in discrete domains. Building upon the measure-theoretic framework established in Chapter 3, we construct a discrete counterpart that preserves the essential structure of MOST while ensuring algorithmic consistency and reproducibility.

The central objective is to demonstrate that discrete MOST is not an ad hoc extension, but rather a natural consequence of replacing the Lebesgue measure with the counting measure.

4.1. Discrete Search Space and Counting Measure

Let the search space be a finite ordered set:

$$\begin{array}{cccc}& D=\{x_1,x_2,\dots ,x_N\},x_1<x_2<\dots <x_N.& & \end{array}$$

(26)

We equip $D$ with the counting measure $\nu$, defined by

$$\begin{array}{cccc}& \nu \left(A\right)=|A|,A\subset D.& & \end{array}$$

(27)

Under this measure, the integral of a function $f:D\to \mathbb{R}$ reduces to a finite sum:

$$\begin{array}{cccc}& {\int }_{A}f\left(x\right)\hspace{0.17em}d\nu \left(x\right)=\sum _{x\in A}f\left(x\right).& & \end{array}$$

(28)

Accordingly, the MOST evaluation functional becomes

$$\begin{array}{cccc}& J_f\left(A\right)={\displaystyle \frac{1}{|A|}}\sum _{x\in A}f\left(x\right),& & \end{array}$$

(29)

which corresponds to the arithmetic mean over the subset $A$. Thus, discrete MOST is obtained directly from the general formulation (17) by choosing $\mu =\nu$.

4.2. Bisection Strategy: Even and Odd Cardinalities

A key component of MOST is recursive domain partitioning. In discrete spaces, this requires careful treatment to preserve symmetry and consistency.

4.2.1. Even Cardinality

If $N=2m$, the set $D$ is partitioned into two disjoint subsets:

$$\begin{array}{cccc}& D_L=\left\{x_1,\dots ,x_m\right\},D_R=\left\{x_{m+1},\dots ,x_{2m}\right\}.& & \end{array}$$

(30)

These subsets satisfy:

$$\begin{array}{cccc}& |D_L|=|D_R|=m.& & \end{array}$$

(31)

This corresponds to a direct discrete analogue of continuous bisection.

4.2.2. Odd Cardinality

If $N=2m+1$, a symmetric partition is achieved by sharing the midpoint:

$$\begin{array}{cccc}& D_L=\{x_1,\dots ,x_m,x_{m+1}\},D_R=\{x_{m+1},x_{m+2},\dots ,x_{2m+1}\}.& & \end{array}$$

(32)

To ensure consistency with the measure-theoretic formulation, the midpoint $x_{m+1}$ is assigned half weight in each subset. Define weight functions $w_L, w_R:D\to \left[\mathrm{0,1}\right]$ as:

$$\begin{array}{cccc}& w_L\left(x_i\right)=\left\{\begin{array}{cc}1,& i\le m,\\ {\displaystyle \frac{1}{2}},& i=m+1,\\ 0,& i\ge m+2,\end{array}\right.w_R\left(x_i\right)=\left\{\begin{array}{cc}0,& i\le m,\\ {\displaystyle \frac{1}{2}},& i=m+1,\\ 1,& i\ge m+2.\end{array}\right.& & \end{array}$$

(33)

The corresponding evaluation functionals are:

$$\begin{array}{cccc}& J_L={\displaystyle \frac{\sum _{x\in D}{w}_L\left(x\right)f\left(x\right)}{\sum _{x\in D}{w}_L\left(x\right)}},J_R={\displaystyle \frac{\sum _{x\in D}{w}_R\left(x\right)f\left(x\right)}{\sum _{x\in D}{w}_R\left(x\right)}}.& & \end{array}$$

(34)

This construction ensures:

·

Symmetry of partitioning

·

Equal effective measure

·

Consistency with continuous domain splitting