The Market Control Illusion: A Separated Control and Controllability Framework for Non-Monotonic Prediction–Profit Dynamics in Financial Markets

I. Introduction

The architecture of modern quantitative trading is built upon the implicit assumption that the fidelity of a predictive model directly translates to the efficacy of a trading strategy. This assumption, while intuitively appealing, frequently disintegrates when subjected to the rigors of control-theoretic analysis. The disconnect, often referred to as the market control illusion, arises because statistical precision—typically quantified through measures of central tendency such as Mean Squared Error (MSE) or Mean Absolute Error (MAE)—fails to account for the dynamical reachability of the system state under constraints. While a predictive model seeks to minimize the discrepancy between an estimate and a realized value, an economic control strategy must navigate a state-space governed by physical and liquidity limitations, feedback loops, and non-Markovian dependencies.

The architecture of systematic trading systems typically follows a unidirectional flow: data ingestion, feature extraction, signal generation, and execution. This structure presupposes that the market is a passive entity, an exogenous process that can be predicted with sufficient data and computational power. However, once a strategy moves from the backtest to the live market, it transitions from a passive observer to an active participant. The act of trading introduces market impact, a phenomenon that mirrors the "actuator saturation" found in aerospace and robotic control systems. When a trading system issues a command that exceeds the available liquidity at a given price level, the "actuator"—the market’s limit order book—saturates, leading to slippage and "windup" effects that can destabilize the entire strategy.

To facilitate a rigorous analysis of market dynamics through the lens of engineering, it is necessary to establish a common taxonomy. The following Table 1 maps fundamental concepts from control theory—traditionally applied to physical systems like aerospace vehicles—to their operational equivalents in financial market microstructure. This mapping provides a conceptual ’Rosetta Stone’ for interpreting how feedback-driven stability and mechanical constraints govern trading outcomes independently of predictive accuracy.

By adopting this nomenclature, the ’Market Control Illusion’ can be defined more precisely: it is the category error of optimizing the Observer (predictive model) while ignoring the Actuator (execution limits) and the resulting Feedback (market impact). As demonstrated in the following sections, a strategy may possess a high-fidelity observer but remain fundamentally ’uncontrollable’ if the control law ignores the saturation limits of the market plant.

This paper explores the theoretical and empirical foundations of the Market Control Illusion. It synthesizes insights from Hubáček and Šír’s paradox—which proves that systematic profits can be generated using "bad" predictive models—and links these findings to the broader field of stochastic optimal control and non-Markovian dynamics. We investigate how market controllability is constrained by the rank of the liquidity matrix and how the geometry of market impact requires a shift from predictive modeling to a Linear Parameter-Varying (LPV) control framework. By analyzing the results of multi-market optimization for Battery Energy Storage Systems (BESS) in the Swedish energy sector and the universality of signature portfolios in path-dependent markets, we propose a new design philosophy for quantitative systems that prioritizes economic controllability over statistical precision.

II. Related Literature

The historical development of financial modeling has often lagged behind the advancements in control theory. The classical Markowitz Mean-Variance framework, while elegant, is essentially a static optimization problem that ignores the time-varying nature of market parameters and the path-dependency of returns. Later, the Kelly Criterion provided a more dynamic approach to capital allocation, but it remained rooted in the assumption that the agent possesses an accurate estimate of the underlying probability distribution. This "informed investor" model, common in the literature on market inefficiencies, posits that profits are the reward for superior information leading to more accurate price estimations.

However, a growing body of research suggests that accuracy is a poor proxy for economic utility. Hubáček and Šír demonstrate that the market taker possesses an inherent advantage over the market maker. While the market maker must price an asset to balance aggregate information and manage risk across a wide pool of participants, the market taker only needs to identify specific biases or over-reactions. Their work on "beating the market with a bad predictive model" highlights that decorrelation from the market’s consensus is often more profitable than alignment with the "true" price. This aligns with findings in sports betting and electricity markets, where traditional accuracy measures fail to capture the economic value of identifying extreme "peak" events.[1,2].

In the realm of control theory, the "Separation Principle" has long defined the relationship between estimation and control. In linear-quadratic-Gaussian (LQG) systems, the tasks of filtering (estimating the state) and regulation (controlling the state) can be solved independently. Yet, the complexity of financial markets—characterized by partial observability, non-Gaussian jumps, and "rough" volatility—frequently breaks this separation. The Duncan-Mortensen-Zakai (DMZ) equations and the study of the "information state" reveal that optimal control in such environments requires a more nuanced understanding of the unnormalized conditional density of the market’s latent variables.[3,4].

Recent literature has also seen the emergence of "Signature Portfolios," which leverage rough path theory to handle the non-Markovian nature of market dynamics. These portfolios are universal approximators of path-dependent functions, allowing for the capture of geometric features of price movement that traditional point-predictive models ignore. Furthermore, the application of anti-windup and LPV frameworks from aerospace engineering to market impact modeling represents a significant step toward addressing the "saturation" challenges inherent in large-scale execution.[5,6].

III. The Paradox of the "Bad" Predictive Model

A seminal challenge to the predictive paradigm is found in the work of Hubáček and Šír, who demonstrate that systematic profits can be generated using price-predicting models that are statistically inferior to the market consensus. This counter-intuitive result is explained by the fundamental asymmetry between market makers and market takers[1].

A. The Advantage of the Market Taker

A market maker acts as a price estimator, penalized symmetrically for errors. If the true value of an asset is V, and the market maker sets the price at $P_m$, their error ${ϵ}_m=P_m-V$ results in a loss to arbitrageurs if $|{ϵ}_m|$ is large. To survive, the market maker must minimize their total estimation error across all trades.

In contrast, the market taker (the trader) only executes a trade when their own estimate $P_t$ differs significantly from $P_m$. If the trader’s error is ${ϵ}_t=P_t-V$, their profit condition is not based on the magnitude of ${ϵ}_t$ being smaller than ${ϵ}_m$, but rather on the trader’s ability to identify instances where the market maker’s bias ${ϵ}_m$ is skewed in a predictable direction.

B. The Decorrelation Objective

Standard machine learning models are trained to minimize MSE, which is equivalent to maximizing the correlation between the prediction $\widehat{y}$ and the true outcome y. Hubáček and Šír argue that this objective is suboptimal for traders because it aligns the trader’s errors with the market’s errors. Instead, they propose a decorrelation objective:

$$\underset{\theta }{min}\mathbb{E}\left[{({\widehat{y}}_{\theta }-y)}^2\right]+\lambda \mathrm{Corr}\left({\widehat{y}}_{\theta }-y,{\widehat{y}}_{\mathrm{market}}-y\right)$$

(1)

where $\lambda$ is a hyperparameter that penalizes the partial correlation between the trader’s residual error and the market’s error. By explicitly decorrelating from the market, the model learns to ignore the information already captured by the market price and instead focuses on "extra information" or inconspicuous biases in the market maker’s pricing.

Empirical validation across stock trading and sports betting shows that models with significantly higher MSE (worse predictive power) can yield consistently higher returns than those optimized for accuracy. This demonstrates that the information-theoretic value of a model w.r.t. the true price can be lower than that of the market, yet the model can remain profitable by being "wrong" in a way that the market is not.

IV. Mathematical Derivations of System Controllability

The formal investigation of why predictive power does not imply control begins with the state-space representation of a linear dynamical system. Consider the discrete-time model:

$$\mathbf{x}(t+1)=\mathbf{A}\mathbf{x}\left(t\right)+\mathbf{B}\mathbf{u}\left(t\right)$$

(2)

where $\mathbf{x}\left(t\right)\in {\mathbb{R}}^n$ is the state vector at time t, $\mathbf{A}\in {\mathbb{R}}^{n\times n}$ is the transition matrix defining the intrinsic dynamics, $\mathbf{B}\in {\mathbb{R}}^{n\times p}$ is the input matrix, and $\mathbf{u}\left(t\right)\in {\mathbb{R}}^p$ is the control input (the trade or investment).

A. The Kalman Reachability Criterion

To determine if the state $\mathbf{x}$ can be steered to any target ${\mathbf{x}}_f$ from an initial ${\mathbf{x}}_0$, we examine the sequence of states generated by the control inputs. Starting from $\mathbf{x}\left(0\right)=0$, the state at time n is given by:

$$\mathbf{x}\left(n\right)=\mathbf{B}\mathbf{u}(n-1)+\mathbf{A}\mathbf{B}\mathbf{u}(n-2)+\cdots +{\mathbf{A}}^{n-1}\mathbf{B}\mathbf{u}\left(0\right)$$

(3)

This can be rewritten as a matrix-vector product:

$$\mathbf{x}\left(n\right)=\left[\begin{array}{ccccc}\mathbf{B}& \mathbf{A}\mathbf{B}& {\mathbf{A}}^2\mathbf{B}& \dots & {\mathbf{A}}^{n-1}\mathbf{B}\end{array}\right]\left[\begin{array}{c}\mathbf{u}(n-1)\\ \mathbf{u}(n-2)\\ ⋮\\ \mathbf{u}\left(0\right)\end{array}\right]$$

(4)

The system is controllable if and only if the controllability matrix $\mathcal{C}=$ has full row rank n.3 If the rank of $\mathcal{C}$ is less than n, there exist regions of the state-space that the agent cannot reach, regardless of the precision of their estimate of $\mathbf{A}$. In quantitative trading, if the market’s response to an agent’s input ($\mathbf{B}$) does not span the necessary dimensions of the portfolio state, the agent lacks economic control. This often occurs in illiquid markets where certain assets or factors cannot be traded at the required scale.[7]

B. The Controllability Gramian and Energy Requirements

While the Kalman rank condition provides a binary assessment of controllability, the controllability Gramian offers a quantitative measure of the "effort" or "energy" required to steer the system. For a stable system where the spectral radius of $\mathbf{A}$ is less than unity ($\rho \left(\mathbf{A}\right)<1$), the discrete controllability Gramian $\mathbf{W}$ is defined as the solution to the discrete Lyapunov equation:

$$\mathbf{W}-\mathbf{A}\mathbf{W}{\mathbf{A}}^{\top }=\mathbf{B}{\mathbf{B}}^{\top }$$

(5)

which expands to the infinite series

$$\mathbf{W}=\sum _{m=0}^{\infty }{\mathbf{A}}^m\mathbf{B}{\mathbf{B}}^{\top }{\left({\mathbf{A}}^{\top }\right)}^m.$$

(6)

The system is controllable if and only if $\mathbf{W}$ is positive definite. The eigenvalues of $\mathbf{W}$ indicate the ease of control in different directions of the state space. If the determinant of $\mathbf{W}$ is near zero, the system is “almost” uncontrollable, meaning that the control inputs required to move the state in certain directions would need to be extremely large, likely exceeding the saturation limits of the market.

Interestingly, the Gramian can be used to reduce the complexity of large-scale systems by removing redundant or "almost" redundant information. In high-dimensional portfolio management, this suggests that instead of trying to predict every asset price, an agent should focus on the subspace of the market where they possess the highest controllability.

Table II. System Properties and Their Practical Interpretation.

Property	Mathematical Indicator	Practical Interpretation
Full Controllability	$\mathrm{rank}\left(\mathcal{C}\right)=n$	Any portfolio target can be reached
Directional Energy	${\lambda }_i\left(\mathbf{W}\right)$	The input effort required to steer factor i
System Stability	$max\lambda \left(\mathbf{A}\right)$	Largest eigenvalue indicates tendency toward stability or divergence
Linear Independence	$det\left(\mathbf{G}\right)\ne 0$	Data features provide non-redundant control signals

Table II. System Properties and Their Practical Interpretation.

Property	Mathematical Indicator	Practical Interpretation
Full Controllability	$\mathrm{rank}\left(\mathcal{C}\right)=n$	Any portfolio target can be reached
Directional Energy	${\lambda }_i\left(\mathbf{W}\right)$	The input effort required to steer factor i
System Stability	$max\lambda \left(\mathbf{A}\right)$	Largest eigenvalue indicates tendency toward stability or divergence
Linear Independence	$det\left(\mathbf{G}\right)\ne 0$	Data features provide non-redundant control signals

C. Nonlinear and Fractional-Order Extensions

The linear model is often an approximation of more complex, non-integer order dynamics found in financial time series with long-range dependence. For systems governed by fractional-order differential equations of the form:

$${}^{C}D^{\alpha }\mathbf{x}\left(t\right)=\mathbf{A}\mathbf{x}\left(t\right)+\mathbf{B}\mathbf{u}\left(t\right)$$

(7)

where $\alpha \in (1,2)$ represents the order of the derivative, the solution involves Mittag-Leffler matrix functions rather than standard exponentials.Controllability in these settings is established via the Schaefer fixed-point theorem and the Arzela-Ascoli theorem, proving that even under fractional "memory" effects, a suitable control function exists if the associated Gramian is non-singular. This is particularly relevant for "rough" market models where prices exhibit sub-diffusive or super-diffusive behavior, necessitating a control strategy that accounts for the path-dependent nature of the state evolution[2].

1. Price Impact as a Nonlinear Limit

The impact of a trade on the market price can be modeled using a price impact function $g\left(z\right)$. A common model for the return $r\left(t\right)$ resulting from excess demand $D\left(t\right)$ is:

$$r\left(t\right)=arctan\left({\displaystyle \frac{D\left(t\right)}{\lambda }}\right)$$

(8)

where $\lambda$ represents the market depth.4 The arctan function is a classic example of actuator saturation: as the size of the trade $D\left(t\right)$ increases, the marginal impact on the price return saturates, but the total slippage cost continues to grow. This non-linearity means that a control input (a trade) that is "optimal" based on a prediction may be impossible to execute without driving the price so far that the profit is erased.

2. Anti-Windup Control in Trading

In control systems, ignoring saturation leads to "windup," where an integrator continues to accumulate error even when the actuator is at its limit, resulting in massive overshoot once the system becomes responsive again. In trading, windup manifests as "execution-driven momentum," where an algorithm keeps trying to buy a rising asset, accumulating a massive position that it cannot exit without crashing the price.

To combat this, anti-windup control synthesis uses a Linear Parameter-Varying (LPV) framework. The controller is scheduled according to the "saturation indicator" (e.g., the current ratio of order size to top-of-book liquidity). This allows for "graceful performance degradation" where the trading system automatically reduces its aggression as market impact increases, maintaining economic control even when the predictive signal is strong.

The following table contrasts the properties of an estimation-centric system with those of a control-centric system in the context of financial execution.

Table III. Estimation-Centric vs Control-Centric Trading Paradigms.

Feature	Estimation-Centric Paradigm	Control-Centric Paradigm
Primary Goal	Minimize forecast error (RMSE/MAE)	Maximize utility / reach target state
System View	Market is an exogenous signal	Market is a feedback-driven plant
Input Analysis	Ignored (assumes zero impact)	Modeled as control effort with saturation
State Feedback	Open-loop (predict then trade)	Closed-loop (trade, observe, adapt)
Risk Metric	Forecast variance	System stability and windup potential

Table III. Estimation-Centric vs Control-Centric Trading Paradigms.

Feature	Estimation-Centric Paradigm	Control-Centric Paradigm
Primary Goal	Minimize forecast error (RMSE/MAE)	Maximize utility / reach target state
System View	Market is an exogenous signal	Market is a feedback-driven plant
Input Analysis	Ignored (assumes zero impact)	Modeled as control effort with saturation
State Feedback	Open-loop (predict then trade)	Closed-loop (trade, observe, adapt)
Risk Metric	Forecast variance	System stability and windup potential