Anisotropy-Based Robust Performance Criteria for Linear Continuous Time Invariant Systems with Statistically Uncertain Noise

1. Introduction

The idea of quantifying the “amplification” (or “attenuation”) properties of a linear operator from one normed space to another in terms of its induced norm is ubiquitous in functional analysis, matrix theory and numerical methods, to mention some of the relevant areas. Using such a norm for operators on Hilbert spaces underlies the ${\mathcal{H}}_{\infty }$-control theory [20] for linear systems with square integrable inputs. When it is preferable to have low sensitivity of the output variables of the system to the input disturbances, the corresponding performance criteria are concerned with stabilizing the closed-loop system (thus making it a bounded input-output operator) and minimizing its operator norm by an appropriate choice of a controller [17]. This includes not only control settings as such, but also filtering problems, where the role of the output process is played by the state estimation error [42]. Due to submultiplicativity of the operator norm, its minimization leads not only to disturbance attenuation with respect to the input but also with respect to perturbations in the system itself. The small-gain theorem [74] models such perturbations as feedback loops with norm-bounded uncertainties and is closely related to the invertibility of a perturbed identity operator and von Neumann series in Banach algebras [18].

The Rayleigh quotient [27], whose maximization defines the induced norm of a bounded operator on a Hilbert space (or a matrix in a finite-dimensional case), reduces to a quadratic form on the unit sphere. In a generic case, its maximum over the sphere is achieved at two points, which specify the worst-case direction for the input. This direction is exceptional and is not necessarily taken by the unit vector of the normalized input disturbance. Therefore, the operator norm, as a measure of the gain, is conservative in the case when the disturbance is not targeted at specific directions. At its extreme, this suggests the averaging of the Rayleigh quotient over the uniform probability distribution on the unit sphere, which leads to an appropriately rescaled Frobenius norm of the operator. This approach is applicable only to a finite-dimensional Hilbert space, where the unit sphere has a (unique up to a multiplicative constant) finite measure which is invariant under the group of rotations (see, for example, [1,11,39] and references therein). The normalization of this hypersurface Lebesgue measure yields the uniform distribution on the sphere (while there is no uniform distribution on an infinite-dimensional sphere, which is closely related to its noncompactness).

The deviation of an arbitrary probability measure on the sphere from the uniform distribution (as a reference measure) can be quantified in terms of entropy-theoretic proximity criteria. For example, the Kullback-Leibler (KL) relative entropy [12] (with respect to the uniform distribution on the sphere) leads to the anisotropy functional [58]. If the input disturbance is random and its direction distribution is not “too far” from being uniform in the sense that its anisotropy does not exceed a given nonnegative level a, then the maximization of the averaged Rayleigh quotient over such distributions leads to the a-anisotropic norm of the matrix. This norm occupies an intermediate position between the rescaled Frobenius norm, mentioned above, and the operator norm. Moreover, the latter two norms are the extreme cases of the a-anisotropic norm at $a=0$ and as $a\to +\infty$, respectively. The advantage of this norm is that the degree of its conservativeness is controlled by the parameter a which specifies the amount of statistical uncertainty in the direction distribution (with the uniform distribution playing the role of a “centre” of the uncertainty class). An equivalent interpretation is that a quantifies how far a hypothetical opponent can go in approximating the worst-case direction (corresponding to the largest singular value of the matrix) by an absolutely continuous probability distribution on the unit sphere. Although the anisotropy functional does not lend itself to closed-form calculation even for the direction distribution of a Gaussian random vector (except for isotropic Gaussian distributions with zero mean and scalar covariance matrices, in which case the anisotropy vanishes), its asymptotic growth rate (per time step) is computable for unboundedly growing fragments of stationary Gaussian random sequences, leading to the mean anisotropy [58].

Later, the “spherical” anisotropy functional was complemented with its more tractable counterpart [65] (on the space of inputs themselves rather than the unit sphere of their directions) in the form of the minimum relative entropy of the actual probability distribution with respect to isotropic Gaussian distributions, which coincides with a multivariable version of a power-entropy construct considered in a different context and for different purposes in [5]. Although this second anisotropy functional is an upper bound for the original one, it has the same infinite-horizon growth rate in the stationary Gaussian case mentioned above, with the resulting mean anisotropy being expressed in terms of the spectral density and the mean value of the sequence. Accordingly, the averaged Rayleigh quotient was replaced with the ratio of root-mean-square (RMS) values of the output and input.

The anisotropy functionals, anisotropy-constrained norms and anisotropy-based theory of stochastic robust control and filtering for linear systems (without internal perturbations) in infinite-horizon time-invariant and finite-horizon time-varying settings, as well as for Toeplitz operators acting on homogenous Gaussian random fields on multidimensional lattices, were originated by the author in a series of papers and research reports in the mid 1990s – early 2000s, starting from [58] (and its subsequently written conference version [47]) and including [13,59,60,61,62] whose extended account was provided in [14] and followed by [15,33,63,64,65]. Those founding works also included a set of MATLAB functions for anisotropy-based robust performance analysis and controller design reported in [13]. Being motivated as a stochastic extension of the ${\mathcal{H}}_{\infty }$-control theory to statistically uncertain linear discrete-time systems, the above mentioned developments aimed to bridge the gap between the deterministic approach of ${\mathcal{H}}_{\infty }$-control and the stochastic paradigm of linear quadratic Gaussian (LQG) control, which includes Kalman filtering [28] and its predecessor — the Kolmogorov-Wiener-Hopf theory of smoothing, filtering and prediction for stationary random processes [31,69]. This extension also addressed the robustness issues of the LQG approach which is oriented to an idealized scenario, when the random input disturbances have precisely known statistical characteristics and are organized as a Gaussian white-noise sequence or a standard Wiener process (from which a more complicated covariance structure can be obtained by using a shaping filter).

Not discussing here the other ways of combining the ${\mathcal{H}}_{\infty }$-theory and LQG approaches (see, for example, [6]), we note that the anisotropy-based theory offers probabilistic, system theoretic and computational tools (including a homotopy method for solving specific sets of cross-coupled Riccati, Lyapunov and log-determinant matrix algebraic equations) for robust performance analysis and synthesis of controllers and filters. This approach addresses robustness with respect to spatially and temporally coloured random disturbances (in the sense of statistical correlations between the entries of a multivariable noise at the same or different moments of time) with imprecisely known statistical properties. This is achieved by quantifying the statistical uncertainty (as a deviation from the nominal Gaussian white noise model) in terms of anisotropy, so that the resulting anisotropy-constrained version of the induced operator norm describes the worst-case RMS gain of the system over this uncertainty class.

Subsequently (in the 2000s and more recently), the theory was also being developed in the form of a suboptimal guaranteed approach to systems with internal perturbations [32], towards convexification of the anisotropy-based control synthesis [40,52,53] and suboptimal observer design [54,55], control settings with nonzero-mean input disturbances [34] and multiplicative noise [35,51,73] in the system, estimation problems with sensor failures [72], roundoff noise effects in digital controllers [37], parameterization of anisotropy-based controllers [36] and their asymptotic behaviour at low mean anisotropy levels [3], as well as other developments, including an adaptation of the anisotropy-based theory to descriptor systems [2].

The discrete-time setting is essential for the anisotropy-based theory, which stems from the anisotropy functionals using finite-dimensional spheres and finite segments of random sequences. At the same time, entropy-theoretic constructs are also known for infinite-dimensional objects such as diffusion processes. For example, the relative entropy (with respect to the Wiener measure) is correctly defined for the probability distribution of a diffusion process governed by a stochastic differential equation (SDE) with a nonzero drift (satisfying Novikov’s condition [43]) and the identity diffusion matrix. This is due to Girsanov’s theorem [23], where the preservation of the diffusion matrix of the reference standard Wiener process (so that only the drift is changed) is essential for the absolute continuity of measures, which plays an important role in relative entropy formulations of statistical uncertainty for continuous time stochastic control systems driven by diffusion processes [10,56,68].

The present paper (whose brief preprint version was archived in [67]) provides one of possible ways of rigorously extending the anisotropy-based approach (which underlies stochastic minimax formulations of robust filtering and control using worst-case RMS gains under entropy-theoretic constraints in terms of anisotropy) to linear continuous time invariant (LCTI) systems driven by statistically uncertain Ito processes. To this end, we employ a parametric family of conformal maps (specified by an auxiliary time scale parameter and related to the Cayley transform [48]) between the unit disk and the right half-plane for the discrete and continuous time transfer functions, respectively. Up to a multiplicative constant, these maps are identical to Tustin’s transform [8] for converting LCTI systems to linear discrete-time invariant (LDTI) systems and the other way around. This results in a subsidiary LDTI system which lends itself to application of the anisotropy-based theory. The spectral densities of the input and output of this subsidiary system are related to those of filtered versions of the original continuous-time processes (with the time scale parameter specifying the transient time of the filter) due to the relation between the spectral density of the Ornstein-Uhlenbeck (OU) process [29] and the logarithmic derivative of the Cayley map. The conformal correspondence is closely related to the presence of a denominator (which secures integrability in the continuous time case) in the inner-outer factorizability condition for spectral densities (describing the physical realizability of such a density with the aid of a stable causal shaping filter) [21,71]. This is one of the reasons why discrete-time results (many of which rely on boundedness of the frequency range) cannot merely be adopted without due modification (in particular, without taking the above mentioned denominator into account) for the continuous-time case, where the processes may contain arbitrarily fast components accommodated by the infinite frequency range making certain integrals divergent. The conformal relation allows the RMS gain of the LCTI system output with respect to the input noise to be defined in terms of the filtered processes. The resulting continuous-time extension of the anisotropic norm is the largest value of the gain subject to an upper constraint on the mean anisotropy of the noise as an entropy-theoretic measure of its deviation from isotropic white-noise disturbances which, up to scaling, are organized as the standard Wiener process. We also discuss the probabilistic and information-theoretic aspects of the worst-case disturbance which inherits from its discrete-time counterpart the structure of an additive mixture of a state-dependent drift with an innovation martingale process.

The paper is organized as follows. Section 2 describes the class of LCTI systems under consideration governed by linear SDEs driven by random disturbances in the form of Ito processes. Section 3 specifies the class of random disturbances produced by LCTI noise shaping filters from a standard Wiener process. Section 4 describes filtered versions of the underlying processes for such a system. Section 5 defines the RMS gain of the system in terms of the filtered input and output. Section 6 calculates the RMS gain in the nominal case of isotropic white-noise disturbances. Section 7 specifies the parameter-dependent conformal correspondence to a discrete-time system with the same RMS gain. Section 8 defines a two-parameter norm of the underlying system in terms of the anisotropic norm of the effective LDTI system. Section 9 describes state-space equations for computing this continuous-time extension of the anisotropic norm along with the structure of the worst-case disturbance. Section 10 discusses the Renyi [44] and KL relative entropy rates for the worst-case noise with respect to its martingale part and links them with the temporal component of the mean anisotropy level. Section 11 provides a numerical example of computing the two-parameter continuous-time extension of the anisotropic norm for an RLC circuit with coupled inductors and statistically uncertain noise. Section 12 makes concluding remarks.

2. Linear Stochastic Systems Being Considered

Consider an LCTI system F with an input W, which represents the external noise acting on the system, and an output Z. The latter consists of variables whose small values are beneficial for the system performance. For example, Z can be formed from reference trajectory tracking errors or state estimation errors which have to be minimized in some sense. It is assumed that W and Z are Ito processes [29] (with respect to a common filtration on the underlying probability space) on the real time axis, which take values in ${\mathbb{R}}^m$ and ${\mathbb{R}}^p$, respectively, and are related in a causal fashion as

$$Z\left(t\right)={\int }_{-\infty }^{t}f(t-\tau )\mathrm{d}W\left(\tau \right)+f_{0}W\left(t\right),t\in \mathbb{R}.$$

(1)

Here, $f_0\in {\mathbb{R}}^{p\times m}$ is a static gain matrix, and $f:{\mathbb{R}}_{+}\to {\mathbb{R}}^{p\times m}$ is the step response of the system as a linear input-output operator $W\mapsto Z$, with ${\mathbb{R}}_{+}:=[0,+\infty )$. In particular, if the noise W is a standard Wiener process [29] in ${\mathbb{R}}^m$, then the Ito integral in (1) is well-defined for any square integrable function $f\in L^2({\mathbb{R}}_{+},{\mathbb{R}}^{p\times m})$. In this case, the Ito isometry yields

$$\mathbf{E}\left(|{\int }_{-\infty }^{t}f(t-\tau )\mathrm{d}W\left(\tau \right){|}^2\right)={\int }_{{\mathbb{R}}_{+}}{\parallel f\left(\tau \right)\parallel }_{\mathrm{F}}^2\mathrm{d}\tau <+\infty$$

(2)

for any time $t\in \mathbb{R}$, where $\mathbf{E}(·)$ is expectation, and ${\parallel ·\parallel }_{\mathrm{F}}$ is the Frobenius norm of matrices [27]. The increments of such a process W have the identity covariance operator on the Hilbert space $L^2(\mathbb{R},{\mathbb{R}}^m)$ in the sense that the bilinear form

$$H(\phi ,\psi ):=\mathbf{E}\left({\int }_{\mathbb{R}}\phi {\left(s\right)}^{\mathrm{T}}\mathrm{d}W\left(s\right){\int }_{\mathbb{R}}\psi {\left(t\right)}^{\mathrm{T}}\mathrm{d}W\left(t\right)\right)$$

(3)

reduces to the usual inner product

$$H(\phi ,\psi )={\int }_{\mathbb{R}}\phi {\left(t\right)}^{\mathrm{T}}\psi \left(t\right)\mathrm{d}t,\phi ,\psi \in L^2(\mathbb{R},{\mathbb{R}}^m),$$

(4)

with (2) being a corollary of (4). Otherwise, that is, when W is not a standard Wiener process, its covariance structure can be more complicated, and the equalities (2), (4) are no longer valid.

If $f\left(\tau \right)$ is a linear combination of quasi-polynomials [19] which decay exponentially fast, as $\tau \to +\infty$, then there exist constant matrices

$$A\in {\mathbb{R}}^{n\times n},B\in {\mathbb{R}}^{n\times m},C\in {\mathbb{R}}^{p\times n},$$

(5)

with A Hurwitz, such that

$$f\left(\tau \right)=C{A}^{-1}{\mathrm{e}}^{\tau A}B,\tau ⩾0.$$

(6)

Then the Ito stochastic differential of the system output process Z in (1) acquires the form

$$\begin{array}{cc}\hfill \mathrm{d}Z\left(t\right)& =\left({\int }_{-\infty }^t{f}^{\prime }(t-\tau )\mathrm{d}W\left(\tau \right)\right)\mathrm{d}t+(f\left(0\right)+f_0)\mathrm{d}W\left(t\right)\hfill \\ \hfill & =CX\left(t\right)\mathrm{d}t+D\mathrm{d}W\left(t\right),\hfill \end{array}$$

(7)

where

$$X\left(t\right):={\int }_{-\infty }^t{\mathrm{e}}^{(t-\tau )A}B\mathrm{d}W\left(\tau \right),$$

(8)

and

$$D:=f_0+C{A}^{-1}B$$

(9)

is the feedthrough $(p\times m)$-matrix. The relation (7), which can also be obtained by using the integration by parts

$${\int }_{-\infty }^{t}f(t-\tau )\mathrm{d}W\left(\tau \right)=f\left(0\right)W\left(t\right)+{\int }_{-\infty }^t{f}^{\prime }(t-\tau )W\left(\tau \right)\mathrm{d}\tau ,$$

shows that the corresponding impulse response of the system is given by the time derivative of (6):

$$f^{\prime }\left(\tau \right)=C{\mathrm{e}}^{\tau A}B,\tau ⩾0.$$

(10)

The ${\mathbb{R}}^n$-valued Ito process X in (8) is an internal state of the system F and satisfies the linear SDE

$$\mathrm{d}X\left(t\right)=AX\left(t\right)\mathrm{d}t+B\mathrm{d}W\left(t\right)$$

(11)

whose solution on a time interval $[s,+\infty )$, with an arbitrary start time $s\in \mathbb{R}$, is given by

$$X\left(t\right)={\mathrm{e}}^{(t-s)A}X\left(s\right)+{\int }_s^t{\mathrm{e}}^{(t-\tau )A}B\mathrm{d}W\left(\tau \right),t⩾s.$$

(12)

As opposed to the finite-dimensional setting (5)–(12), the system state can, in general, be essentially infinite-dimensional, as in the case of systems with delay [30]. For example, such a system arises if the drift $AX\left(t\right)$ in (11) is replaced with a linear function of the past history of the process X:

$$\mathrm{d}X\left(t\right)=\left({\int }_{{\mathbb{R}}_{+}}\vartheta \left(\mathrm{d}\tau \right)X(t-\tau )\right)\mathrm{d}t+B\mathrm{d}W\left(t\right),$$

(13)

where $\vartheta$ is an ${\mathbb{R}}^{n\times n}$-valued countably additive measure on the $\sigma$-algebra of Borel subsets of ${\mathbb{R}}_{+}$, and the integral is understood in the pathwise sense. The SDE (11) is a particular case of (13) with an atomic measure $\vartheta$ concentrated at the origin as $\vartheta \left(S\right)={\chi }_S\left(0\right)A$, where ${\chi }_S$ is the indicator function of a set S. More generally, if $\vartheta$ is of bounded support, so that $\mathrm{supp}\vartheta \subset [0,r]$ for some $r⩾0$, then the effective state of the system at time t is the past history of X over the time interval $[t-r,t]$.

Returning to the SDEs (7), (11), we note that there also are alternative ways to model the input-output operator. For example, instead of (7), the output channel can be (and often is) described by a different relation:

$$Z=CX.$$

However, in view of (11), it can also be represented in the form (7) with appropriately modified matrices as

$$\mathrm{d}Z=\tilde{C}X\mathrm{d}t+D\mathrm{d}W,\tilde{C}:=CA,D:=CB.$$

Furthermore, a convenient feature of the representation (7) is that both the input W and the output Z enter the SDEs (7), (11) in a unified fashion (through their Ito increments), thus allowing such systems to be easily concatenated.

3. Random Disturbances Under Consideration

In what follows, we will be concerned with a setting where the ${\mathbb{R}}^m$-valued input W of the system F does not necessarily obey the nominal model assumption of being a standard Wiener process. Rather, W can be a more general Ito process with respect to a standard Wiener process V in ${\mathbb{R}}^m$ as the primary source of randomness. Therefore, the ${\mathbb{R}}^n$-valued state X and the ${\mathbb{R}}^p$-valued output Z, which are driven by W according to the SDEs (7), (11), are also Ito processes with respect to V.

More precisely, the process W is assumed to be the output of a causal LCTI system G playing the role of a noise shaping filter with a static gain matrix $g_0\in {\mathbb{R}}^{m\times m}$ and a square integrable step response $g\in L^2({\mathbb{R}}_{+},{\mathbb{R}}^{m\times m})$:

$$W\left(t\right)={\int }_{-\infty }^{t}g(t-\tau )\mathrm{d}V\left(\tau \right)+g_{0}V\left(t\right),$$

(14)

see Figure 1.

Similarly to the system F, the filter G can also have a finite-dimensional state-space realization

$$\mathrm{d}\xi =\alpha \xi \mathrm{d}t+\beta \mathrm{d}V,\mathrm{d}W=c\xi \mathrm{d}t+d\mathrm{d}V$$

(15)

(the time arguments are omitted for the sake of brevity), where $\xi$ is an ${\mathbb{R}}^{\nu }$-valued Ito process, and

$$\alpha \in {\mathbb{R}}^{\nu \times \nu },\beta \in {\mathbb{R}}^{\nu \times m},c\in {\mathbb{R}}^{m\times \nu },d\in {\mathbb{R}}^{m\times m},$$

(16)

with $\alpha$ Hurwitz. In this case, similarly to (6), (9), the static gain matrix and the step response in (14) are given by

$$g_0=d-c{\alpha }^{-1}\beta ,g\left(\tau \right)=c{\alpha }^{-1}{\mathrm{e}}^{\tau \alpha }\beta ,\tau ⩾0.$$

(17)

The presence of the drift $c\xi$ in (15) makes the noise process W temporally “coloured”, so that its increments on disjoint time intervals acquire statistical dependence. Also, nonzero off-diagonal entries of the diffusion matrix $d{d}^{\mathrm{T}}$ (which, in general, is nondiagonal) make the Ito increment of W spatially correlated in the sense of dependence between its components. Moreover, the combined effect of the drift and diffusion can lead to dependence between the components of W itself.

We denote the state-space realizations of the system F in (7), (11) and the noise shaping filter G in (15) as

$$F\leftrightarrow \left[\begin{array}{cc}A& B\\ C& D\end{array}\right],G\leftrightarrow \left[\begin{array}{cc}\alpha & \beta \\ c& d\end{array}\right].$$

The resulting LCTI system $FG$ in Figure 1 has the same output Z as the system F and is driven by the standard Wiener process V at the input of the noise shaping filter G. Its equivalent state-space realizations

$$FG\leftrightarrow \left[\begin{array}{ccc}\alpha & 0& \beta \\ Bc& A& Bd\\ Dc& C& Dd\end{array}\right]\leftrightarrow \left[\begin{array}{ccc}A& Bc& Bd\\ 0& \alpha & \beta \\ C& Dc& Dd\end{array}\right]$$

(18)

are expressed in terms of the matrices (5), (9), (16) and involve the corresponding augmented state processes

$$\left[\begin{array}{c}\xi \\ X\end{array}\right],\left[\begin{array}{c}X\\ \xi \end{array}\right].$$

If not only the system F is known precisely, but so also is the noise shaping filter G, then the coloured noise setting can be reduced to the case of the standard Wiener process V at the input by replacing F with $FG$ and thus incorporating the dynamics of the system F and that of the external disturbance W in the augmented state space ${\mathbb{R}}^{n+\nu }$. Therefore, the colouredness of W as such does not lead to statistical uncertainty. Actual uncertainty arises when G is known imprecisely, in which case, the augmented system $FG$ acquires internal uncertainty.

Note that there is an important class of noise shaping filters G, which depend on the system F and do not lead to the state augmentation in $FG$ mentioned above. More precisely, if G is organized as a “parasitic” feedback specified by matrices $L\in {\mathbb{R}}^{m\times n}$ and $M\in {\mathbb{R}}^{m\times m}$ as

$$\mathrm{d}W=LX\mathrm{d}t+M\mathrm{d}V,$$

(19)

whereby the internal state X of the system F gives rise to the drift $LX$ in the disturbance W (see Figure 2), then substitution of (19) into (7), (11) yields

$$FG\leftrightarrow \left[\begin{array}{cc}A+BL& BM\\ C+DL& DM\end{array}\right].$$

(20)

In this case, the system F and the noise shaping filter G share the common state X in ${\mathbb{R}}^n$, thus making the state-space realizations (18) nonminimal. Despite being special, the noise structure (19) is typical for the worst-case random disturbances in the framework of anisotropy-based approach to system robustness with respect to statistically uncertain noise. This is discussed in Section 8, Section 9 and related to its discrete-time counterpart [60] through the conformal correspondence described in Section 7. Also, the above example blurs the line between internal uncertainties in the system and uncertainties in the dynamics of external disturbances.

Being a Gaussian Markov process with almost surely continuous sample paths and independent stationary increments with the covariance matrix

$$\mathbf{cov}(V\left(s\right)-V\left(t\right))=|s-t|I_m,s,t\in \mathbb{R},$$

(21)

in accordance with the Ito isometry (here, $I_m$ is the identity matrix of order m), the standard Wiener process V has almost surely nowhere differentiable sample paths of unbounded variation and nonzero quadratic variation over any nondegenerate time interval [29]. Such nonsmoothness is inherited by the Ito processes Z, X, W in (7), (8), (14). Furthermore, in the case of nonzero static gain matrices $f_0$, $g_0$, the variances of Z, W grow unboundedly over time (despite the stability of the system F and the noise shaping filter G due to their dynamics matrices A, $\alpha$ being Hurwitz). We will therefore consider filtered versions of these processes.

4. Filtered Processes and Transfer Functions

As a more regular (in the sense of the variances) representation of the input and output processes V, W, Z of the system F and the noise shaping filter G in Figure 1, consider their filtered versions $V_T$, $W_T$, $Z_T$ governed by SDEs

$$\begin{array}{cc}\hfill \mathrm{d}{V}_T& =-\Omega V_T\mathrm{d}t+\sqrt{2\Omega }\mathrm{d}V,\hfill \end{array}$$

(22)

$$\begin{array}{cc}\hfill \mathrm{d}{W}_T& =-\Omega W_T\mathrm{d}t+\sqrt{2\Omega }\mathrm{d}W,\hfill \end{array}$$

(23)

$$\begin{array}{cc}\hfill \mathrm{d}{Z}_T& =-\Omega Z_T\mathrm{d}t+\sqrt{2\Omega }\mathrm{d}Z.\hfill \end{array}$$

(24)

Here, $T>0$ is an auxiliary time scale parameter which gives rise to the frequency parameter

$$\Omega :={\displaystyle \frac{1}{T}}.$$

(25)

With V being a standard Wiener process (to which W in (14) reduces when $g=0$, $g_0=I_m$), the SDE (22) has a unique invariant measure in the form of the standard normal distribution and generates an OU process [29] in ${\mathbb{R}}^m$. The latter is a zero-mean stationary Gaussian diffusion process with the covariance function

$$\mathbf{cov}(V_T\left(s\right),V_T\left(t\right))={\mathrm{e}}^{-\Omega |s-t|}{I}_m,s,t\in \mathbb{R},$$

(26)

so that the variance

$$\mathbf{E}\left(\right|V_T\left(t\right){|}^2)=m$$

remains constant, and T quantifies the correlation time of the process. However, since we will be concerned with the infinite-horizon asymptotic behaviour of stable systems, it is convenient to endow (22)–(24) with zero initial conditions

$$V_T\left(0\right)=0,W_T\left(0\right)=0,Z_T\left(0\right)=0.$$

(27)

Then the filtered processes $V_T$, $W_T$, $Z_T$ are represented in terms of V, W, Z at any time $t⩾0$ as

$$\begin{array}{cc}\hfill V_T\left(t\right)& ={\int }_0^t\varphi (t-\tau )\mathrm{d}V\left(\tau \right),\hfill \end{array}$$

(28)

$$\begin{array}{cc}\hfill W_T\left(t\right)& ={\int }_0^t\varphi (t-\tau )\mathrm{d}W\left(\tau \right),\hfill \end{array}$$

(29)

$$\begin{array}{cc}\hfill Z_T\left(t\right)& ={\int }_0^t\varphi (t-\tau )\mathrm{d}Z\left(\tau \right),\hfill \end{array}$$

(30)

with

$$\varphi \left(\tau \right):=\sqrt{2\Omega }{\mathrm{e}}^{-\Omega \tau },\tau ⩾0,$$

(31)

which makes them only asymptotically stationary in the long run. The asymptotic input-output properties of the system F, governed by (7), (11) with a Hurwitz dynamics matrix A, and those of the noise shaping filter G in (14) can be formulated in terms of the transfer functions

$$F\left(s\right):=C{(s{I}_n-A)}^{-1}B+D$$

(32)

(analytic in a neighbourhood of the closed right half-plane ${\mathbb{C}}_{+}:=\{s\in \mathbb{C}:\mathrm{Re}s⩾0\}$), and

$$G\left(s\right):={\int }_{{\mathbb{R}}_{+}}{\mathrm{e}}^{-st}{g}^{\prime }\left(t\right)\mathrm{d}t+d=s{\int }_{{\mathbb{R}}_{+}}{\mathrm{e}}^{-st}g\left(t\right)\mathrm{d}t+g_0$$

(33)

(with a slight abuse of notation, we identify the LCTI systems under consideration with their transfer functions). Here, for simplicity, the noise shaping filter G is also assumed to be a stable system with a finite-dimensional state $\xi$, described by (15)–(17), so that (33) is a rational function

$$G\left(s\right)=c{(s{I}_{\nu }-\alpha )}^{-1}\beta +d,$$

(34)

analytic in a neighbourhood of ${\mathbb{C}}_{+}$. The transfer function G factorizes the spectral density $S:\mathbb{R}\to {\mathbb{H}}_m^{+}$ for the Gaussian process W from (15) with zero-mean stationary increments as

$$S\left(\omega \right):=G\left(i\omega \right)G{\left(i\omega \right)}^{*},\omega \in \mathbb{R},$$

(35)

where ${\mathbb{H}}_m^{+}$ is the set of complex positive semi-definite Hermitian matrices of order m, and ${(·)}^{*}:={\left(\overline{(·)}\right)}^{\mathrm{T}}$ is the complex conjugate transpose. Their covariance structure, described by (3), is represented as

$$H(\phi ,\psi )={\displaystyle \frac{1}{2\pi }}{\int }_{\mathbb{R}}\widehat{\phi }{\left(\omega \right)}^{*}S\left(\omega \right)\widehat{\psi }\left(\omega \right)\mathrm{d}\omega$$

(36)

for any functions $\phi ,\psi \in L^2(\mathbb{R},{\mathbb{R}}^m)$ in terms of the Fourier transforms

$$\widehat{\phi }\left(\omega \right):={\int }_{\mathbb{R}}{\mathrm{e}}^{-i\omega t}\phi \left(t\right)\mathrm{d}t,\widehat{\psi }\left(\omega \right):={\int }_{\mathbb{R}}{\mathrm{e}}^{-i\omega t}\psi \left(t\right)\mathrm{d}t,$$

with the nominal standard Wiener process model (4) being a particular case of (36) when $S=I_m$.

Similarly to deterministic linear systems, the transfer functions F, G specify the linear relations

$$\widehat{Z}\left(s\right)=F\left(s\right)\widehat{W}\left(s\right),\widehat{W}\left(s\right)=G\left(s\right)\widehat{V}\left(s\right)$$

(37)

between the Laplace transforms of the input and output in the case of zero initial states $X\left(0\right)=0$ and $\xi \left(0\right)=0$:

$$\begin{array}{cc}\hfill \widehat{V}\left(s\right)& :={\int }_{{\mathbb{R}}_{+}}{\mathrm{e}}^{-st}\mathrm{d}V\left(t\right),\hfill \end{array}$$

(38)

$$\begin{array}{cc}\hfill \widehat{W}\left(s\right)& :={\int }_{{\mathbb{R}}_{+}}{\mathrm{e}}^{-st}\mathrm{d}W\left(t\right),\hfill \end{array}$$

(39)

$$\begin{array}{cc}\hfill \widehat{Z}\left(s\right)& :={\int }_{{\mathbb{R}}_{+}}{\mathrm{e}}^{-st}\mathrm{d}Z\left(t\right),\hfill \end{array}$$

(40)

where the Ito integrals are well-defined for any $s\in \mathbb{C}$ with $\mathrm{Re}s>0$. The same relations, as in (37),

$${\widehat{Z}}_T\left(s\right)=F\left(s\right){\widehat{W}}_T\left(s\right),{\widehat{W}}_T\left(s\right)=G\left(s\right){\widehat{V}}_T\left(s\right)$$

(41)

hold for the usual Laplace transforms (as opposed to the Ito incremental ones in (38)–(40))

$$\begin{array}{cc}\hfill {\widehat{V}}_T\left(s\right)& :={\int }_{{\mathbb{R}}_{+}}{\mathrm{e}}^{-st}{V}_T\left(t\right)\mathrm{d}t=\Phi \left(s\right)\widehat{V}\left(s\right),\hfill \end{array}$$

(42)

$$\begin{array}{cc}\hfill {\widehat{W}}_T\left(s\right)& :={\int }_{{\mathbb{R}}_{+}}{\mathrm{e}}^{-st}{W}_T\left(t\right)\mathrm{d}t=\Phi \left(s\right)\widehat{W}\left(s\right),\hfill \end{array}$$

(43)

$$\begin{array}{cc}\hfill {\widehat{Z}}_T\left(s\right)& :={\int }_{{\mathbb{R}}_{+}}{\mathrm{e}}^{-st}{Z}_T\left(t\right)\mathrm{d}t=\Phi \left(s\right)\widehat{Z}\left(s\right)\hfill \end{array}$$

(44)

of the filtered processes $V_T$, $W_T$, $Z_T$ in (28)–(30) as illustrated by Figure 3, where the transfer function

$$\Phi \left(s\right):={\int }_{{\mathbb{R}}_{+}}{\mathrm{e}}^{-st}\varphi \left(t\right)\mathrm{d}t={\displaystyle \frac{\sqrt{2\Omega }}{s+\Omega }}={\displaystyle \frac{\sqrt{2T}}{1+Ts}},\mathrm{Re}s>-\Omega ,$$

(45)

specifies a low-pass filter with the impulse response $\varphi$ in (31) and the cutoff frequency $\Omega$ in (25).

Indeed, in view of (42)–(45), the relations (41) are equivalent to (37) since the scalar-valued function $\Phi$ is nonzero everywhere and commutes with any transfer matrix. In the linear time invariant setting, this commutativity is equivalent to the commutativity of the corresponding input-output operators.

Note that, although the filtered processes $Z_T$, $W_T$, $V_T$ are related by the same input-output operators F, G as Z, W, V, their dynamics (represented by the lower part of the diagram in Figure 3) are governed in state space by pathwise ordinary differential equations (ODEs)

$$\begin{array}{cc}\hfill {\dot{X}}_T& =A{X}_T+B{W}_T,Z_T=C{X}_T+D{W}_T,\hfill \end{array}$$

(46)

$$\begin{array}{cc}\hfill {\dot{\xi }}_T& =\alpha {\xi }_T+\beta V_T,W_T=c{\xi }_T+d{V}_T,\hfill \end{array}$$

(47)

where $\dot{\left(\right)}$ is the time derivative. Accordingly, the filtered state processes $X_T$, ${\xi }_T$ have almost surely continuously differentiable sample paths and are related to the original states X, $\xi$ (see the upper part of the diagram in Figure 3) by the ODEs

$${\dot{X}}_T=-\Omega X_T+\sqrt{2\Omega }X,{\dot{\xi }}_T=-\Omega {\xi }_T+\sqrt{2\Omega }\xi ,$$

(48)

where the low-pass filter $\Phi$ in the usual ODE form (instead of the incremental SDE structure) does provide a smoothing effect. These ODEs are also endowed with zero initial conditions

$$X_T\left(0\right)=0,{\xi }_T\left(0\right)=0.$$

(49)

In the nominal white noise regime, when W reproduces the standard Wiener process V, so that $c=0$, $d=I_m$ in (15), and $W_T=V_T$ in accordance with the second equality in (47), the filtered system dynamics (46) are driven by the OU process $V_T$.

The above mentioned smoothing effect manifests itself in that, upon appropriate scaling, the filtered state processes $X_T$, ${\xi }_T$ asymptotically reproduce the original state processes X, $\xi$ in the sense that

$${\displaystyle \frac{1}{\sqrt{2T}}}\left[\begin{array}{c}{X}_T\left(t\right)\\ {\xi }_T\left(t\right)\end{array}\right]=\Omega {\int }_0^t{\mathrm{e}}^{\Omega (\tau -t)}\left[\begin{array}{c}X\left(\tau \right)\\ \xi \left(\tau \right)\end{array}\right]\mathrm{d}\tau \stackrel{\mathrm{a}.\mathrm{s}.}{⟶}\left[\begin{array}{c}X\left(t\right)\\ \xi \left(t\right)\end{array}\right],\mathrm{as}T\to 0+,$$

(50)

with $\Omega$ depending on T according to (25). Here, the almost sure convergence to $X\left(t\right)$, $\xi \left(t\right)$ holds for any fixed but otherwise arbitrary $t⩾0$ since X, $\xi$ have almost surely continuous sample paths (which are therefore bounded on any finite time interval) and due to the weak convergence [7] of the exponential distribution with the probability density function (PDF)

$$\sqrt{{\displaystyle \frac{\Omega }{2}}}\varphi \left(\tau \right)=\Omega {\mathrm{e}}^{-\Omega \tau },\tau ⩾0$$

(arising from (31)) to the atomic probability measure on Borel subsets of ${\mathbb{R}}_{+}$ concentrated at the origin 0, as $\Omega \to +\infty$.

Although the processes $V_T$, $W_T$, $Z_T$ in (22)–(24) inherit nonsmoothness of sample paths from V, W, Z, they capture (approximately) only relatively slow components of the original processes (with frequencies $\ll \Omega$ or, equivalently, time scales $\gg T$). Since practical applications are usually concerned with the system behaviour in a bounded frequency range (specified by the cutoff frequency $\Omega$), we will describe the sensitivity of the system output to the input noise in terms of the filtered processes by taking advantage of the smoothing effect of their ODE dynamics (46)–(48).

5. Root-Mean-Square Gain for Filtered Processes

For a given value of the time scale parameter $T>0$, the RMS gain of the system F under consideration can be quantified in terms of the filtered input and output $W_T$, $Z_T$ in (23), (24) by

$$\left[\right]F\left[\right]:=\underset{t\to +\infty }{lim\; sup}\sqrt{{\displaystyle \frac{{\int }_0^t\mathbf{E}\left(\right|Z_T\left(\tau \right){|}^2)\mathrm{d}\tau }{{\int }_0^t\mathbf{E}\left(\right|W_T\left(\tau \right){|}^2)\mathrm{d}\tau }}}$$

(51)

(the trivial case of zero inputs is not considered). The following theorem, which is a continuous-time counterpart of [58, Lemma 2], shows that the upper limit in (51) is in fact a limit. Its formulation employs an auxiliary function

$$\Sigma \left(\omega \right):={|\Phi \left(i\omega \right)|}^2={\displaystyle \frac{2\Omega }{{\Omega }^2+{\omega }^2}}={\displaystyle \frac{2T}{1+{\left(\omega T\right)}^2}},\omega \in \mathbb{R},$$

(52)

which is associated with (45). The scalar function $\Sigma$ specifies the spectral density $\Sigma I_m$ of the OU process $V_T$ in ${\mathbb{R}}^m$ from (22) as the Fourier transform of the covariance function (26) in the stationary regime:

$$\Sigma \left(\omega \right)={\int }_{\mathbb{R}}{\mathrm{e}}^{-i\omega t-\Omega \left|t\right|}\mathrm{d}t.$$

(53)

In a similar fashion, the functions $\Sigma \left(\omega \right)S\left(\omega \right)$ and $\Sigma \left(\omega \right)F\left(i\omega \right)S\left(\omega \right)F{\left(i\omega \right)}^{*}$, with $\omega \in \mathbb{R}$, describe the spectral densities of the filtered noise and output processes $W_T$, $Z_T$ in (23), (24), respectively, where S is the spectral density (35) for the increments of the process W in the sense of (3), (36).

Theorem 1. 

Suppose the system F in (7), (11) and the noise shaping filter G in (15) are stable. Then the RMS gain (51) holds as a limit and is computed as

$$\left[\right]F\left[\right]=\sqrt{{\displaystyle \frac{{lim}_{t\to +\infty }\mathbf{E}\left(\right|Z_T\left(t\right){|}^2)}{{lim}_{t\to +\infty }\mathbf{E}\left(\right|W_T\left(t\right){|}^2)}}}=\sqrt{{\displaystyle \frac{{\int }_{\mathbb{R}}\Sigma \left(\omega \right)\langle \Lambda \left(\omega \right),S\left(\omega \right)\rangle \mathrm{d}\omega }{{\int }_{\mathbb{R}}\Sigma \left(\omega \right)\mathrm{Tr}S\left(\omega \right)\mathrm{d}\omega }}}.$$

(54)

Here, the function Σ is given by (52), $\langle ·,·\rangle$ is the Frobenius inner product of matrices, and the spectral density S of the process W and the function $\Lambda :\mathbb{R}\to {\mathbb{H}}_m^{+}$ are associated with the transfer functions (32)–(34) by (35) and

$$\begin{array}{cc}\hfill \Lambda \left(\omega \right)& :=F{\left(i\omega \right)}^{*}F\left(i\omega \right),\omega \in \mathbb{R}.\hfill \end{array}$$

(55)

Proof. 

In view of (15), the solution (29) of the SDE (23) with the zero initial condition from (27) takes the form

$$\begin{array}{cc}\hfill W_T\left(t\right)& ={\int }_0^t\varphi (t-\tau )(c\xi \left(\tau \right)\mathrm{d}\tau +d\mathrm{d}V\left(\tau \right))\hfill \\ \hfill & ={\int }_0^t\varphi (t-\tau )\left(c{\int }_0^{\tau }{\mathrm{e}}^{(\tau -u)\alpha }\beta \mathrm{d}V\left(\mathrm{d}u\right)\mathrm{d}\tau +d\mathrm{d}V\left(\tau \right)\right)\hfill \\ \hfill & ={\int }_0^{t}h(t-u)\mathrm{d}V\left(u\right).\hfill \end{array}$$

(56)

Here,

$$\begin{array}{cc}\hfill h\left(\tau \right)& :=(\varphi *\psi )\left(\tau \right)={\int }_0^{\tau }\varphi (\tau -u)\psi \left(u\right)\mathrm{d}u\hfill \\ \hfill & ={\int }_0^{\tau }\varphi (\tau -u)c{\mathrm{e}}^{u\alpha }\beta \mathrm{d}u+\varphi \left(\tau \right)d,\tau ⩾0,\hfill \end{array}$$

(57)

is the convolution of the impulse responses of the low-pass filter $\Phi$ and the noise shaping filter G, which are given by (31) and

$$\psi \left(\tau \right):=c{\mathrm{e}}^{\tau \alpha }\beta +d\delta \left(\tau \right),\tau ⩾0,$$

with $\delta$ the Dirac delta function [57]. Since V is a standard Wiener process in ${\mathbb{R}}^m$, application of the Ito isometry to (56) leads to

$$\mathbf{E}\left(\right|W_T{\left(t\right)|}^2)={\int }_0^t{\parallel h\left(u\right)\parallel }_{\mathrm{F}}^2\mathrm{d}u,t⩾0.$$

(58)

Due to the square integrability of the transfer function $\Phi G$ (which is the Laplace transform of the function $h:{\mathbb{R}}_{+}\to {\mathbb{R}}^{m\times m}$ from (57)) over the imaginary axis combined with the Plancherel theorem, the function h is square integrable over ${\mathbb{R}}_{+}$, and (58) leads to the monotonic convergence

$$\begin{array}{cc}\hfill \underset{t\to +\infty }{lim}\mathbf{E}\left(\right|W_T\left(t\right){|}^2)& ={\int }_{{\mathbb{R}}_{+}}{\parallel h\left(u\right)\parallel }_{\mathrm{F}}^2\mathrm{d}u\hfill \\ \hfill & ={\displaystyle \frac{1}{2\pi }}{\int }_{\mathbb{R}}{|\Phi \left(i\omega \right)|}^2{\parallel G\left(i\omega \right)\parallel }_{\mathrm{F}}^2\mathrm{d}\omega \hfill \\ \hfill & ={\displaystyle \frac{1}{2\pi }}{\int }_{\mathbb{R}}\Sigma \left(\omega \right)\mathrm{Tr}S\left(\omega \right)\mathrm{d}\omega \hfill \end{array}$$

in view of (52), (35). The same limit is shared by the Cesaro means of $\mathbf{E}\left(\right|W_T\left(\tau \right){|}^2)$ as a function of $\tau ⩾0$:

$$\begin{array}{cc}\hfill \underset{t\to +\infty }{lim}\left({\displaystyle \frac{1}{t}}{\int }_0^t\mathbf{E}\left(\right|W_T\left(\tau \right){|}^2)\mathrm{d}\tau \right)& =\underset{t\to +\infty }{lim}\mathbf{E}\left(\right|W_T\left(t\right){|}^2)\hfill \\ \hfill & ={\displaystyle \frac{1}{2\pi }}{\int }_{\mathbb{R}}\Sigma \left(\omega \right)\mathrm{Tr}S\left(\omega \right)\mathrm{d}\omega .\hfill \end{array}$$

(59)

By applying a similar reasoning to the filtered system output $Z_T$ governed by (24), it follows that

$$\begin{array}{cc}\hfill \underset{t\to +\infty }{lim}\left({\displaystyle \frac{1}{t}}{\int }_0^t\mathbf{E}\left(\right|Z_T\left(\tau \right){|}^2)\mathrm{d}\tau \right)& =\underset{t\to +\infty }{lim}\mathbf{E}\left(\right|Z_T\left(t\right){|}^2)\hfill \\ \hfill & ={\displaystyle \frac{1}{2\pi }}{\int }_{\mathbb{R}}\Sigma \left(\omega \right){\parallel F\left(i\omega \right)G\left(i\omega \right)\parallel }_{\mathrm{F}}^2\mathrm{d}\omega \hfill \\ \hfill & ={\displaystyle \frac{1}{2\pi }}{\int }_{\mathbb{R}}\Sigma \left(\omega \right)\mathrm{Tr}\left(F\left(i\omega \right)S\left(\omega \right)F{\left(i\omega \right)}^{*}\right)\mathrm{d}\omega \hfill \\ \hfill & ={\displaystyle \frac{1}{2\pi }}{\int }_{\mathbb{R}}\Sigma \left(\omega \right)\langle \Lambda \left(\omega \right),S\left(\omega \right)\rangle \mathrm{d}\omega ,\hfill \end{array}$$

(60)

where (55) is also used. A combination of (59), (60) implies the existence of a limit in (51) which is given by (54). □

The RMS gain $\left[\right]F\left[\right]$ in (54) is a semi-norm of the system transfer function F in (32). It is well-defined for any bounded (and not necessarily integrable) spectral density S of the noise (the trivial case when $S=0$ almost everywhere is not considered). This is secured by the presence of the integrable function $\Sigma$ from (52) as a factor in the integrands. The dependence of the RMS gain on the time scale parameter $T>0$ and the noise spectral density S will be indicated by the subscripts as

$$\left[\right]F\left[\right]=\left[\right]F{\left[\right]}_{T,S}.$$

Note that $\left[\right]F{\left[\right]}_{T,S}$ is invariant under the scaling $S\mapsto \kappa S$ with an arbitrary constant $\kappa >0$ (which cancels out in the numerator and denominator in (54)):

$$\left[\right]F{\left[\right]}_{T,\kappa S}=\left[\right]F{\left[\right]}_{T,S}.$$

(61)

Also, if the system F is isometric (that is, inner) up to a nonzero multiplicative constant, so that there exists $\lambda >0$ such that the function (55) satisfies $\Lambda \left(\omega \right)=\lambda I_m$ for all $\omega \in \mathbb{R}$, then the RMS gain reduces to

$$\left[\right]F{\left[\right]}_{T,S}=\sqrt{{\displaystyle \frac{{\int }_{\mathbb{R}}\Sigma \left(\omega \right)\langle \lambda I_m,S\left(\omega \right)\rangle \mathrm{d}\omega }{{\int }_{\mathbb{R}}\Sigma \left(\omega \right)\mathrm{Tr}S\left(\omega \right)\mathrm{d}\omega }}}=\sqrt{\lambda }$$

(62)

and does not depend on the noise spectral density S. In what follows, such systems F will be referred to as round systems (with all the other systems being called nonround). In general, application of the inequalities

$${\lambda }_{min}\left(L\right)\mathrm{Tr}M⩽\langle L,M\rangle ⩽{\lambda }_{max}\left(L\right)\mathrm{Tr}M,$$

which hold for arbitrary Hermitian matrices L, M of equal dimensions with $M\succcurlyeq 0$, leads to

$$\underset{\omega \in \mathbb{R}}{inf}\sqrt{{\lambda }_{min}(\Lambda \left(\omega \right))}⩽\left[\right]F{\left[\right]}_{T,S}⩽\underset{\omega \in \mathbb{R}}{sup}\sqrt{{\lambda }_{max}(\Lambda \left(\omega \right))}={\parallel F\parallel }_{\infty }.$$

(63)

Here, ${\lambda }_{min}(·)$, ${\lambda }_{max}(·)$ are the smallest and largest eigenvalues of a Hermitian matrix, and ${\parallel ·\parallel }_{\infty }$ is the ${\mathcal{H}}_{\infty }$-norm [20] of a transfer function, so that (62) is a particular case of (63). If the input and output dimensions of the system satisfy $m>p$, then the matrix $\Lambda \left(\omega \right)$ in (55) is singular (at every frequency $\omega \in \mathbb{R}$), thus making the left-hand side of (63) vanish. For arbitrary dimensions, the RMS gain $\left[\right]F{\left[\right]}_{T,S}$ can approach any intermediate value in (63) by an appropriate choice of the spectral density S as discussed below.

Theorem 2. 

Suppose the system F in (7), (11) is stable. Then for any μ from the interval in (63), that is, satisfying

$$\underset{\omega \in \mathbb{R}}{inf}\sqrt{{\lambda }_{min}(\Lambda \left(\omega \right))}⩽\mu ⩽{\parallel F\parallel }_{\infty },$$

and any $ϵ>0$, there exists a rational spectral density S (associated with a stable noise shaping filter G in (15)) such that

$$|\left[\right]F{\left[\right]}_{T,S}-\mu |<ϵ.$$

Proof. 

With any $\gamma >0$, a frequency ${\omega }_0\in \mathbb{R}$ and a unit vector $u\in {\mathbb{C}}^m$, we associate a rational spectral density $S_{\gamma }:\mathbb{R}\to {\mathbb{H}}_m^{+}$:

$$S_{\gamma }\left(\omega \right):={\displaystyle \frac{\gamma }{\pi }}\left({\displaystyle \frac{1}{{\gamma }^2+{(\omega -{\omega }_0)}^2}}u{u}^{*}+{\displaystyle \frac{1}{{\gamma }^2+{(\omega +{\omega }_0)}^2}}\overline{u}{u}^{\mathrm{T}}\right),$$

(64)

which satisfies ${\int }_{\mathbb{R}}\mathrm{Tr}{S}_{\gamma }\left(\omega \right)\mathrm{d}\omega =2$. Due to the weak convergence of the Cauchy distribution with the PDF

$${\displaystyle \frac{1}{\pi }}{\displaystyle \frac{\gamma }{{\gamma }^2+{(\omega -{\omega }_0)}^2}},\omega \in \mathbb{R},$$

to the atomic probability measure concentrated at ${\omega }_0$, as $\gamma \to 0+$ (and similarly for the opposite frequency $-{\omega }_0$), the spectral density (64) is convergent in the distributional sense [57] as

$$\underset{\gamma \to 0+}{lim}{S}_{\gamma }\left(\omega \right)=\delta (\omega -{\omega }_0)u{u}^{*}+\delta (\omega +{\omega }_0)\overline{u}{u}^{\mathrm{T}}.$$

(65)

Since the functions $\Sigma$, $\Lambda$ in (52), (55) are bounded and continuous, then (65) implies that

$$\begin{array}{cc}\hfill \underset{\gamma \to 0+}{lim}{\int }_{\mathbb{R}}\Sigma \left(\omega \right)\langle \Lambda \left(\omega \right),S_{\gamma }\left(\omega \right)\rangle \mathrm{d}\omega & =2\Sigma \left({\omega }_0\right){\parallel u\parallel }_{\Lambda \left({\omega }_0\right)}^2,\hfill \end{array}$$

(66)

$$\begin{array}{cc}\hfill \underset{\gamma \to 0+}{lim}{\int }_{\mathbb{R}}\Sigma \left(\omega \right)\mathrm{Tr}{S}_{\gamma }\left(\omega \right)\mathrm{d}\omega & =2\Sigma \left({\omega }_0\right),\hfill \end{array}$$

(67)

where ${\parallel v\parallel }_M:=\left|\sqrt{M}v\right|=\sqrt{v^{*}Mv}$ is a weighted Euclidean semi-norm of a complex vector v specified by an appropriately dimensioned positive semi-definite Hermitian matrix M. Here, use is also made of the assumption $\left|u\right|=1$ and the property

$$\Lambda (-\omega )=\overline{\Lambda \left(\omega \right)}=\Lambda {\left(\omega \right)}^{\mathrm{T}},$$

(68)

whereby $\Lambda (-\omega )$ is isospectral to $\Lambda \left(\omega \right)$, and $\parallel \overline{u}{\parallel }_{\Lambda (-\omega )}={\parallel u\parallel }_{\Lambda \left(\omega \right)}$. In view of (66), (67), the corresponding RMS gain in (54) satisfies

$$\underset{\gamma \to 0+}{lim}\left[\right]F{\left[\right]}_{T,S_{\gamma }}={\parallel u\parallel }_{\Lambda \left({\omega }_0\right)}\in \theta \left({\omega }_0\right),$$

(69)

where $\theta$ is an auxiliary set-valued function which maps a frequency $\omega \in \mathbb{R}$ to the interval

$$\theta \left(\omega \right):=\left[\sqrt{{\lambda }_{min}(\Lambda \left(\omega \right))},\sqrt{{\lambda }_{max}(\Lambda \left(\omega \right))}\right].$$

(70)

Any given point of $\theta \left({\omega }_0\right)$ in (69) is achievable by an appropriate choice of a unit vector $u\in {\mathbb{C}}^m$ since

$$\left\{{\parallel u\parallel }_{\Lambda \left(\omega \right)}:u\in {\mathbb{C}}^m,\left|u\right|=1\right\}=\theta \left(\omega \right)$$

for any $\omega \in \mathbb{R}$. With the endpoints of this interval being continuous even functions of $\omega$ (the continuity is inherited from $\Lambda$), the map $\theta$ in (70) covers the interior of the interval in (63) in the sense that

$$\left(\underset{\omega \in \mathbb{R}}{inf}\sqrt{{\lambda }_{min}(\Lambda \left(\omega \right))},{\parallel F\parallel }_{\infty }\right)\subset \bigcup _{\omega \in \mathbb{R}}\theta \left(\omega \right).$$

Therefore, for any $\mu$ from the interval in (63) and any $ϵ>0$, there exists ${\omega }_0\in \mathbb{R}$ and a unit vector $u\in {\mathbb{C}}^m$ such that ${|\parallel u\parallel }_{\Lambda \left({\omega }_0\right)}-\mu |<ϵ$. In view of (69), the corresponding spectral density $S_{\gamma }$ in (64), associated with such ${\omega }_0$ and u, delivers the RMS gain which satisfies $|\left[\right]F{\left[\right]}_{T,S_{\gamma }}-{\parallel u\parallel }_{\Lambda \left({\omega }_0\right)}|<ϵ$ for all sufficiently small $\gamma >0$, and hence, $|\left[\right]F{\left[\right]}_{T,S_{\gamma }}-\mu |<2ϵ$ by the triangle inequality. □

It follows from Theorem 2 that, in the absence of specific additional constraints on the spectral density S of the noise W, the second inequality in (63) cannot be improved in the sense that

$$\underset{S}{sup}\left[\right]F{\left[\right]}_{T,S}={\parallel F\parallel }_{\infty }.$$

Moreover, the supremum can be restricted (without affecting its value) to the class of rational spectral densities $S:\mathbb{R}\to {\mathbb{H}}_m^{+}$. Indeed, the proof of the theorem provides a particular way to construct a maximizing sequence of rational spectral densities $S_{\gamma }$ in (64), with u being a unit eigenvector of the matrix $\Lambda \left({\omega }_0\right)$ associated with its largest eigenvalue. Such spectral densities exhibit “energy concentration” about certain frequencies and in certain directions in ${\mathbb{C}}^m$, which depend on the system F and, being exceptional in this sense, are not necessarily “targeted” by the noise W, especially when the latter, up to scaling, is not far from the isotropic standard Wiener process.

6. RMS Gain with Respect to Isotropic White-Noise Inputs

As opposed to the energy concentration used in the proof of Theorem 2, consider an isotropic white-noise case when the spectral density of the input W is a constant scalar matrix:

$$S\left(\omega \right)=\kappa I_m,\omega \in \mathbb{R},$$

(71)

where $\kappa >0$ is a scalar parameter. In this case, W is a standard Wiener process up to the multiplicative constant $\sqrt{\kappa }$ (and can be obtained from V by letting $g=0$, $g_0=\sqrt{\kappa }{I}_m$ in (14)), so that the SDE (23) in its stationary regime makes ${\displaystyle \frac{1}{\sqrt{\kappa }}}{W}_T$ an OU process. Then, by substituting (71) into (54), the RMS gain takes the form

$$\begin{array}{cc}\hfill \left[\right]F{\left[\right]}_{T,\kappa I_m}& =\left[\right]F{\left[\right]}_{T,I_m}=\sqrt{{\displaystyle \frac{{\int }_{\mathbb{R}}\Sigma \left(\omega \right)\mathrm{Tr}\Lambda \left(\omega \right)\mathrm{d}\omega }{m{\int }_{\mathbb{R}}\Sigma \left(\omega \right)\mathrm{d}\omega }}}\hfill \\ \hfill & =\sqrt{{\displaystyle \frac{1}{2\pi m}}{\int }_{\mathbb{R}}\Sigma \left(\omega \right)\mathrm{Tr}\Lambda \left(\omega \right)\mathrm{d}\omega }\hfill \\ \hfill & =\sqrt{{\displaystyle \frac{1}{2\pi m}}{\int }_{\mathbb{T}}\mathrm{Tr}\Lambda \left(\Omega tan{\displaystyle \frac{\phi }{2}}\right)\mathrm{d}\phi },\hfill \end{array}$$

(72)

since the function $\Sigma$ in (52) satisfies

$${\int }_{\mathbb{R}}\Sigma \left(\omega \right)\mathrm{d}\omega =2\pi ,$$

(73)

in accordance with the Fourier transform (53). In (72), use is also made of the frequency transformation

$$\omega =\Omega tan{\displaystyle \frac{\phi }{2}},\phi :=2arctan\left(\omega T\right),$$

(74)

so that

$${\partial }_{\omega }\phi ={\displaystyle \frac{2T}{1+{\left(\omega T\right)}^2}}=\Sigma \left(\omega \right)$$

(75)

(or, equivalently, $\Sigma \left(\omega \right)\mathrm{d}\omega =\mathrm{d}\phi$), where the new integration variable $\phi$ takes values in the interval

$$\mathbb{T}:=(-\pi ,\pi ),$$

(76)

which represents the unit circle

$$\{{\mathrm{e}}^{i\phi }:\phi \in \mathbb{T}\}=\{z\in \mathbb{C}:|z|=1\}\setminus \{-1\}$$

in the complex plane, punctured at $-1$. The right-hand side of (72) does not depend on $\kappa$ in accordance with the scale invariance (61) and is organized as a weighted ${\mathcal{H}}_2$-norm [20] of the transfer function F, which involves the parameter $\Omega$ from (25).

We will now discuss the asymptotic behaviour of the RMS gain (72) as a function of T, when $T\to +\infty$ and $T\to 0+$.

As $T\to +\infty$ (so that $\Omega \to 0+$), the OU process $W_T$ acquires long-range correlations. In this case, the spectral density $\Sigma$ in (52) converges to $2\pi \delta$ in the distributional sense. The effect of such a process on the system is, in essence, equivalent to that of a constant input. The corresponding limit

$$\underset{T\to +\infty }{lim}\left[\right]F{\left[\right]}_{T,I_m}=\underset{\Omega \to 0+}{lim}\left[\right]F{\left[\right]}_{T,I_m}=\sqrt{{\displaystyle \frac{1}{m}}\mathrm{Tr}\Lambda \left(0\right)}={\displaystyle \frac{1}{\sqrt{m}}}{\parallel F\left(0\right)\parallel }_{\mathrm{F}},$$

(77)

which is obtained by applying Lebesgue’s dominated convergence theorem to the integral in the RMS gain (72), involves the static gain matrix $F\left(0\right)=D-C{A}^{-1}B$ of the system in view of (32).

By a similar reasoning, as the correlation time T of the OU input $W_T$ goes to zero (or, equivalently, as $\Omega \to +\infty$), the spectral density (52) becomes constant ($\Sigma \left(\omega \right)\approx 2T$) over the widening frequency interval $\left|\omega \right|\ll \Omega$, in which case, the RMS gain (72) approaches a different limit:

$$\underset{T\to 0+}{lim}\left[\right]F{\left[\right]}_{T,I_m}=\underset{\Omega \to +\infty }{lim}\left[\right]F{\left[\right]}_{T,I_m}=\sqrt{{\displaystyle \frac{1}{m}}\mathrm{Tr}{lim}_{\omega \to \infty }\Lambda \left(\omega \right)}={\displaystyle \frac{1}{\sqrt{m}}}{\parallel D\parallel }_{\mathrm{F}}.$$

(78)

The limits (77), (78) of the RMS gain (with respect to an isotropic white noise in the sense of (71)) do not reduce to the standard ${\mathcal{H}}_2$-norm

$${\parallel F\parallel }_2:=\sqrt{{\displaystyle \frac{1}{2\pi }}{\int }_{\mathbb{R}}\mathrm{Tr}\Lambda \left(\omega \right)\mathrm{d}\omega }$$

(79)

which is finite only when the system transfer function (32) is strictly proper, that is, if $D=0$. In the latter case, the limit in (78) vanishes, and the asymptotic behaviour of the RMS gain is described by

$$\left[\right]F{\left[\right]}_{T,I_m}=\sqrt{{\displaystyle \frac{T}{\pi m}}{\int }_{\mathbb{R}}{\displaystyle \frac{\mathrm{Tr}\Lambda \left(\omega \right)}{1+{\left(\omega T\right)}^2}}\mathrm{d}\omega }\sim {\parallel F\parallel }_2\sqrt{{\displaystyle \frac{2T}{m}}},\mathrm{as}T\to 0+,$$

where use is made of the last equality in (52) in combination with the limit relation

$$\underset{T\to 0}{lim}{\int }_{\mathbb{R}}{\displaystyle \frac{\mathrm{Tr}\Lambda \left(\omega \right)}{1+{\left(\omega T\right)}^2}}\mathrm{d}\omega =2\pi {\parallel F\parallel }_2^2,$$

which follows from (79) by Lebesgue’s dominated convergence theorem since $\mathrm{Tr}\Lambda \left(\omega \right)$ is integrable over $\omega \in \mathbb{R}$ for strictly proper stable systems F.

The limits (77), (78) in the isotropic white-noise case being discussed manifest themselves when the time scale parameter T of the low-pass filtering is large, or, respectively, small, in comparison with the transient times in the system F, which can be described in terms of the set

$$\mathcal{T}:=\left\{{\displaystyle \frac{1}{\left|\lambda \right|}}:\lambda \in \mathfrak{S}\right\},$$

where $\mathfrak{S}$ denotes the spectrum of the Hurwitz matrix A. The fulfillment of either of the relations

$$T\ll min\mathcal{T}={\displaystyle \frac{1}{\mathbf{r}\left(A\right)}},max\mathcal{T}=\mathbf{r}\left(A^{-1}\right)\ll T$$

(80)

(with $\mathbf{r}(·)$ the spectral radius of a square matrix) indicates whether the OU process $W_T$ is nearly white or strongly coloured for the system F, respectively. The relative simplicity of this comparison comes from the fact that the OU process has only one time scale parameter T (which specifies the characteristic width $\Omega$ of the effective frequency range for the process), whereas more complicated random inputs can have multiple time scales, similarly to the system F itself.

The behaviour of the RMS gain with respect to isotropic white-noise inputs, considered above, is qualitatively different from its discrete-time counterpart [58] despite similarities in their definitions. Indeed, in the discrete-time settings, the frequency range is finite and can be identified with the interval $\mathbb{T}$ in (76), whereas the continuous-time processes may contain arbitrarily fast components (with arbitrarily short time scales or, equivalently, high frequencies) and have the infinite frequency range.

On the other hand, the last representation in (72) is in terms of the ${\mathcal{H}}_2$-norm of a discrete-time transfer function related to (32) by a conformal correspondence. This correspondence applies not only to the isotropic white-noise case but also to a wide class of stationary Gaussian disturbances discussed below.

7. Conformal Correspondence between Continuous and Discrete Time Settings

For what follows, consider an involutive conformal map (a modified Cayley transform [48])

$$K\left(z\right)={\displaystyle \frac{1-z}{1+z}}$$

(81)

between the open unit disk and the open right half-plane in the complex plane:

$$K:\{z\in \mathbb{C}:|z|<1\}\to \{s\in \mathbb{C}:\mathrm{Re}s>0\}.$$

(82)

These two domains pertain to the discrete and continuous time settings, respectively. The map K is a smooth bijection of the punctured unit circle $\{z\in \mathbb{C}:|z|=1\}\setminus \{-1\}$ onto the imaginary axis:

$$K\left({\mathrm{e}}^{i\phi }\right)={\displaystyle \frac{{\mathrm{e}}^{-i\phi /2}-{\mathrm{e}}^{i\phi /2}}{{\mathrm{e}}^{-i\phi /2}+{\mathrm{e}}^{i\phi /2}}}=-itan{\displaystyle \frac{\phi }{2}},\phi \in \mathbb{T},$$

(83)

with $\mathbb{T}$ from (76). This property is inherited by the scaled version $\Omega K$ of the map K for any cutoff frequency $\Omega >0$ in (25). The resulting conformal map $\Omega K$ gives rise to a linear operator $\Psi$ which maps the transfer functions F, G of the system and the noise shaping filter in (32)–(34) to the transfer functions

$$\begin{array}{cc}\hfill F_T\left(z\right)& :=F(\Omega K(z\left)\right),\hfill \end{array}$$

(84)

$$\begin{array}{cc}\hfill G_T\left(z\right)& :=G(\Omega K(z\left)\right),\hfill \end{array}$$

(85)

which are analytic in the open unit disk $\{z\in \mathbb{C}:|z|<1\}$ and correspond to stable LDTI systems. Up to a factor of two in $\Omega K$, the operator $\Psi$ is identical to Tustin’s transform [8] which converts LCTI systems to LDTI systems and vice versa. Similarly to (55), (35), we associate with $F_T$, $G_T$ the functions ${\Lambda }_T,S_T:\mathbb{T}\to {\mathbb{H}}_m^{+}$ by

$$\begin{array}{cc}\hfill {\Lambda }_T\left(\phi \right)& :=F_T{\left({\mathrm{e}}^{i\phi }\right)}^{*}{F}_T\left({\mathrm{e}}^{i\phi }\right)=\Lambda \left(-\Omega tan{\displaystyle \frac{\phi }{2}}\right),\hfill \end{array}$$

(86)

$$\begin{array}{cc}\hfill S_T\left(\phi \right)& :=G_T\left({\mathrm{e}}^{i\phi }\right)G_T{\left({\mathrm{e}}^{i\phi }\right)}^{*}=S\left(-\Omega tan{\displaystyle \frac{\phi }{2}}\right),\hfill \end{array}$$

(87)

where the rightmost equalities follow from (83). The significance of these functions is clarified below by using the connection (75) between the spectral density $\Sigma$ of the OU process in (52) and the logarithmic derivative

$${(lnK\left(s\right))}^{\prime }={\displaystyle \frac{2}{s^2-1}}$$

of the conformal map K in (81) (recall that, being an involution, K coincides with its functional inverse $K^{-1}$).

Theorem 3. 

Suppose the system F in (7), (11) and the noise shaping filter G in (15) are stable. Then the RMS gain (54) can be represented in terms of the functions ${\Lambda }_T$, $S_T$ from (86), (87) as

$$\begin{array}{cc}\hfill \left[\right]F{\left[\right]}_{T,S}& =\sqrt{{\displaystyle \frac{{\int }_{\mathbb{T}}\langle {\Lambda }_T\left(\phi \right),S_T\left(\phi \right)\rangle \mathrm{d}\phi }{{\int }_{\mathbb{T}}\mathrm{Tr}{S}_T\left(\phi \right)\mathrm{d}\phi }}}.\hfill \end{array}$$

(88)

Proof. 

Similarly to (68), the spectral density S in (35) satisfies $S(-\omega )=\overline{S\left(\omega \right)}=S{\left(\omega \right)}^{\mathrm{T}}$, and hence, both $\langle \Lambda \left(\omega \right),S\left(\omega \right)\rangle =\langle \Lambda (-\omega ),S(-\omega )\rangle$ and $\mathrm{Tr}S\left(\omega \right)=\mathrm{Tr}S(-\omega )$ are even functions of $\omega \in \mathbb{R}$. In combination with the frequency transformation in (74), (75), this allows the numerator and denominator in (54) to be represented as

$$\begin{array}{cc}\hfill {\int }_{\mathbb{R}}\Sigma \left(\omega \right)\langle \Lambda \left(\omega \right),S\left(\omega \right)\rangle \mathrm{d}\omega & ={\int }_{\mathbb{R}}\Sigma \left(\omega \right)\langle \Lambda (-\omega ),S(-\omega )\rangle \mathrm{d}\omega \hfill \\ \hfill & ={\int }_{\mathbb{T}}\langle \Lambda \left(-\Omega tan{\displaystyle \frac{\phi }{2}}\right),S\left(-\Omega tan{\displaystyle \frac{\phi }{2}}\right)\rangle \mathrm{d}\phi \hfill \\ \hfill & ={\int }_{\mathbb{T}}\langle {\Lambda }_T\left(\phi \right),S_T\left(\phi \right)\rangle \mathrm{d}\phi ,\hfill \end{array}$$

(89)

$$\begin{array}{cc}\hfill {\int }_{\mathbb{R}}\Sigma \left(\omega \right)\mathrm{Tr}S\left(\omega \right)\mathrm{d}\omega & ={\int }_{\mathbb{R}}\Sigma \left(\omega \right)\mathrm{Tr}S(-\omega )\mathrm{d}\omega \hfill \\ \hfill & ={\int }_{\mathbb{T}}\mathrm{Tr}S\left(-\Omega tan{\displaystyle \frac{\phi }{2}}\right)\mathrm{d}\phi \hfill \\ \hfill & ={\int }_{\mathbb{T}}\mathrm{Tr}{S}_T\left(\phi \right)\mathrm{d}\phi ,\hfill \end{array}$$

(90)

where the integration variable change is used along with (86), (87). Substitution of (89), (90) into (54) establishes (88). □

Note that the function $S_T:\mathbb{T}\to {\mathbb{H}}_m^{+}$ in (87) is the spectral density of an auxiliary stationary zero-mean Gaussian random sequence

$$\varpi :={\left({\varpi }_k\right)}_{k\in \mathbb{Z}}$$

(91)

in ${\mathbb{R}}^m$ whose elements are indexed by the set of integers $\mathbb{Z}$. Their variance (which is constant in time due to stationarity) is

$$\begin{array}{cc}\hfill \mathbf{E}\left(\right|{\varpi }_k{|}^2)& ={\displaystyle \frac{1}{2\pi }}{\int }_{\mathbb{T}}\mathrm{Tr}{S}_T\left(\phi \right)\mathrm{d}\phi \hfill \\ \hfill & ={\displaystyle \frac{1}{2\pi }}{\int }_{\mathbb{R}}\Sigma \left(\omega \right)\mathrm{Tr}S\left(\omega \right)\mathrm{d}\omega ,k\in \mathbb{Z}.\hfill \end{array}$$

(92)

The sequence $\varpi$ is the output of an LDTI shaping filter (with the transfer function $G_T$ in (85)) driven by a Gaussian white-noise sequence

$$\upsilon :={\left({\upsilon }_k\right)}_{k\in \mathbb{Z}}$$

(93)

which consists of mutually independent standard normal random vectors in ${\mathbb{R}}^m$, so that

$$\mathbf{E}{\upsilon }_k=0,\mathbf{E}\left({\upsilon }_j{\upsilon }_k^{\mathrm{T}}\right)={\delta }_{jk}{I}_m,j,k\in \mathbb{Z},$$

(94)

where ${\delta }_{jk}$ is the Kronecker delta. At the same time, $\varpi$ from (91) is the input to an LDTI system (with the transfer function $F_T$ in (84)) whose output is a stationary zero-mean Gaussian random sequence

$$\zeta :={\left({\zeta }_k\right)}_{k\in \mathbb{Z}}$$

(95)

in ${\mathbb{R}}^p$. The resulting setup is shown in Figure 4.

Since the spectral density of the sequence $\zeta$ is given by $F_T\left({\mathrm{e}}^{i\phi }\right)S_T\left(\phi \right)F_T{\left({\mathrm{e}}^{i\phi }\right)}^{*}$, the common variance of its elements is computed as

$$\begin{array}{cc}\hfill \mathbf{E}\left(\right|{\zeta }_k{|}^2)& ={\displaystyle \frac{1}{2\pi }}{\int }_{\mathbb{T}}\mathrm{Tr}\left(F_T\left({\mathrm{e}}^{i\phi }\right)S_T\left(\phi \right)F_T{\left({\mathrm{e}}^{i\phi }\right)}^{*}\right)\mathrm{d}\phi \hfill \\ \hfill & ={\displaystyle \frac{1}{2\pi }}{\int }_{\mathbb{T}}\langle {\Lambda }_T\left(\phi \right),S_T\left(\phi \right)\rangle \mathrm{d}\phi \hfill \\ \hfill & ={\displaystyle \frac{1}{2\pi }}{\int }_{\mathbb{R}}\Sigma \left(\omega \right)\langle \Lambda \left(\omega \right),S\left(\omega \right)\rangle \mathrm{d}\omega ,k\in \mathbb{Z}.\hfill \end{array}$$

(96)

In view of (92), (96) (see also [58]), the RMS gain (88) of the system F is identical to that of its discrete-time counterpart $F_T$ specified above:

$$\left[\right]F{\left[\right]}_{T,S}=\sqrt{{\displaystyle \frac{\mathbf{E}\left(\right|{\zeta }_0{|}^2)}{\mathbf{E}\left(\right|{\varpi }_0{|}^2)}}}=\left[\right]F_T{\left[\right]}_{S_T},$$

(97)

where, without loss of generality, the variances in the numerator and denominator are taken at the initial moment of time due to stationarity of the sequences (91), (95). Note that in the stationary case, it is redundant to consider Cesaro means of moments (such as variances) instead of their constant values. The right-hand side of (97) is the RMS gain of the discrete-time transfer function $F_T$ with respect to stationary zero-mean Gaussian random sequences $\varpi$ with the spectral density $S_T$. This quantity depends on the spectral density S of the continuous time process W through its discrete-time image $S_T$ in (87) and can be computed for a given S both in the continuous and discrete time domains due to the T-dependent conformal correspondence between them.

For example, if W is a standard Wiener process up to a multiplicative constant (the isotropic white-noise case (71)), then the spectral density $S_T$ in (87) is the same constant scalar matrix, and $\varpi$ is a zero-mean Gaussian white-noise sequence with the scalar covariance matrix. In this case, (97) reproduces (72). However, the conformal correspondence applies to a broader setting of worst-case RMS gains with respect to statistically uncertain disturbances, where the uncertainty can be formulated in terms of the discrete-time counterparts of the system and the noise shaping filter.

8. Anisotropy-Constrained Statistical Uncertainty and Worst-Case RMS Gain

Suppose the input W of the system F is a statistically uncertain random disturbance, which is produced at the output of an imprecisely known noise shaping filter G in Figure 1, with the only prior information about its spectral density S being that it belongs to a given class $\mathcal{S}$. Then of interest is the worst-case value

$$\left[\right]F{\left[\right]}_{T,\mathcal{S}}:=\underset{S\in \mathcal{S}}{sup}\left[\right]F{\left[\right]}_{T,S}$$

(98)

of the RMS gain (97) (we have slightly abused the notation). In particular, the class $\mathcal{S}$ can be described in terms of the deviation of S from constant scalar matrices (which correspond to the nominal isotropic white-noise model (71)). Such deviation is quantified, for example, by the mean anisotropy [58,60] of the stationary Gaussian random sequence (91) in ${\mathbb{R}}^m$, which, in view of (86), (92), takes the form

$$\begin{array}{cc}\hfill {\mathbf{A}}_T\left(S\right)& :=\overline{\mathbf{A}}\left(S_T\right)=-{\displaystyle \frac{1}{2}}lndet\left({\displaystyle \frac{m}{\mathbf{E}\left(\right|{\varpi }_0{|}^2)}}\Gamma \right)\hfill \\ \hfill & =-{\displaystyle \frac{1}{4\pi }}{\int }_{\mathbb{T}}lndet\left({\displaystyle \frac{m}{\mathbf{E}\left(\right|{\varpi }_0{|}^2)}}{S}_T\left(\phi \right)\right)\mathrm{d}\phi \hfill \\ \hfill & =-{\displaystyle \frac{1}{4\pi }}{\int }_{\mathbb{R}}\Sigma \left(\omega \right)lndet\left({\displaystyle \frac{2\pi m}{{\int }_{\mathbb{R}}\Sigma \left(\lambda \right)\mathrm{Tr}S\left(\lambda \right)\mathrm{d}\lambda }}S\left(\omega \right)\right)\mathrm{d}\omega \hfill \\ \hfill & =-{\displaystyle \frac{T}{2\pi }}{\int }_{\mathbb{R}}{\displaystyle \frac{1}{1+{\left(\omega T\right)}^2}}lndet\left({\displaystyle \frac{\pi m}{T{\int }_{\mathbb{R}}\frac{\mathrm{Tr}S\left(\lambda \right)}{1+{\left(\lambda T\right)}^2}\mathrm{d}\lambda }}S\left(\omega \right)\right)\mathrm{d}\omega ,\hfill \end{array}$$

(99)

where $0\prec \Gamma ={\Gamma }^{\mathrm{T}}\in {\mathbb{R}}^{m\times m}$ is an auxiliary matrix which is discussed below along with the main properties of (99). The quantity ${\mathbf{A}}_T\left(S\right)$ is finite and nonnegative if the spectral density S of the noise W satisfies the following inequality:

$${\int }_{\mathbb{T}}lndet{S}_T\left(\phi \right)\mathrm{d}\phi =2T{\int }_{\mathbb{R}}{\displaystyle \frac{lndetS\left(\omega \right)}{1+{\left(\omega T\right)}^2}}\mathrm{d}\omega >-\infty .$$

(100)

Although the fulfillment of this condition does not depend on a particular choice of $T>0$, an important role for the convergence of the second integral in (100) is played by the denominator. This is explained by the fact that an arbitrary spectral density $S:\mathbb{R}\to {\mathbb{H}}_m^{+}$ and its log-determinant cannot be simultaneously integrable on the real axis in view of the relations

$${\int }_{-b}^b(\mathrm{Tr}S\left(\omega \right)-lndetS\left(\omega \right))\mathrm{d}\omega ⩾2bm\to +\infty ,\mathrm{as}b\to +\infty ,$$

(101)

whereby ${\int }_{\mathbb{R}}\mathrm{Tr}S\left(\omega \right)\mathrm{d}\omega <+\infty$ implies that ${\int }_{\mathbb{R}}lndetS\left(\omega \right)\mathrm{d}\omega =-\infty$. The inequality (101) employs the following lower bound for the integrand:

$$\underset{Q\in {\mathbb{H}}_m^{+}}{min}(\mathrm{Tr}Q-lndetQ)=m,$$

(102)

where the function being minimized is strictly convex and achieves its unique minimum at $Q=I_m$. This can be verified by using the Frechet derivatives ${\partial }_Q\mathrm{Tr}Q=I_m$ and ${\partial }_{Q}lndetQ=Q^{-1}$, with the latter assuming that $Q\succ 0$.

Since the transfer function $G_T$ in (85) belongs to the Hardy space of ${\mathbb{C}}^{m\times m}$-valued functions, analytic in the open unit disk and having a square integrable boundary value over the unit circle, then (100) is equivalent to $det{G}_T\left(\phi \right)\ne 0$ for almost all $\phi \in \mathbb{T}$. This full-rank condition, associated with the inner-outer factorization [71] of the spectral density $S_T$ in (87), reflects the absence of linear dependencies between the entries of the random vectors ${\varpi }_k$ at the same or different moments of time. Also, this property is closely related to the total unpredictability [21,46] of such random sequences in the sense of nonsingularity of the conditional covariance matrix

$$\Gamma :=\mathbf{cov}({\varpi }_k\mid {\mathcal{F}}_{k-1})$$

(103)

given the past history ${\left({\varpi }_j\right)}_{j<k}$ of the sequence $\varpi$ in (91). Here, $\mathcal{F}:={\left({\mathcal{F}}_k\right)}_{k\in \mathbb{Z}}$ is the natural filtration for $\varpi$ consisting of the the $\sigma$-algebras

$${\mathcal{F}}_k:=\sigma \{{\varpi }_k,{\varpi }_{k-1},{\varpi }_{k-2},\dots \}.$$

In the stationary Gaussian case being considered, the matrix $\Gamma$ (which is used in (99)) does not depend on discrete time k and is nonrandom, whereby it coincides with the unconditional covariance matrix of the one-step prediction error ${\tilde{\varpi }}_k$ associated with the sequence $\varpi$ by

$${\tilde{\varpi }}_k:={\varpi }_k-{\widehat{\varpi }}_k,{\widehat{\varpi }}_k:=\mathbf{E}({\varpi }_k\mid {\mathcal{F}}_{k-1}).$$

(104)

The $\mathcal{F}$-predictable sequence $\widehat{\varpi }:={\left({\widehat{\varpi }}_k\right)}_{k\in \mathbb{Z}}$ and the $\mathcal{F}$-martingale difference sequence $\tilde{\varpi }:={\left({\tilde{\varpi }}_k\right)}_{k\in \mathbb{Z}}$ provide the Doob decomposition [49] for $\varpi$:

$${\varpi }_k={\widehat{\varpi }}_k+{\tilde{\varpi }}_k,k\in \mathbb{Z}.$$

(105)

By the Szegö-Kolmogorov theorem [24,31], as discussed in the proof of [60, Lemma 1], the matrix (103) is linked to the spectral density $S_T$ of $\varpi$ as

$$lndet\Gamma =\underset{N\to +\infty }{lim}{\displaystyle \frac{lndet\mathbf{cov}\left({\varpi }^N\right)}{N}}={\displaystyle \frac{1}{2\pi }}{\int }_{\mathbb{T}}lndet{S}_T\left(\phi \right)\mathrm{d}\phi ,$$

(106)

where

$${\varpi }^N:={\left({\varpi }_k\right)}_{0⩽k<N}$$

(107)

is an ${\mathbb{R}}^{mN}$-valued zero-mean Gaussian random vector formed from N consecutive elements ${\varpi }_0,\dots ,{\varpi }_{N-1}$ of the sequence $\varpi$ in (91), whose covariance matrix is recovered from $S_T$ as

$$\mathbf{E}\left({\varpi }_j{\varpi }_k^{\mathrm{T}}\right)={\displaystyle \frac{1}{2\pi }}{\int }_{\mathbb{T}}{\mathrm{e}}^{i(j-k)\phi }{S}_T\left(\phi \right)\mathrm{d}\phi .$$

In application to the stationary zero-mean Gaussian sequence $\varpi$ with the spectral density $S_T$, the origin of the mean anisotropy functional (99) is clarified by the limit theorems from [58] and [65] as

$$\underset{N\to +\infty }{lim}{\displaystyle \frac{{\mathbf{A}}_{\circ }\left({\mho }_N\right)}{N}}=\underset{N\to +\infty }{lim}{\displaystyle \frac{\mathbf{A}\left({\varpi }^N\right)}{N}}=\overline{\mathbf{A}}\left(S_T\right).$$

Here, the original “spherical” anisotropy functional from [58]

$$\begin{array}{cc}\hfill {\mathbf{A}}_{\circ }\left({\mho }_N\right)& :=\mathbf{D}(Q_N\parallel U_{mN})=\mathbf{E}ln{q}_N\left({\mho }_N\right)\hfill \\ \hfill & ={\int }_{{\mathbb{S}}_{mN}}{q}_N\left(u\right)ln{q}_N\left(u\right)U_{mN}\left(\mathrm{d}u\right)\hfill \end{array}$$

(108)

is the KL relative entropy [12] of the distribution $Q_N$ of the unit random vector

$${\mho }_N:={\displaystyle \frac{1}{|{\varpi }^N|}}{\varpi }^N$$

with a PDF $q_N:{\mathbb{S}}_{mN}\to {\mathbb{R}}_{+}$ with respect to the uniform distribution $U_{mN}$ on the unit sphere

$${\mathbb{S}}_{mN}:=\{u\in {\mathbb{R}}^{mN}:\left|u\right|=1\}.$$

The other anisotropy functional, introduced subsequently in [65] (and inspired by a similar yet one-dimensional power-entropy functional from [5]), is given by

$$\begin{array}{cc}\hfill \mathbf{A}\left({\varpi }^N\right)& :=\underset{\lambda >0}{min}\mathbf{D}({\Pi }_N\parallel {\mathcal{N}}_{\lambda I_{mN}})\hfill \\ \hfill & ={\displaystyle \frac{mN}{2}}ln\left({\displaystyle \frac{2\pi \mathrm{e}}{mN}}\mathbf{E}\left(\right|{\varpi }^N{|}^2)\right)-\mathbf{h}\left({\varpi }^N\right)\hfill \\ \hfill & =-{\displaystyle \frac{1}{2}}lndet\left({\displaystyle \frac{m}{\mathbf{E}\left(\right|{\varpi }_0{|}^2)}}\mathbf{cov}\left({\varpi }^N\right)\right),\hfill \end{array}$$

(109)

which is the minimum relative entropy of the distribution ${\Pi }_N$ of the random vector ${\varpi }^N$ from (107) with respect to isotropic Gaussian distributions ${\mathcal{N}}_{\lambda I_{mN}}$ in ${\mathbb{R}}^{mN}$ with zero mean and scalar covariance matrices $\lambda I_{mN}$. Here, $\mathbf{h}(·)$ is the differential entropy (with respect to the usual Lebesgue measure) which lends itself to closed-form computation for Gaussian random vectors, thus making the second anisotropy functional (109) more practical than its predecessor (108). Indeed,

$$\begin{array}{cc}\hfill \mathbf{h}\left({\varpi }^N\right)& :=-\mathbf{E}ln{p}_N\left({\varpi }^N\right)\hfill \\ \hfill & ={\displaystyle \frac{mN}{2}}ln\left(2\pi \mathrm{e}\right)+{\displaystyle \frac{1}{2}}lndet\mathbf{cov}\left({\varpi }^N\right),\hfill \end{array}$$

(110)

which is obtained by using the PDF $p_N:{\mathbb{R}}^{mN}\to {\mathbb{R}}_{+}$ of the random vector ${\varpi }^N$ from (107) given by

$$p_N\left(w\right)={\displaystyle \frac{{\left(2\pi \right)}^{-mN/2}}{\sqrt{det\mathbf{cov}\left({\varpi }^N\right)}}}{\mathrm{e}}^{-{\displaystyle \frac{1}{2}}{\parallel w\parallel }_{\mathbf{cov}{\left({\varpi }^N\right)}^{-1}}^2},w\in {\mathbb{R}}^{mN}.$$

Note that (110) clarifies the entropy-theoretic meaning of (106) by its connection with the differential entropy rate

$$\underset{N\to +\infty }{lim}{\displaystyle \frac{\mathbf{h}\left({\varpi }^N\right)}{N}}={\displaystyle \frac{m}{2}}ln\left(2\pi \mathrm{e}\right)+{\displaystyle \frac{1}{2}}lndet\Gamma ={\displaystyle \frac{1}{2}}lndet(2\pi \mathrm{e}\Gamma )$$

as a particular case of the Shannon-McMillan-Breiman theorem [12] in application to the stationary Gaussian sequence $\varpi$.

The mean anisotropy ${\mathbf{A}}_T\left(S\right)$ in (99) is invariant under the scaling and “rotation” transformations $S\left(\omega \right)\mapsto \lambda U\left(i\omega \right)S\left(\omega \right)U{\left(i\omega \right)}^{*}$ for any constant $\lambda >0$ and all-pass transfer functions U (that is, such that the matrix $U\left(i\omega \right)$ is unitary at all frequencies $\omega \in \mathbb{R}$). Furthermore, ${\mathbf{A}}_T\left(S\right)$, which is always nonnegative, vanishes if and only if the spectral density $S_T$ of the auxiliary sequence $\varpi$ in (91), and hence, S in view of (87), is a constant scalar matrix, which (in the Gaussian case) holds if and only if the input disturbance W is a standard Wiener process up to a multiplicative constant. The above properties are inherited by ${\mathbf{A}}_T\left(S\right)$ from its discrete-time counterpart, and so also is the closely related decomposition

$${\mathbf{A}}_T\left(S\right)={\mathbf{A}}_T^{\mathrm{spat}}\left(S\right)+{\mathbf{A}}_T^{\mathrm{temp}}\left(S\right)$$

(111)

of the mean anisotropy functional (99) into the “spatial” and ”temporal” components (cf. [14])

$$\begin{array}{cc}\hfill {\mathbf{A}}_T^{\mathrm{spat}}\left(S\right)& :=-{\displaystyle \frac{1}{2}}lndet\left({\displaystyle \frac{m}{\mathbf{E}\left(\right|{\varpi }_0{|}^2)}}\mathbf{E}\left({\varpi }_0{\varpi }_0^{\mathrm{T}}\right)\right)\hfill \end{array}$$

$$\begin{array}{cc}\hfill & =-{\displaystyle \frac{1}{2}}lndet\left({\displaystyle \frac{m}{{\int }_{\mathbb{R}}\frac{\mathrm{Tr}S\left(\lambda \right)}{1+{\left(\lambda T\right)}^2}\mathrm{d}\lambda }}{\int }_{\mathbb{R}}{\displaystyle \frac{1}{1+{\left(\omega T\right)}^2}}S\left(\omega \right)\mathrm{d}\omega \right),\hfill \\ \hfill {\mathbf{A}}_T^{\mathrm{temp}}\left(S\right)& :={\displaystyle \frac{1}{2}}lndet\left({\Gamma }^{-1}\mathbf{E}\left({\varpi }_0{\varpi }_0^{\mathrm{T}}\right)\right)\hfill \end{array}$$

(112)

$$\begin{array}{cc}\hfill & ={\displaystyle \frac{T}{2\pi }}{\int }_{\mathbb{R}}{\displaystyle \frac{1}{1+{\left(\omega T\right)}^2}}lndet\left({\displaystyle \frac{T}{\pi }}S{\left(\omega \right)}^{-1}{\int }_{\mathbb{R}}{\displaystyle \frac{1}{1+{\left(\lambda T\right)}^2}}S\left(\lambda \right)\mathrm{d}\lambda \right)\mathrm{d}\omega ,\hfill \end{array}$$

(113)

which are also nonnegative. In particular, ${\mathbf{A}}_T^{\mathrm{temp}}\left(S\right)⩾0$ because this quantity coincides with Shannon’s mutual information [12] between an element ${\varpi }_k$ of the stationary Gaussian sequence $\varpi$ and its past history ${\left({\varpi }_j\right)}_{j<k}$:

$$\begin{array}{cc}\hfill {\mathbf{A}}_T^{\mathrm{temp}}\left(S\right)& =\mathbf{I}({\varpi }_k;{\left({\varpi }_j\right)}_{j<k})=\mathbf{h}\left({\varpi }_k\right)-\mathbf{h}({\varpi }_k\mid {\left({\varpi }_j\right)}_{j<k})\hfill \\ \hfill & =\underset{N\to +\infty }{lim}\mathbf{I}({\varpi }_0;{\left({\varpi }_{-j}\right)}_{1⩽j⩽N}),\hfill \end{array}$$

where $\mathbf{h}(·\mid ·)$ is the conditional differential entropy. The nonnegativeness of ${\mathbf{A}}_T^{\mathrm{temp}}\left(S\right)$ can also be verified directly by using the inequality $\Gamma \preccurlyeq \mathbf{E}\left({\varpi }_0{\varpi }_0^{\mathrm{T}}\right)$ for the matrix (103) in the Gaussian case or, alternatively, Jensen’s inequality in combination with the normalization property of $\Sigma$ in (73) and the strict concavity of $lndet(·)$ on the cone of complex positive definite Hermitian matrices (which was used in (102)). The last argument also shows that ${\mathbf{A}}_T^{\mathrm{temp}}\left(S\right)=0$ in (113) if and only if the spectral density S is a constant positive definite matrix, that is, when the noise process W is temporally white in the sense of having independent increments.

Similarly to discrete-time settings, the properties of ${\mathbf{A}}_T\left(S\right)$ in (99) allow it to be used as a measure of deviation from the nominal isotropic white-noise model for specifying a particular uncertainty class $\mathcal{S}$ in (98):

$${\mathcal{S}}_{T,a}:=\{S:\mathbb{R}\to {\mathbb{H}}_m^{+}:{\mathbf{A}}_T\left(S\right)⩽a\}.$$

(114)

This class consists of all those spectral densities S of the input disturbance process W, whose discrete-time counterparts $S_T$ in (87) satisfy the upper constraint on the mean anisotropy (99) of the corresponding stationary Gaussian sequence $\varpi$ with the variance (92). Since the map $S\mapsto S_T$ is bijective, the worst-case RMS gain (98), associated with (114), takes the form

$$\begin{array}{cc}\hfill \left|\right||F{\left|\right||}_{T,a}& :=\left[\right]F{\left[\right]}_{T,{\mathcal{S}}_{T,a}}=\underset{S\in {\mathcal{S}}_{T,a}}{sup}\left[\right]F{\left[\right]}_{T,S}\hfill \\ \hfill & =\underset{S\in {\mathcal{S}}_{T,a}}{sup}\sqrt{{\displaystyle \frac{{\int }_{\mathbb{T}}\langle {\Lambda }_T\left(\phi \right),S_T\left(\phi \right)\rangle \mathrm{d}\phi }{{\int }_{\mathbb{T}}\mathrm{Tr}{S}_T\left(\phi \right)\mathrm{d}\phi }}}=\left|\right||F_T{\left|\right||}_a\hfill \end{array}$$

(115)

and coincides with the a-anisotropic norm of the discrete-time system $F_T$. The latter norm (and hence, the two-parameter $(T,a)$-anisotropic norm $\left|\right||F{\left|\right||}_{T,a}$ of the underlying LCTI system) lends itself to state-space computation [60].

9. Computation of Continuous-Time Anisotropic Norm in State Space

In order to compute the norm $\left|\right||F{\left|\right||}_{T,a}$ of the LCTI system F in state space through its discrete-time counterpart $\left|\right||F_T{\left|\right||}_a$ in (115), we note that, in view of (25), (32), (81), the transfer function $F_T$ in (84) takes the form

$$\begin{array}{cc}\hfill F_T\left(z\right)& =C{\left(\Omega {\displaystyle \frac{1-z}{1+z}}{I}_n-A\right)}^{-1}B+D\hfill \\ \hfill & =(1+z)TC{(I_n-TA-z(I_n+TA))}^{-1}B+D\hfill \\ \hfill & =(1+z)C{(I_n-z{A}_T)}^{-1}{B}_T+D\hfill \\ \hfill & =z{C}_T{(I_n-z{A}_T)}^{-1}{B}_T+D_T.\hfill \end{array}$$

(116)

This is the generating function for the impulse response with the z-transform $F_T(1/z)$ which corresponds to the LDTI system

$$x_{k+1}=A_T{x}_k+B_T{\varpi }_k,{\zeta }_k=C_T{x}_k+D_T{\varpi }_k$$

(117)

with an ${\mathbb{R}}^n$-valued state sequence $x:={\left(x_k\right)}_{k\in \mathbb{Z}}$ and the state-space realization matrices

$$\begin{array}{cc}\hfill A_T& :=(I_n+TA){(I_n-TA)}^{-1}=K(-TA),\hfill \end{array}$$

(118)

$$\begin{array}{cc}\hfill B_T& :=T{(I_n-TA)}^{-1}B,\hfill \end{array}$$

(119)

$$\begin{array}{cc}\hfill C_T& :=2C{(I_n-TA)}^{-1},\hfill \end{array}$$

(120)

$$\begin{array}{cc}\hfill D_T& :=TC{(I_n-TA)}^{-1}B+D,\hfill \end{array}$$

(121)

where the function K from (81) is evaluated [26] in (118) at the Hurwitz matrix $TA$. The system (117) can be viewed as a finite-difference scheme for numerical integration of the SDEs (7), (11) by the trapezoidal rule with stepsize $2T$. However, such interpretation assumes smallness of the time scale parameter T (for example, in the sense of the first relation in (80)), whereas the conformal correspondence between the LCTI and LDTI setups is valid for any $T>0$.

Now, for a fixed but otherwise arbitrary $T>0$, the relations (118)–(121) describe a smooth bijection

$${\rm Y}:(A,B,C,D)\mapsto (A_T,B_T,C_T,D_T)$$

(122)

between the open sets of matrix quadruples such that A is Hurwitz and $\mathbf{r}\left(A_T\right)<1$. The structure of dependencies between the matrices under the map ${\rm Y}$ is shown in Figure 5 and is inherited by the inverse map

$${{\rm Y}}^{-1}:(A_T,B_T,C_T,D_T)\mapsto (A,B,C,D),$$

(123)

which, in view of (25), takes the form

$$\begin{array}{cc}\hfill A& =\Omega {(I_n+A_T)}^{-1}(A_T-I_n)=-\Omega K\left(A_T\right),\hfill \end{array}$$

(124)

$$\begin{array}{cc}\hfill B& =2\Omega {(I_n+A_T)}^{-1}{B}_T,\hfill \end{array}$$

(125)

$$\begin{array}{cc}\hfill C& =C_T{(I_n+A_T)}^{-1},\hfill \end{array}$$

(126)

$$\begin{array}{cc}\hfill D& =D_T-C_T{(I_n+A_T)}^{-1}{B}_T.\hfill \end{array}$$

(127)

Relations, similar to (116), (118)–(127), also hold for the transfer function $G_T$ of the discrete-time noise shaping filter in (85). As in the discrete-time case, the computation of the $(T,a)$-anisotropic norm (115) for the LCTI system F being considered is of interest only if F is nonround and the mean anisotropy level a is strictly positive (otherwise, the norm reduces trivially to the weighted ${\mathcal{H}}_2$-norm ${\displaystyle \frac{1}{\sqrt{m}}}{\parallel F_T\parallel }_2$).

Theorem 4. 

Suppose the system F in (7), (11) is stable and nonround. Then its $(T,a)$-anisotropic norm (115) can be computed for any given $T>0$ and $a>0$ as

$$\left|\right||F{\left|\right||}_{T,a}=\sqrt{{\displaystyle \frac{1}{q}}\left(1-{\displaystyle \frac{m}{\mathrm{Tr}(L_T{P}_T{L}_T^{\mathrm{T}}+M_T^2)}}\right)}.$$

(128)

Here, $P_T=P_T^{\mathrm{T}}\in {\mathbb{R}}^{n\times n}$ is a positive semi-definite matrix which is the controllability Gramian for the pair

$${\alpha }_T:=A_T+B_T{L}_T,{\beta }_T:=B_T{M}_T$$

(129)

and is a unique solution of the discrete algebraic Lyapunov equation (ALE)

$$P_T={\alpha }_T{P}_T{\alpha }_T^{\mathrm{T}}+{\beta }_T{\beta }_T^{\mathrm{T}},$$

(130)

where the matrices $A_T,B_T,C_T,D_T$ are given by (118)–(121), and the matrices $L_T\in {\mathbb{R}}^{m\times n}$ and $0\prec M_T=M_T^{\mathrm{T}}\in {\mathbb{R}}^{m\times m}$ are associated with a unique admissible solution $R_T=R_T^{\mathrm{T}}\in {\mathbb{R}}^{n\times n}$ of the discrete algebraic Riccati equation (ARE)

$$\begin{array}{cc}\hfill R_T& =A_T^{\mathrm{T}}{R}_T{A}_T+q{C}_T^{\mathrm{T}}{C}_T+L_T^{\mathrm{T}}{M}_T^{-2}{L}_T,\hfill \end{array}$$

(131)

$$\begin{array}{cc}\hfill L_T& :=M_T^2(B_T^{\mathrm{T}}{R}_T{A}_T+q{D}_T^{\mathrm{T}}{C}_T),\hfill \end{array}$$

(132)

$$\begin{array}{cc}\hfill M_T& :={(I_m-q{D}_T^{\mathrm{T}}{D}_T-B_T^{\mathrm{T}}{R}_T{B}_T)}^{-1/2}.\hfill \end{array}$$

(133)

The admissibility of $R_T$ is understood in the sense that $I_m-q{D}_T^{\mathrm{T}}{D}_T-B_T^{\mathrm{T}}{R}_T{B}_T\succ 0$ is satisfied together with the stability condition

$$\mathbf{r}\left({\alpha }_T\right)<1,$$

(134)

and the parameter

$$0<q<{\displaystyle \frac{1}{{\parallel F\parallel }_{\infty }^2}}$$

(135)

(on which the matrices $R_T,L_T,M_T,P_T$ also depend) is a unique solution of the equation

$$-{\displaystyle \frac{1}{2}}lndet\left({\displaystyle \frac{m}{\mathrm{Tr}(L_T{P}_T{L}_T^{\mathrm{T}}+M_T^2)}}{M}_T^2\right)=a.$$

(136)

The worst-case noise spectral density is unique up to a multiplicative positive constant and is given by

$$S\left(\omega \right)={(I_m-q\Lambda \left(\omega \right))}^{-1},\omega \in \mathbb{R},$$

(137)

in terms of (55). It is delivered by an input disturbance W according to the SDE (19), where the matrices $L\in {\mathbb{R}}^{m\times n}$ and $M\in {\mathbb{R}}^{m\times m}$ (with M not necessarily symmetric) are computed as

$$L=L_T{(I_n+{\alpha }_T)}^{-1},M=M_T-L{\beta }_T.$$

(138)

Proof. 

In regard to computing the $(T,a)$-anisotropic norm (115) of the LCTI system F, the assertion of the theorem is obtained by applying the discrete-time result of [60, Theorem 2] to the LDTI system $F_T$ in (117)–(121). However, for a proper adaptation of that result to the continuous-time setting, we will provide its key points on the probabilistic structure of the worst-case disturbance. To this end, we note that the stability and nonroundness of the system F are equivalent to the corresponding properties of its discrete-time counterpart $F_T$. Furthermore, both systems have equal ${\mathcal{H}}_{\infty }$-norms in the appropriate Hardy spaces associated with the right half-plane and the unit disk: ${\parallel F\parallel }_{\infty }={\parallel F_T\parallel }_{\infty }$. Recall that the accompanying material of [60, Section 5] (given in more detail in [14, Section 8]) describes the structure

$${\varpi }_k=L_T{x}_k+M_T{\upsilon }_k$$

(139)

of the input disturbance of the discrete-time system (see Figure 6), implementing the worst-case spectral density $S_T$ in (87), which is unique up to a multiplicative positive constant [60, Theorem 1] and has the form

$$S_T\left(\phi \right)={(I_m-q{\Lambda }_T\left(\phi \right))}^{-1},\phi \in \mathbb{T}.$$

(140)

Here, the parameter q, satisfying (135), is found from the saturation $\overline{\mathbf{A}}\left(S_T\right)=a$ of the constraint $\overline{\mathbf{A}}\left(S_T\right)⩽a$ on the mean anisotropy (99) in (114), (115).

The representation (140) of $S_T$ in terms of ${\Lambda }_T$ from (86) is equivalent to the isometric property of an auxiliary LDTI system

$${\Theta }_T:=\left[\begin{array}{c}\sqrt{q}{F}_T\\ G_T^{-1}\end{array}\right]\leftrightarrow \left[\begin{array}{cc}{A}_T& B_T\\ \sqrt{q}{C}_T& \sqrt{q}{D}_T\\ -{\rho }_T& M_T^{-1}\end{array}\right],$$

(141)

which employs a matrix

$${\rho }_T:=M_T^{-1}{L}_T$$

(142)

and has the input $\varpi$ and the output

$$\left[\begin{array}{c}\sqrt{q}\zeta \\ \upsilon \end{array}\right].$$

The right-hand side of (141) describes the state-space realization matrices of the system ${\Theta }_T$, with its bottom block row corresponding to the equation

$${\upsilon }_k=-{\rho }_T{x}_k+M_T^{-1}{\varpi }_k$$

obtained from (139) due to $det{M}_T\ne 0$. The condition that the system ${\Theta }_T$ is isometric is understood in the sense that if $\varpi$ were an arbitrary square summable sequence in ${\mathbb{R}}^m$ (instead of being a stationary Gaussian sequence whose sample paths are square summable only in the trivial case $\varpi =0$) and hence, so also were $\upsilon$, $\zeta$ in (93), (95), then their ${\ell }_2$-norms would be related by ${\parallel \varpi \parallel }_2^2={q\parallel \zeta \parallel }_2^2+{\parallel \upsilon \parallel }_2^2$. However, in the case of the worst-case stationary Gaussian sequence $\varpi$ being considered, the isometric property of the system (141) manifests itself in terms of the variances as

$$\mathbf{E}\left(\right|{\varpi }_k{|}^2)=q\mathbf{E}(|{\zeta }_k{|}^2)+m,$$

where use is also made of the relation $\mathbf{E}\left(\right|{\upsilon }_k{|}^2)=\mathrm{Tr}{I}_m=m$ which follows from (94). With the matrix $R_T$ in (131) being the observability Gramian of the system (141), the other two equations in the ARE (131)–(133) describe a sufficient state-space condition for the system ${\Theta }_T$ to be isometric [60]. This condition is also necessary if the pair $(A_T,B_T)$ is controllable [25]. Similarly to its inverse $G_T^{-1}$ in (141), the shaping filter

$$G_T\leftrightarrow \left[\begin{array}{cc}{\alpha }_T& {\beta }_T\\ L_T& M_T\end{array}\right],$$

(143)

governed by the first of the equations (117) in combination with (139), shares the common state x with the system $F_T$:

$$x_{k+1}={\alpha }_T{x}_k+{\beta }_T{\upsilon }_k,{\varpi }_k=L_T{x}_k+M_T{\upsilon }_k,$$

(144)

where the matrices ${\alpha }_T$, ${\beta }_T$ are given by (129) and, as before, the input of the filter is the Gaussian white-noise sequence $\upsilon$ in ${\mathbb{R}}^m$ with zero mean and identity covariance matrix. The stability of the filter (144) is secured by the condition (134) and, in combination with (94), leads to the equation (130) for the covariance matrix

$$P_T:=\mathbf{cov}\left(x_k\right)$$

(145)

of the state of the LDTI system $F_T$ under the worst-case input disturbance (139) being considered. Accordingly, the covariance matrix

$$\mathbf{cov}\left({\varpi }_k\right)=L_T{P}_T{L}_T^{\mathrm{T}}+M_T^2$$

(146)

of the output of the discrete-time noise shaping filter (143) gives rise to its variance in the common denominator of (128), (136):

$$\mathbf{E}\left(\right|{\varpi }_k{|}^2)=\mathrm{Tr}(L_T{P}_T{L}_T^{\mathrm{T}}+M_T^2)$$

in view of the symmetry $M_T=M_T^{\mathrm{T}}$. Due to the stability of the shaping filter $G_T$ and the system $F_T$ itself (recall that (134) is fulfilled along with $\mathbf{r}\left(A_T\right)<1$), and also since $det{M}_T\ne 0$, the sequence $\varpi$ generates the same filtration $\mathcal{F}$ as $\upsilon$, with (139) providing its Doob decomposition (105) into the $\mathcal{F}$-predictable and innovation components $\widehat{\varpi }$, $\tilde{\varpi }$ in (104) as

$${\widehat{\varpi }}_k=L_T{x}_k,{\tilde{\varpi }}_k=M_T{\upsilon }_k.$$

(147)

Hence, with the matrix $M_T$ being symmetric, the one-step prediction error covariance matrix (103) takes the form

$$\Gamma =\mathbf{cov}(M_T{\upsilon }_k\mid {\mathcal{F}}_{k-1})=M_T\mathbf{cov}\left({\upsilon }_k\right)M_T^{\mathrm{T}}=M_T^2,$$

(148)

thus leading to the numerator $M_T^2$ in (136). The structure of the equivalent continuous-time input disturbance W is obtained by applying the inverse map ${{\rm Y}}^{-1}$ from (123) (in accordance with (124)–(127)) to the worst-case shaping filter (143), which leads to (19) with the matrices (138). Therefore, in view of (11), the worst-case continuous-time noise shaping filter G is governed by

$$\mathrm{d}X=\alpha X\mathrm{d}t+\beta \mathrm{d}V,\mathrm{d}W=LX\mathrm{d}t+M\mathrm{d}V,$$

(149)

so that

$$G\leftrightarrow \left[\begin{array}{cc}\alpha & \beta \\ L& M\end{array}\right],$$

(150)

where

$$\alpha :=A+BL,\beta :=BM$$

(151)

in accordance with (20). In view of the conformal correspondence (86), (87), the worst-case noise spectral density is obtained from (140) and takes the form of (137). Due to the properties of the Cayley transform (81) and the map ${{\rm Y}}^{-1}$ (see also Figure 5), the dynamics matrix $\alpha$ in (149)–(151) is related to its discrete-time counterpart ${\alpha }_T$ in (129), (143), (144) by

$$\alpha =\Omega {(I_n+{\alpha }_T)}^{-1}({\alpha }_T-I_n)=-\Omega K\left({\alpha }_T\right)$$

and, in view of the condition (134), is Hurwitz. The latter follows from (82) and the transformation of the spectrum for a function of a matrix [26]. □

Note that, by the same conformal correspondence argument as in the proof of Theorem 4 (using the bijection between the unit circle and the imaginary axis), the LCTI system

$$\Theta :=\left[\begin{array}{c}\sqrt{q}F\\ G^{-1}\end{array}\right]\leftrightarrow \left[\begin{array}{cc}A& B\\ \sqrt{q}C& \sqrt{q}D\\ -\rho & M^{-1}\end{array}\right],$$

(152)

which involves an auxiliary matrix

$$\rho :=M^{-1}L$$

(153)

and is associated with the worst-case disturbance W described in the theorem, inherits the property of being inner from its discrete-time counterpart ${\Theta }_T$ in (141), so that its transfer function satisfies

$$\Theta {\left(i\omega \right)}^{*}\Theta \left(i\omega \right)=I_m,\omega \in \mathbb{R}.$$

Moreover, the structure (137) of the worst-case noise spectral density S is equivalent to the innerness of $\Theta$. In particular, by continuity, the feedthrough matrix in (152) describes an isometric embedding from ${\mathbb{R}}^m$ into ${\mathbb{R}}^{p+m}$:

$$\begin{array}{cc}\hfill \underset{\omega \to \infty }{lim}\left(\Theta {\left(i\omega \right)}^{*}\Theta \left(i\omega \right)\right)& =\left[\begin{array}{cc}\sqrt{q}{D}^{\mathrm{T}}& M^{-\mathrm{T}}\end{array}\right]\left[\begin{array}{c}\sqrt{q}D\\ M^{-1}\end{array}\right]\hfill \\ \hfill & =q{D}^{\mathrm{T}}D+{\left(M{M}^{\mathrm{T}}\right)}^{-1}=I_m,\hfill \end{array}$$

(154)

where $M^{-\mathrm{T}}:={\left(M^{-1}\right)}^{\mathrm{T}}$. From (154), or from a combination of (137) with (55), (32), (35), (150), it follows that

$$\Delta :=M{M}^{\mathrm{T}}=\underset{\omega \to \infty }{lim}S\left(\omega \right)={(I_m-q{D}^{\mathrm{T}}D)}^{-1}.$$

(155)

The matrix on the right-hand side of (155) is well-defined and positive definite. Indeed, ${\lambda }_{max}\left(D^{\mathrm{T}}D\right)⩽{\parallel F\parallel }_{\infty }^2$ and hence, (135) implies that $q<{\displaystyle \frac{1}{{\parallel F\parallel }_{\infty }^2}}⩽{\displaystyle \frac{1}{{\lambda }_{max}\left(D^{\mathrm{T}}D\right)}}$, whereby $q{D}^{\mathrm{T}}D\prec I_m$ and $\Delta \succ 0$.

As discussed above, similarly to the discrete-time representation in Figure 6, the worst-case disturbance W at the input of the LCTI system F in (7), (11) is formed in (19) by the feedback loop (see Figure 2), through which the system state X enters the drift $LX$ of the disturbance. In line with this, we note that the stability condition (that the matrix $\alpha$ in (149)–(151) has to be Hurwitz) and the mean anisotropy constraint in (114) forbid the hypothetical “noise player” to use an arbitrarily large matrix L for destabilizing the system and force the opponent to mix the predictable component $L{\int }_0^{t}X\left(\tau \right)\mathrm{d}\tau$ of the disturbance W with the innovation process $MV$.

Accordingly, the matrix $\Delta$ in (155), which is the diffusion matrix of the worst-case noise W governed by (149), plays a role similar to that of $\Gamma$ in (148). Indeed, both matrices quantify the second moments of the innovation components $M\mathrm{d}V$ and $M_T{\upsilon }_k$ of the input disturbances in the continuous and discrete-time representations (149) and (144), respectively.

In the worst-case noise scenario at the mean anisotropy level $a⩾0$, the latter admits the decomposition

$$a=a_{\mathrm{spat}}+a_{\mathrm{temp}}$$

(156)

into the spatial and temporal components which are found from (136) according to (112), (113) as

$$\begin{array}{cc}\hfill a_{\mathrm{spat}}& =-{\displaystyle \frac{1}{2}}lndet\left({\displaystyle \frac{m}{\mathrm{Tr}(L_T{P}_T{L}_T^{\mathrm{T}}+M_T^2)}}(L_T{P}_T{L}_T^{\mathrm{T}}+M_T^2)\right)\hfill \end{array}$$

$$\begin{array}{cc}\hfill & =-{\displaystyle \frac{1}{2}}lndet\left({\displaystyle \frac{m}{\langle M_T^2,I_m+{\Xi }_T{\Xi }_T^{\mathrm{T}}\rangle }}(I_m+{\Xi }_T{\Xi }_T^{\mathrm{T}})M_T^2\right),\hfill \\ \hfill a_{\mathrm{temp}}& ={\displaystyle \frac{1}{2}}lndet\left(M_T^{-2}(L_T{P}_T{L}_T^{\mathrm{T}}+M_T^2)\right)\hfill \end{array}$$

(157)

$$\begin{array}{cc}\hfill & ={\displaystyle \frac{1}{2}}lndet(I_m+{\Xi }_T{\Xi }_T^{\mathrm{T}}),\hfill \end{array}$$

(158)

where

$${\Xi }_T:={\rho }_T\sqrt{P_T}.$$

(159)

Here, use has been made of the matrix ${\rho }_T$ from (142) along with the fact that the matrix

$$L_T{P}_T{L}_T^{\mathrm{T}}+M_T^2=M_T(I_m+{\Xi }_T{\Xi }_T^{\mathrm{T}})M_T^2{M}_T^{-1}$$

in (146) is related by a similarity transformation and is therefore isospectral to the matrix

$$(I_m+{\Xi }_T{\Xi }_T^{\mathrm{T}})M_T^2.$$

Note that the temporal part $a_{\mathrm{temp}}$ in (158) is completely specified by the singular values of the matrix ${\Xi }_T$ from (159).

While the mean anisotropy functional (99) and its spatio-temporal decomposition (111)–(113) have an entropy-theoretic interpretation in terms of the auxiliary Gaussian sequence (91) discussed in Section 8, and $a_{\mathrm{spat}}$ in (157) admits two-sided bounds [14, p. 13] in terms of the condition number of the matrix (146), it is relevant to link the temporal component $a_{\mathrm{temp}}$ in (158) with relative entropy rates for the worst-case noise W in its native continuous time.

10. Relative Entropy Rates for Worst-Case Disturbance

Consider the system F subject to the worst-case noise W from Theorem 4 (at a given mean anisotropy level $a>0$) governed by the SDEs (149) with zero initial conditions $W\left(0\right)=0$ and $V\left(0\right)=0$. For any $t⩾0$, the restriction ${W|}_{[0,t]}$ of W to the time interval $[0,t]$ is a random element with values in the Banach space $C_0([0,t],{\mathbb{R}}^m)$ of continuous functions $w:[0,t]\to {\mathbb{R}}^m$ satisfying $w\left(0\right)=0$ and equipped with the uniform norm. By Girsanov’s theorem [23], the probability distribution of ${W|}_{[0,t]}$ on Borel subsets of $C_0([0,t],{\mathbb{R}}^m)$ is absolutely continuous with respect to that of the Wiener process $MV$ with the diffusion matrix $\Delta \succ 0$ in (155), and its Radon-Nikodym derivative, evaluated at ${W|}_{[0,t]}$, is found from

$$\begin{array}{cc}\hfill lnR\left(t\right)& ={\int }_0^{t}X{\left(\tau \right)}^{\mathrm{T}}{L}^{\mathrm{T}}{\Delta }^{-1}\mathrm{d}W\left(\tau \right)-{\displaystyle \frac{1}{2}}{\int }_0^t{\parallel LX\left(\tau \right)\parallel }_{{\Delta }^{-1}}^2\mathrm{d}\tau \hfill \\ \hfill & ={\int }_0^{t}X{\left(\tau \right)}^{\mathrm{T}}{\rho }^{\mathrm{T}}\mathrm{d}V\left(\tau \right)+{\displaystyle \frac{1}{2}}{\int }_0^t{\left|\rho X\left(\tau \right)\right|}^2\mathrm{d}\tau ,\hfill \end{array}$$

(160)

where use is also made of the matrix $\rho$ from (153). The validity of the representation (160) is guaranteed whenever an appropriately modified version of Novikov’s condition [43] is satisfied:

$${\mathbf{E}}_{*}{\mathrm{e}}^{{\displaystyle \frac{1}{2}}{\int }_0^t{\left|\rho X\left(\tau \right)\right|}^2\mathrm{d}\tau }<+\infty ,$$

(161)

where ${\mathbf{E}}_{*}(·)$ is the expectation over a reference probability measure ${\mathbf{P}}_{*}$ which considers $M^{-1}W$ a standard Wiener process in ${\mathbb{R}}^m$. The fulfillment of (161) for any $t>0$ secures the martingale property (in the sense of ${\mathbf{P}}_{*}$) on any bounded time interval for the random process R in (160) with respect to the natural filtration of W. Returning to the actual probability measure $\mathbf{P}$ with the expectation $\mathbf{E}(·)$, according to which W is governed by (149) where V is a standard Wiener process, we note that (160) allows the corresponding Renyi’s relative entropy [44] of order $1+ϵ$ to be computed for any $ϵ>0$ small enough (in the sense specified below) as

$$\begin{array}{cc}\hfill {\mathbf{R}}_{ϵ}\left(t\right)& :={\displaystyle \frac{1}{ϵ}}ln\mathbf{E}\left(R{\left(t\right)}^{ϵ}\right)={\displaystyle \frac{1}{ϵ}}ln\mathbf{E}{\mathrm{e}}^{ϵlnR\left(t\right)}\hfill \\ \hfill & ={\displaystyle \frac{1}{ϵ}}ln\mathbf{E}{\mathrm{e}}^{ϵ{\int }_0^{t}X{\left(\tau \right)}^{\mathrm{T}}{\rho }^{\mathrm{T}}\mathrm{d}V\left(\tau \right)+{\displaystyle \frac{ϵ}{2}}{\int }_0^t{\left|\rho X\left(\tau \right)\right|}^2\mathrm{d}\tau }\hfill \\ \hfill & ={\displaystyle \frac{1}{ϵ}}ln\mathbf{E}\left({\mu }_{ϵ}\left(t\right){\mathrm{e}}^{{\displaystyle \frac{1}{2}}ϵ(1+ϵ){\int }_0^t{\left|\rho X\left(\tau \right)\right|}^2\mathrm{d}\tau }\right)\hfill \\ \hfill & ={\displaystyle \frac{1}{ϵ}}ln\mathbf{E}{\mathrm{e}}^{{\displaystyle \frac{1}{2}}ϵ(1+ϵ){\int }_0^t{\left|\rho X\left(\tau \right)\right|}^2\mathrm{d}\tau }.\hfill \end{array}$$

(162)

Here,

$${\mu }_{ϵ}\left(t\right):={\mathrm{e}}^{ϵ{\int }_0^{t}X{\left(\tau \right)}^{\mathrm{T}}{\rho }^{\mathrm{T}}\mathrm{d}V\left(\tau \right)-{\displaystyle \frac{1}{2}}{ϵ}^2{\int }_0^t{\left|\rho X\left(\tau \right)\right|}^2\mathrm{d}\tau },$$

(163)

and use is made of its martingale property on any bounded time interval under the sufficient condition

$$\mathbf{E}{\mathrm{e}}^{{\displaystyle \frac{1}{2}}{ϵ}^2{\int }_0^t{\left|\rho X\left(\tau \right)\right|}^2\mathrm{d}\tau }<+\infty ,t>0.$$

(164)

The KL relative entropy of the probability distribution of ${W|}_{[0,t]}$ with respect to that of ${MV|}_{[0,t]}$ is recovered from (162), (160) as

$$\begin{array}{cc}\hfill {\mathbf{R}}_0\left(t\right)& :=\underset{ϵ\to 0+}{lim}{\mathbf{R}}_{ϵ}\left(t\right)=\mathbf{E}lnR\left(t\right)\hfill \\ \hfill & ={\displaystyle \frac{1}{2}}{\int }_0^t{\mathbf{E}\left(\right|\rho X\left(\tau \right)|}^2)\mathrm{d}\tau ⩽{\mathbf{R}}_{ϵ}\left(t\right),\hfill \end{array}$$

(165)

where the inequality (provided here for completeness of exposition) follows from Jensen’s inequality.

Note that the relative entropy (162) (and its limiting case (165)) takes the same value for the probability law of the filtered noise $W_T{|}_{[0,t]}$ with respect to that of $M{V}_T{|}_{[0,t]}$ because $W_T$ is obtained from W by the same bijective causal transformation as $M{V}_T$ from $MV$ as specified by (22), (23), and this transformation preserves the natural filtration of the processes.

The theorem below on the relative entropy rates for the worst-case noise W employs the transfer function

$$N\left(s\right):={(s{I}_n-\alpha )}^{-1}\beta ,s\in \mathbb{C}$$

(166)

of an auxiliary LCTI system which maps V to the state process X governed by the first SDE in (149) with the Hurwitz matrix $\alpha$. In this worst-case noise scenario, the invariant covariance matrix of the process X, given by

$$P:=\underset{t\to +\infty }{lim}\mathcal{P}\left(t\right)={\int }_{{\mathbb{R}}_{+}}{\mathrm{e}}^{t\alpha }\beta {\beta }^{\mathrm{T}}{\mathrm{e}}^{t{\alpha }^{\mathrm{T}}}\mathrm{d}t={\displaystyle \frac{1}{2\pi }}{\int }_{\mathbb{R}}N\left(i\omega \right)N{\left(i\omega \right)}^{*}\mathrm{d}\omega ,$$

(167)

coincides with the controllability Gramian of the pair $(\alpha ,\beta )$ from (151) and satisfies a continuous ALE

$$\alpha P+P{\alpha }^{\mathrm{T}}+\beta {\beta }^{\mathrm{T}}=0.$$

(168)

Here, the covariance matrix

$$\mathcal{P}\left(t\right):=\mathbf{cov}\left(X\left(t\right)\right)={\int }_0^t{\mathrm{e}}^{\tau \alpha }\beta {\beta }^{\mathrm{T}}{\mathrm{e}}^{\tau {\alpha }^{\mathrm{T}}}\mathrm{d}\tau =P-{\mathrm{e}}^{t\alpha }P{\mathrm{e}}^{t{\alpha }^{\mathrm{T}}},t⩾0,$$

(169)

converges to its limit P in (167) monotonically, so that $0\preccurlyeq \mathcal{P}\left(u\right)\preccurlyeq \mathcal{P}\left(s\right)\preccurlyeq P$ for any $s⩾u⩾0$.

Theorem 5. 

Suppose the system F in (7), (11) is subject to the worst-case noise W (at a given mean anisotropy level $a>0$) governed by the SDEs (149) as described in Theorem 4 and its proof. Then for any ϵ satisfying

$$0<ϵ<{\displaystyle \frac{1}{2}}\left(\sqrt{1+{4\parallel \rho N\parallel }_{\infty }^{-2}}-1\right)$$

(170)

in terms of the matrix ρ from (153) and the ${\mathcal{H}}_{\infty }$-norm of the transfer function (166), the Renyi and KL relative entropies (162), (165) have the following rates:

$$\begin{array}{cc}\hfill \underset{t\to +\infty }{lim}{\displaystyle \frac{{\mathbf{R}}_{ϵ}\left(t\right)}{t}}& =-{\displaystyle \frac{1}{4\pi ϵ}}{\int }_{\mathbb{R}}lndet(I_m-ϵ(1+ϵ)\rho N\left(i\omega \right)N{\left(i\omega \right)}^{*}{\rho }^{\mathrm{T}})\mathrm{d}\omega \hfill \\ \hfill & ⩾\underset{t\to +\infty }{lim}{\displaystyle \frac{{\mathbf{R}}_0\left(t\right)}{t}}={\displaystyle \frac{1}{2}}{\parallel \Xi \parallel }_{\mathrm{F}}^2,\hfill \end{array}$$

(171)

where

$$\Xi :=\rho \sqrt{P},$$

(172)

and P is the matrix from (167), (168).

Proof. 

The zero-initialized Gaussian state process X in (149) has zero mean and a continuous covariance function

$$\varkappa (s,u):=\mathbf{cov}(X\left(s\right),X\left(u\right))=\left\{\begin{array}{ccc}{\mathrm{e}}^{(s-u)\alpha }\mathcal{P}\left(u\right)& \mathrm{for}& s⩾u⩾0\\ \mathcal{P}\left(s\right){\mathrm{e}}^{(u-s){\alpha }^{\mathrm{T}}}& \mathrm{for}& u>s⩾0\end{array}\right.,$$

(173)

where use is made of the covariance matrix from (169). By the convergence (167), the resulting limit of (173) takes the form

$$\begin{array}{cc}\hfill {\varkappa }_{\infty }\left(\tau \right)& :=\underset{u\to +\infty }{lim}\varkappa (u+\tau ,u)=\left\{\begin{array}{ccc}{\mathrm{e}}^{\tau \alpha }P& \mathrm{if}& \tau ⩾0\\ P{\mathrm{e}}^{-\tau {\alpha }^{\mathrm{T}}}& \mathrm{if}& \tau <0\end{array}\right.\hfill \\ \hfill & ={\displaystyle \frac{1}{2\pi }}{\int }_{\mathbb{R}}{\mathrm{e}}^{i\omega \tau }N\left(i\omega \right)N{\left(i\omega \right)}^{*}\mathrm{d}\omega ,\tau \in \mathbb{R},\hfill \end{array}$$

(174)

which depends only on the time difference and is the covariance function of the process X in its stationary regime, where N is the transfer function from (166). For a fixed but otherwise arbitrary time horizon $t>0$, the function (173) gives rise to a compact positive semi-definite self-adjoint integral operator $H_t$ on the Hilbert space $L^2([0,t],{\mathbb{R}}^m)$, whose ${\mathbb{R}}^{m\times m}$-valued kernel function on the square ${[0,t]}^2$ is the covariance function $\rho \varkappa {\rho }^{\mathrm{T}}$ of the process $\rho X$, so that, similarly to (3),

$$\begin{array}{cc}\hfill \mathbf{E}\left({\int }_0^t\phi {\left(s\right)}^{\mathrm{T}}\rho X\left(s\right)\mathrm{d}s{\int }_0^t\psi {\left(u\right)}^{\mathrm{T}}\rho X\left(u\right)\mathrm{d}u\right)& ={\int }_0^t\phi {\left(s\right)}^{\mathrm{T}}{H}_t\left(\psi \right)\left(s\right)\mathrm{d}s,\hfill \\ \hfill H_t\left(\psi \right)\left(s\right)& :=\rho {\int }_0^t\varkappa (s,u){\rho }^{\mathrm{T}}\psi \left(u\right)\mathrm{d}u,0⩽s⩽t,\hfill \end{array}$$

(175)

for any $\phi ,\psi \in L^2([0,t],{\mathbb{R}}^m)$. The operator $H_t$ is majorized by a compact positive semi-definite self-adjoint Toeplitz integral operator $K_t$ on $L^2([0,t],{\mathbb{R}}^m)$ with the kernel function $\rho {\varkappa }_{\infty }{\rho }^{\mathrm{T}}$ associated with (174):

$$H_t\preccurlyeq K_t.$$

(176)

The $L^2$-induced operator norms of $H_t$ and $K_t$ coincide with their largest eigenvalues and are nondecreasing functions of time t satisfying

$$\begin{array}{cc}\hfill {\lambda }_{max}\left(H_t\right)& ⩽{\lambda }_{max}\left(K_t\right)⩽\underset{\tau \to +\infty }{lim}{\lambda }_{max}\left(K_{\tau }\right)\hfill \\ \hfill & =\underset{\omega \in \mathbb{R}}{sup}{\lambda }_{max}\left(\rho N\left(i\omega \right)N{\left(i\omega \right)}^{*}{\rho }^{\mathrm{T}}\right)={\parallel \rho N\parallel }_{\infty }^2.\hfill \end{array}$$

(177)

Here, the first inequality follows from (176), while the first equality follows from the fact that the function $\mathbb{R}\ni \omega \mapsto \rho N\left(i\omega \right)N{\left(i\omega \right)}^{*}{\rho }^{\mathrm{T}}\in {\mathbb{H}}_m^{+}$ is the Fourier transform of the kernel function $\rho {\varkappa }_{\infty }{\rho }^{\mathrm{T}}$ of the operator $K_t$ in view of (174) and from the asymptotic properties of spectra of Toeplitz operators [24]. Hence, for any $ϵ$ subject to (170), where the second inequality originates from the condition

$$ϵ(1+ϵ)<{\displaystyle \frac{1}{{\parallel \rho N\parallel }_{\infty }^2}},$$

(178)

the operator $ϵ(1+ϵ)H_t$ is a contraction, and so also is ${ϵ}^2{H}_t$. In combination with the fact that X is a zero-mean Gaussian process, the relations (177), (178) secure the fulfillment of the sufficient condition (164) for the martingale property of (163), which is used in (162), and make the right-hand side of (162) well-defined. Application of the results of [22] (see also the references therein) and the Fredholm formula [50] to the right-hand side of (162) yields

$$\begin{array}{cc}\hfill {\mathbf{R}}_{ϵ}\left(t\right)& =-{\displaystyle \frac{1}{2ϵ}}\mathrm{Tr}ln(\mathcal{I}-ϵ(1+ϵ)H_t)\hfill \\ \hfill & ={\displaystyle \frac{1}{2}}{\displaystyle \sum _{k=1}^{+\infty }}{\displaystyle \frac{1}{k}}{ϵ}^{k-1}{(1+ϵ)}^k\mathrm{Tr}\left(H_t^k\right),t>0,\hfill \end{array}$$

(179)

where $\mathcal{I}$ is the identity operator. Since the function $Q\mapsto -\mathrm{Tr}ln(\mathcal{I}-Q)\in {\mathbb{R}}_{+}$ is monotonic with respect to a self-adjoint operator $0\preccurlyeq Q\prec \mathcal{I}$ of finite trace, it follows from (176)–(179) that

$${\mathbf{R}}_{ϵ}\left(t\right)⩽-{\displaystyle \frac{1}{2ϵ}}\mathrm{Tr}ln(\mathcal{I}-ϵ(1+ϵ)K_t).$$

Hence,

$$\begin{array}{cc}\hfill \underset{t\to +\infty }{lim\; sup}{\displaystyle \frac{{\mathbf{R}}_{ϵ}\left(t\right)}{t}}& ⩽-{\displaystyle \frac{1}{2ϵ}}\underset{t\to +\infty }{lim}\left({\displaystyle \frac{1}{t}}\mathrm{Tr}ln(\mathcal{I}-ϵ(1+ϵ)K_t)\right)\hfill \\ \hfill & =-{\displaystyle \frac{1}{4\pi ϵ}}{\int }_{\mathbb{R}}lndet(I_m-ϵ(1+ϵ)\rho N\left(i\omega \right)N{\left(i\omega \right)}^{*}{\rho }^{\mathrm{T}})\mathrm{d}\omega ,\hfill \end{array}$$

(180)

where the last equality follows from the Szegö limit theorem for Toeplitz operators [24]. On the other hand, (162) implies that

$$\begin{array}{cc}\hfill {\mathbf{R}}_{ϵ}\left(t\right)& ⩾{\displaystyle \frac{1}{ϵ}}ln\mathbf{E}{\mathrm{e}}^{{\displaystyle \frac{1}{2}}ϵ(1+ϵ){\int }_{\tau }^t{\left|\rho X\left(s\right)\right|}^2\mathrm{d}s}\hfill \\ \hfill & =-{\displaystyle \frac{1}{2ϵ}}\mathrm{Tr}ln(\mathcal{I}-ϵ(1+ϵ)H_{\tau ,t}),0⩽\tau <t,\hfill \end{array}$$

(181)

where $H_{\tau ,t}$ is the covariance operator of the process $\rho X$ on the time interval $[\tau ,t]$. More precisely, $H_{\tau ,t}$ is an integral operator on the space

$$L^2([\tau ,t],{\mathbb{R}}^m)={\mathfrak{T}}_{\tau }\left(L^2([0,t-\tau ],{\mathbb{R}}^m)\right)$$

with the kernel function $\rho \varkappa {\rho }^{\mathrm{T}}$ on ${[\tau ,t]}^2$, where ${\mathfrak{T}}_{\tau }$ is the unitary operator of translation by $\tau$ acting on a function $\phi$ as ${\mathfrak{T}}_{\tau }\left(\phi \right)\left(s\right)=\phi (s-\tau )$, so that its adjoint is ${\mathfrak{T}}_{\tau }^{†}={\mathfrak{T}}_{-\tau }$. In particular, $H_{0,t}=H_t$ as defined in (175). Due to the Hurwitz property of the matrix $\alpha$ in (151) leading to the monotonic convergence in (167), for any given $\delta \in (0,1]$, there exists $\tau ⩾0$ such that the function (169) satisfies

$$\mathcal{P}\left(t\right)\succcurlyeq \mathcal{P}\left(\tau \right)\succcurlyeq (1-\delta )P,t⩾\tau .$$

(182)

This property can be verified (in the general case, when the pair $(\alpha ,\beta )$ is not necessarily controllable and P in (167) can be singular) by decomposing the state space into the orthogonal sum ${\mathbb{R}}^n=kerP\oplus \mathrm{im}P$ and noting that

$$x^{\mathrm{T}}{\mathrm{e}}^{\tau \alpha }P{\mathrm{e}}^{\tau {\alpha }^{\mathrm{T}}}x={\left|\sqrt{P}{\mathrm{e}}^{\tau {\alpha }^{\mathrm{T}}}x\right|}^2=0,x\in kerP$$

for any $\tau ⩾0$, while the matrix P is positive definite on the reachable subspace $\mathrm{im}P$, whereby $\sqrt{P}x\ne 0$ for any $x\in \mathrm{im}P\setminus \left\{0\right\}$. Therefore, (182) is equivalent to

$$\underset{x\in \mathrm{im}P:|\sqrt{P}x|=1}{max}\left|\sqrt{P}{\mathrm{e}}^{\tau {\alpha }^{\mathrm{T}}}x\right|⩽\sqrt{\delta }$$

and can indeed be satisfied for any given $\delta >0$ by choosing $\tau$ large enough due to $\alpha$ being Hurwitz. By the dependence of the covariance function (173) on $\mathcal{P}$ in (169), the fulfillment of the condition (182) implies that

$$H_{\tau ,t}\succcurlyeq (1-\delta ){\mathfrak{T}}_{\tau }{K}_{t-\tau }{\mathfrak{T}}_{-\tau },t>\tau ,$$

(183)

where use is also made of the translation invariance of the kernel function $\rho {\varkappa }_{\infty }{\rho }^{\mathrm{T}}$ of the Toeplitz operator $K_t$. By combining (181) with (183) and using the monotonicity argument as before along with the unitary equivalence (and thus, isospectrality) of the operators ${\mathfrak{T}}_{\tau }{K}_{t-\tau }{\mathfrak{T}}_{-\tau }$ and $K_{t-\tau }$, it follows that

$$\begin{array}{cc}\hfill {\mathbf{R}}_{ϵ}\left(t\right)& ⩾-{\displaystyle \frac{1}{2ϵ}}\mathrm{Tr}ln(\mathcal{I}-ϵ(1+ϵ)(1-\delta ){\mathfrak{T}}_{\tau }{K}_{t-\tau }{\mathfrak{T}}_{-\tau })\hfill \\ \hfill & =-{\displaystyle \frac{1}{2ϵ}}\mathrm{Tr}ln(\mathcal{I}-ϵ(1+ϵ)(1-\delta )K_{t-\tau }),t>\tau ,\hfill \end{array}$$

and hence,

$$\begin{array}{cc}\hfill \underset{t\to +\infty }{lim\; inf}{\displaystyle \frac{{\mathbf{R}}_{ϵ}\left(t\right)}{t}}& ⩾-{\displaystyle \frac{1}{2ϵ}}\underset{t\to +\infty }{lim}\left({\displaystyle \frac{1}{t}}\mathrm{Tr}ln(\mathcal{I}-ϵ(1+ϵ)(1-\delta )K_{t-\tau })\right)\hfill \\ \hfill & =-{\displaystyle \frac{1}{4\pi ϵ}}{\int }_{\mathbb{R}}lndet(I_m-ϵ(1+ϵ)(1-\delta )\rho N\left(i\omega \right)N{\left(i\omega \right)}^{*}{\rho }^{\mathrm{T}})\mathrm{d}\omega .\hfill \end{array}$$

(184)

Since $\delta \in (0,1]$ is arbitrary, then by letting $\delta \to 0+$, a comparison of (184) with (180) establishes the first equality in (171). The inequality and the second equality in (171) are obtained from (165), (167) by using (174) at $\tau =0$ along with (172). □

In view of the last equality in (162) and the fact that the function $\mathbb{R}\ni \omega \mapsto \rho N\left(i\omega \right)N{\left(i\omega \right)}^{*}{\rho }^{\mathrm{T}}\in {\mathbb{H}}_m^{+}$ is the spectral density of the Gaussian process $\rho X$ in the stationary regime of the first SDE from (149), the first equality in (171) is a particular case of the quadratic-exponential functional rates for stationary Gaussian processes in infinite-horizon risk-sensitive control [4,9,41].

The relative entropy rates (171) have the physical dimension of the inverse of time. The reciprocal quantity

$${\tau }_{*}:={\displaystyle \frac{2}{{\parallel \Xi \parallel }_{\mathrm{F}}^2}},$$

(185)

which involves the matrix (172), can be interpreted as characteristic time for the deviation of the worst-case noise W from the Wiener process $MV$ to manifest itself in terms of the relative entropies (162), (165) in the stationary regime of (149) when they take the form

$${\mathbf{R}}_{ϵ}\left(t\right)⩾{\mathbf{R}}_0\left(t\right)={\displaystyle \frac{t}{{\tau }_{*}}},t⩾0.$$

Note that such deviation can be detected in the long run (at time horizons $t\gg {\tau }_{*}$) by statistical hypothesis testing [38,70], although detailed analysis of this issue is beyond the scope of the present paper. Also, a related quantity

$${\displaystyle \frac{m}{{\parallel \Xi \parallel }_{\mathrm{F}}^2}}={\displaystyle \frac{m}{2}}{\tau }_{*}$$

(186)

describes qualitatively the local time scale at which the drift and diffusion terms in the modified form

$$M^{-1}\mathrm{d}W=\rho X\mathrm{d}t+\mathrm{d}V$$

of the second SDE from (149) make comparable contributions to the increment of the process $M^{-1}W$. This is understood at the level of the second-order moments (in the stationary regime) including the relation

$$\mathbf{E}\left(\right|V\left(s\right)-V\left(t\right){|}^2)=m|s-t|,s,t\in \mathbb{R},$$

(187)

for the standard Wiener process V in ${\mathbb{R}}^m$ from (21). More precisely, the drift-diffusion balance time (186) is obtained by formally equating $m\tau$ from (187) with ${\mathbf{E}\left(\right|\rho X|}^2){\tau }^2={\parallel \Xi \parallel }_{\mathrm{F}}^2{\tau }^2$ for the time difference $\tau :=|s-t|>0$.

The structure of the matrix $\Xi$ in (172) appearing in the KL relative entropy rate (171) is identical to that of the matrix ${\Xi }_T$ in (159) which participates in the temporal component (158) of the mean anisotropy level, with the matrices $\rho$, P in (153), (168) and ${\rho }_T$, $P_T$ in (142), (130) being counterparts of each other in continuous and discrete time, respectively. This resemblance is not coincidental and is substantiated in the proof of the following theorem.

Theorem 6. 

As in Theorem 5, suppose the system F is subject to the worst-case noise W governed by the SDEs (149) as specified in Theorem 4 and its proof. Then the temporal component (157) of the mean anisotropy level $a>0$ is linked to the KL relative entropy rate in (171) through the time scale (185) by

$$a_{\mathrm{temp}}⩽2\sigma {\displaystyle \frac{T}{{\tau }_{*}}},$$

(188)

where

$$\sigma :={\lambda }_{max}({\Gamma }^{-1}\Delta )$$

(189)

is the largest eigenvalue for the linear matrix pencil $\Delta -\lambda \Gamma$ associated with the matrices Γ, Δ from (148), (155).

Proof. 

The inequality (188) can be established as a corollary from the intermediate inequalities

$$a_{\mathrm{temp}}⩽{\displaystyle \frac{1}{2}}\parallel {\Xi }_T{\parallel }_{\mathrm{F}}^2⩽\sigma T{\parallel \Xi \parallel }_{\mathrm{F}}^2,$$

(190)

where the right-hand side coincides with that of (188) in view of (185). The first inequality in (190) is obtained by combining (158) with the fact that the function $lndet(·)$ is concave on the set of positive definite matrices (as mentioned before) and vanishes at the identity matrix, where the Frechet derivative of this function satisfies ${\partial }_Q{lndetQ|}_{Q=I_m}=I_m$, so that

$$lndet(I_m+{\Xi }_T{\Xi }_T^{\mathrm{T}})⩽\langle I_m,{\Xi }_T{\Xi }_T^{\mathrm{T}}\rangle ={\parallel {\Xi }_T\parallel }_{\mathrm{F}}^2.$$

We will now prove the second inequality in (190). The discrete-time state-space realization (144) gives rise to the transfer function for the map $\upsilon \mapsto x$:

$$N_T\left(z\right):=z{(I_n-z{\alpha }_T)}^{-1}{\beta }_T,z\in \mathbb{C},$$

(191)

with the matrices ${\alpha }_T$, ${\beta }_T$ from (129). In view of (134), the invariant covariance matrix $P_T$ of the stationary Gaussian sequence x in (145), which satisfies the discrete ALE (130), is expressed in terms of (191) as

$$P_T={\displaystyle \frac{1}{2\pi }}{\int }_{\mathbb{T}}{N}_T\left({\mathrm{e}}^{i\phi }\right)N_T{\left({\mathrm{e}}^{i\phi }\right)}^{*}\mathrm{d}\phi .$$

(192)

The matrix $P_T$ should not be confused with the invariant covariance matrix of the filtered state process $X_T$ in (46), which, in the worst-case noise scenario, is governed by the ODE

$${\dot{X}}_T=A{X}_T+B(L{X}_T+M{V}_T)=\alpha X_T+\beta V_T,$$

(193)

where the matrices (151) are used, and $V_T$ is the OU process as before. Indeed, by combining the spectral density $\Sigma I_m$ of $V_T$ from (52) with (166), (193) and the initial condition for $X_T$ from (49), it follows that

$$\begin{array}{cc}\hfill J_T& :=\underset{t\to +\infty }{lim}\mathbf{cov}\left(X_T\left(t\right)\right)\hfill \\ \hfill & ={\displaystyle \frac{1}{2\pi }}{\int }_{\mathbb{R}}\Sigma \left(\omega \right)N\left(i\omega \right)N{\left(i\omega \right)}^{*}\mathrm{d}\omega \hfill \\ \hfill & ={\displaystyle \frac{1}{2\pi }}{\int }_{\mathbb{T}}N\left(i\Omega tan{\displaystyle \frac{\phi }{2}}\right)N{\left(i\Omega tan{\displaystyle \frac{\phi }{2}}\right)}^{*}\mathrm{d}\phi \hfill \\ \hfill & ={\displaystyle \frac{1}{2\pi }}{\int }_{\mathbb{T}}N\left(-i\Omega tan{\displaystyle \frac{\phi }{2}}\right)N{\left(-i\Omega tan{\displaystyle \frac{\phi }{2}}\right)}^{*}\mathrm{d}\phi \hfill \\ \hfill & ={\displaystyle \frac{1}{2\pi }}{\int }_{\mathbb{T}}N(\Omega K\left({\mathrm{e}}^{i\phi }\right))N{(\Omega K\left({\mathrm{e}}^{i\phi }\right))}^{*}\mathrm{d}\phi \hfill \\ \hfill & ={\displaystyle \frac{1}{2\pi i}}{\oint }_{\left|z\right|=1}N(\Omega K\left(z\right))N{(\Omega K\left(z\right))}^{*}{\displaystyle \frac{\mathrm{d}z}{z}},\hfill \end{array}$$

(194)

with the integration variable change according to (75), (83). The last integrand in (194) can be computed similarly to (116) by using

$$\begin{array}{cc}\hfill N(\Omega K(z\left)\right)& ={\left(\Omega {\displaystyle \frac{1-z}{1+z}}{I}_n-\alpha \right)}^{-1}\beta \hfill \\ \hfill & =(1+z){(I_n-z{\alpha }_T)}^{-1}{\beta }_T\hfill \\ \hfill & =2z{(I_n-T\alpha )}^{-1}{(I_n-z{\alpha }_T)}^{-1}{\beta }_T+T{(I_n-T\alpha )}^{-1}\beta \hfill \\ \hfill & ={(I_n-T\alpha )}^{-1}(2N_T\left(z\right)+T\beta ),\hfill \end{array}$$

(195)

where the last equality involves the transfer function $N_T$ from (191). Substitution of (195) into (194) yields

$$\begin{array}{cc}\hfill J_T& ={\displaystyle \frac{1}{2\pi i}}{(I_n-T\alpha )}^{-1}{\oint }_{\left|z\right|=1}(2N_T\left(z\right)+T\beta )(2N_T{\left(z\right)}^{*}+T{\beta }^{\mathrm{T}}){\displaystyle \frac{\mathrm{d}z}{z}}{(I_n-T\alpha )}^{-\mathrm{T}}\hfill \\ \hfill & ={(I_n-T\alpha )}^{-1}\left({\displaystyle \frac{2}{\pi i}}{\oint }_{\left|z\right|=1}{N}_T\left(z\right)N_T{\left(z\right)}^{*}{\displaystyle \frac{\mathrm{d}z}{z}}+T^2\beta {\beta }^{\mathrm{T}}\right){(I_n-T\alpha )}^{-\mathrm{T}}\hfill \\ \hfill & ={(I_n-T\alpha )}^{-1}(4P_T+T^2\beta {\beta }^{\mathrm{T}}){(I_n-T\alpha )}^{-\mathrm{T}}\hfill \\ \hfill & =4{(I_n-T\alpha )}^{-1}{P}_T{(I_n-T\alpha )}^{-\mathrm{T}}+{\beta }_T{\beta }_T^{\mathrm{T}}.\hfill \end{array}$$

(196)

Here, (192) is used along with the relations

$${\oint }_{\left|z\right|=1}{N}_T\left(z\right){\displaystyle \frac{\mathrm{d}z}{z}}=0,{\oint }_{\left|z\right|=1}{N}_T{\left(z\right)}^{*}{\displaystyle \frac{\mathrm{d}z}{z}}={\oint }_{\left|z\right|=1}{N}_T{(1/z)}^{\mathrm{T}}{\displaystyle \frac{\mathrm{d}z}{z}}=0$$

in view of the vanishing residues ${\mathrm{Res}}_{z=0}{\displaystyle \frac{N_T\left(z\right)}{z}}=0$ and ${\mathrm{Res}}_{z=\infty }{\displaystyle \frac{N_T(1/z)}{z}}=0$ by the structure of (191). Also, the last equality in (196) employs the relation

$${\beta }_T=T{(I_n-T\alpha )}^{-1}\beta$$

between the input matrices of the worst-case noise shaping filter G in (150) and its discrete-time counterpart $G_T$ in (143). On the other hand, a combination of the second equality in (194) with (52), (167) leads to

$$J_T\preccurlyeq \underset{\omega \in \mathbb{R}}{sup}\Sigma \left(\omega \right){\displaystyle \frac{1}{2\pi }}{\int }_{\mathbb{R}}N\left(i\omega \right)N{\left(i\omega \right)}^{*}\mathrm{d}\omega =2TP.$$

(197)

Therefore, by (196), (197),

$$4{(I_n-T\alpha )}^{-1}{P}_T{(I_n-T\alpha )}^{-\mathrm{T}}+{\beta }_T{\beta }_T^{\mathrm{T}}\preccurlyeq 2TP.$$

(198)

The left and right multiplication of both sides of (198) by the matrices L and $L^{\mathrm{T}}$, respectively, leads to

$$L_T{P}_T{L}_T^{\mathrm{T}}\preccurlyeq L_T{P}_T{L}_T^{\mathrm{T}}+L{\beta }_T{\beta }_T^{\mathrm{T}}{L}^{\mathrm{T}}\preccurlyeq 2TLP{L}^{\mathrm{T}}.$$

(199)

Here, use is also made of the relation

$$L_T=2L{(I_n-T\alpha )}^{-1}$$

between the output matrices of the worst-case noise shaping filter G in (150) and its discrete-time counterpart $G_T$ in (143). The left and right multiplication of (199) by the symmetric matrix $M_T^{-1}$ yields

$$\begin{array}{cc}\hfill {\Xi }_T{\Xi }_T^{\mathrm{T}}& ={\rho }_T{P}_T{\rho }_T^{\mathrm{T}}=M_T^{-1}{L}_T{P}_T{L}_T^{\mathrm{T}}{M}_T^{-1}\preccurlyeq 2T{M}_T^{-1}LP{L}^{\mathrm{T}}{M}_T^{-1}\hfill \\ \hfill & =2T{M}_T^{-1}M\rho P{\rho }^{\mathrm{T}}{M}^{\mathrm{T}}{M}_T^{-1}=2T{M}_T^{-1}M\Xi {\Xi }^{\mathrm{T}}{M}^{\mathrm{T}}{M}_T^{-1},\hfill \end{array}$$

(200)

where use is also made of the matrices ${\Xi }_T$, ${\rho }_T$ from (159), (142) and $\Xi$, $\rho$ from (172), (153). By taking the trace on both sides of (200), it follows that

$$\begin{array}{cc}\hfill \parallel {\Xi }_T{\parallel }_{\mathrm{F}}^2& ⩽2T\mathrm{Tr}(M_T^{-1}M\Xi {\Xi }^{\mathrm{T}}{M}^{\mathrm{T}}{M}_T^{-1})\hfill \\ \hfill & =2T\mathrm{Tr}({\Xi }^{\mathrm{T}}{M}^{\mathrm{T}}{M}_T^{-2}M\Xi )\hfill \\ \hfill & ⩽2T{\lambda }_{max}\left(M^{\mathrm{T}}{\Gamma }^{-1}M\right){\parallel \Xi \parallel }_{\mathrm{F}}^2,\hfill \end{array}$$

(201)

with $\Gamma$ from (148). Since the matrix $M^{\mathrm{T}}{\Gamma }^{-1}M$ is isospectral to ${\Gamma }^{-1}M{M}^{\mathrm{T}}={\Gamma }^{-1}\Delta$ in view of (155), with the largest eigenvalue in (189), then (201) establishes the second inequality in (190), thus completing the proof. □

Since Theorem 6 involves the generalized spectrum of the matrices $\Delta$ and $\Gamma$ which quantify the second-order moments of the innovation components of the worst-case disturbance in its continuous and discrete-time representations, we provide an upper bound for it as follows.

Lemma 1. 

The largest generalized eigenvalue (189) for the matrices Γ, Δ in (148), (155), associated with the worst-case noise shaping filter in Theorem 4, satisfies

$$\sigma ⩽1+q{\lambda }_{max}((D^{\mathrm{T}}D-D_T^{\mathrm{T}}{D}_T)\Delta ),$$

(202)

where q is the parameter from (135).

Proof. 

From (148) and positive semi-definiteness of the solution $R_T$ of the ARE (131)–(133), it follows that

$${\Gamma }^{-1}=M_T^{-2}=I_m-q{D}_T^{\mathrm{T}}{D}_T-B_T^{\mathrm{T}}{R}_T{B}_T\preccurlyeq I_m-q{D}_T^{\mathrm{T}}{D}_T.$$

(203)

Since $\Delta \succ 0$ in (155), the matrix ${\Gamma }^{-1}\Delta$ is isospectral to $\sqrt{\Delta }{\Gamma }^{-1}\sqrt{\Delta }$ which, in view of (203), satisfies

$$\begin{array}{cc}\hfill \sqrt{\Delta }{\Gamma }^{-1}\sqrt{\Delta }& \preccurlyeq \sqrt{\Delta }(I_m-q{D}_T^{\mathrm{T}}{D}_T)\sqrt{\Delta }\hfill \\ \hfill & =\sqrt{\Delta }({\Delta }^{-1}+q(D^{\mathrm{T}}D-D_T^{\mathrm{T}}{D}_T))\sqrt{\Delta }\hfill \\ \hfill & =I_m+q\sqrt{\Delta }(D^{\mathrm{T}}D-D_T^{\mathrm{T}}{D}_T)\sqrt{\Delta }.\hfill \end{array}$$

(204)

Due to monotonicity of the function ${\lambda }_{max}(·)$ on the subspace of real symmetric (or complex Hermitian) matrices with respect to the cone of positive semi-definite matrices, (204) implies that

$${\lambda }_{max}\left(\sqrt{\Delta }{\Gamma }^{-1}\sqrt{\Delta }\right)⩽1+q{\lambda }_{max}\left(\sqrt{\Delta }(D^{\mathrm{T}}D-D_T^{\mathrm{T}}{D}_T)\sqrt{\Delta }\right),$$

which leads to (202) in view of the same similarity transformation and isospectrality argument as above. □

The inequality (188) remains valid if the geometric factor $\sigma$ is replaced with its upper bound from (202). This inequality, or its equivalent form

$${\tau }_{*}⩽{\displaystyle \frac{2\sigma }{a_{\mathrm{temp}}}}T,$$

provides an insight into the relation between the temporal part $a_{\mathrm{temp}}$ of the mean anisotropy level $a>0$ and the degree of distinguishability of the worst-case noise W from its martingale component $MV$ in terms of the characteristic time ${\tau }_{*}$ for their discrepancy to manifest itself through relative entropy. This makes Theorem 6 applicable to a rational choice of the parameter a as the amount of statistical uncertainty in the external disturbance acting on the LCTI system.

11. Continuous-Time Anisotropic Norm Computation for RLC Circuit

As a numerical illustration of the above results, consider a two-loop RLC circuit in Figure 7 with resistance, inductance and capacitance coefficients $R_k,L_k,C_k>0$ for the kth loop, $k=1,2$.

The circuit loops are inductively coupled, which is quantified by the mutual inductance coefficient $L_{12}$, so that the magnetic field energy stored by the coupled inductors is given by the quadratic form

$${\displaystyle \frac{1}{2}}{\displaystyle \sum _{k=1}^2}{L}_k{J}_k^2+L_{12}{J}_1{J}_2={\displaystyle \frac{1}{2}}{\parallel J\parallel }_{\Pi }^2,J:=\left[\begin{array}{c}{J}_1\\ J_2\end{array}\right].$$

(205)

Here, $J_k$ is the current in the kth loop, and use is made of an auxiliary positive definite matrix

$$\Pi :=\left[\begin{array}{cc}{L}_1& L_{12}\\ L_{12}& L_2\end{array}\right].$$

(206)

The loops involve two noise sources (for example, of thermal or crosstalk origin [45,75]) with irregular voltages whose integrals over time (such as flux linkages) are represented by an ${\mathbb{R}}^2$-valued Ito process

$$W:=\left[\begin{array}{c}{W}_1\\ W_2\end{array}\right].$$

(207)

Accordingly, the integrals of the noise voltages are understood in the generalized sense since W is not differentiable with respect to time. The electric charges $Q_k$ at the capacitors have continuously differentiable sample paths with probability one, while their time derivatives

$${\dot{Q}}_k=J_k$$

are Ito processes. With $Q_k$ playing the role of generalized coordinates, the Lagrangian dynamics formalism along with (205) and Kirchhoff’s voltage law lead to a set of two SDEs

$$L_k\mathrm{d}{\dot{Q}}_k+L_{12}\mathrm{d}{\dot{Q}}_{3-k}+\left(R_k{\dot{Q}}_k+{\displaystyle \frac{1}{C_k}}{Q}_k\right)\mathrm{d}t=\mathrm{d}{W}_k,k=1,2,$$

(208)

which are coupled through the mutual inductance $L_{12}$. The SDEs (208) can be assembled into one SDE (11) for the state process X, given by

$$X:=\left[\begin{array}{c}Q\\ \dot{Q}\end{array}\right],Q:=\left[\begin{array}{c}{Q}_1\\ Q_2\end{array}\right],$$

(209)

with the state and input dimensions $n=4$, $m=2$ in view of (207) and the state-space realization matrices

$$A:=\left[\begin{array}{cc}0& I_2\\ -{\Pi }^{-1}E& -{\Pi }^{-1}R\end{array}\right],B:=\left[\begin{array}{c}0\\ {\Pi }^{-1}\end{array}\right].$$

(210)

The latter are expressed in terms of the matrix $\Pi$ from (206) and (also positive definite) matrices

$$E:=\left[\begin{array}{cc}{\displaystyle \frac{1}{C_1}}& 0\\ 0& {\displaystyle \frac{1}{C_2}}\end{array}\right],R:=\left[\begin{array}{cc}{R}_1& 0\\ 0& R_2\end{array}\right].$$

(211)

The resulting system F is a stable linear dissipative stochastic Hamiltonian system [66], with $\Pi$, E, R playing the role of the generalized mass, stiffness and damping matrices, respectively, and the Hamiltonian

$${\displaystyle \frac{1}{2}}{(\parallel Q\parallel }_E^2+\parallel \dot{Q}{\parallel }_{\Pi }^2)={\displaystyle \frac{1}{2}}{(\parallel Q\parallel }_E^2+\parallel \Pi \dot{Q}{\parallel }_{{\Pi }^{-1}}^2)$$

describing the total electric and magnetic field energy stored by the capacitors and inductors of the RLC circuit. Here, the vector $\Pi \dot{Q}$, consisting of the flux linkages at the inductors, corresponds to the generalized momentum in accordance with the electromechanical analogy. The Hurwitz property of the matrix A in (210) is secured by the positive definiteness of the matrices $\Pi$ and (211).

If the robust performance analysis of the RLC circuit is concerned, for example, with the sensitivity of the voltage ${\displaystyle \frac{Q_1}{C_1}}$ at the first capacitor and the voltage $R_2{J}_2=R_2{\dot{Q}}_2$ at the second resistor to the voltage noises, then the system output Z in (7) can be specified by the matrices

$$C:=\left[\begin{array}{cccc}{\displaystyle \frac{1}{C_1}}& 0& 0& 0\\ 0& 0& 0& R_2\end{array}\right],D:=0,$$

(212)

in accordance with (209). Indeed, an appropriately scaled version of the filtered output process $Z_T$ in (46) satisfies

$${\displaystyle \frac{1}{\sqrt{2T}}}{Z}_T\left(t\right)=C{\displaystyle \frac{1}{\sqrt{2T}}}{X}_T\left(t\right)\stackrel{\mathrm{a}.\mathrm{s}.}{⟶}CX\left(t\right)=\left[\begin{array}{c}{\displaystyle \frac{1}{C_1}}{Q}_1\left(t\right)\\ R_2{J}_2\left(t\right)\end{array}\right],\mathrm{as}T\to 0+,$$

for any $t⩾0$ in view of (50), and thus asymptotically reproduces the first capacitor and second resistor voltages of interest.

With the following values of the circuit parameters (their physical dimensions are omitted for brevity)

$$\begin{array}{cc}\hfill R_1& =0.3190,R_2=0.6324,\hfill \\ \hfill L_1& =1.1235,L_2=2.2478,L_{12}=0.8593,\hfill \\ \hfill C_1& =0.4927,C_2=0.2501\hfill \end{array}$$

in (206), (210)–(212), the state-space realization of the system F being considered is given by

$$F\leftrightarrow \left[\begin{array}{cccccc}0& 0& 1.0000& 0& 0& 0\\ 0& 0& 0& 1.0000& 0& 0\\ -2.5530& 1.9227& -0.4013& 0.3041& 1.2579& -0.4809\\ 0.9760& -2.5138& 0.1534& -0.3976& -0.4809& 0.6287\\ 2.0296& 0& 0& 0& 0& 0\\ 0& 0& 0& 0.6324& 0& 0\end{array}\right].$$

The spectrum of the matrix A is

$$\mathfrak{S}=\{-0.3077\pm 1.9516i,-0.0917\pm 1.0747i\}$$

(213)

(so that the system is highly resonant with two lightly damped oscillatory modes), and the bounds in (80) are

$${\displaystyle \frac{1}{\mathbf{r}\left(A\right)}}=0.5062,\mathbf{r}\left(A^{-1}\right)=0.9271.$$

The lower of these bounds (see the first relation in (80)) exceeds noticeably the time scale parameter

$$T=0.0810$$

(214)

of the conformal correspondence which is used in what follows along with the cutoff frequency

$$\Omega =12.3481$$

from (25). Therefore, the effective frequency range of the low-pas filter $\Phi$ in (22)–(24), which specifies the filtered input and output $W_T$, $Z_T$ for the RMS gain (54), is adequate for capturing the transient processes in the system F. The matrices (118)–(121) of its discrete-time counterpart $F_T$ take the form

$$F_T\leftrightarrow \left[\begin{array}{cccccc}0.9685& 0.0229& 0.1545& 0.0055& 0.0078& -0.0029\\ 0.0116& 0.9690& 0.0028& 0.1545& -0.0029& 0.0039\\ -0.3890& 0.2832& 0.9074& 0.0677& 0.0958& -0.0354\\ 0.1438& -0.3832& 0.0342& 0.9084& -0.0354& 0.0479\\ 3.9953& 0.0465& 0.3135& 0.0111& 0.0158& -0.0058\\ 0.0909& -0.2423& 0.0217& 1.2069& -0.0224& 0.0303\end{array}\right].$$

The behaviour of the singular values of the transfer function $F_T$ in (116) on the unit circle (see Figure 8) shows that the system $F_T$ is substantially nonround (and hence, so also is the system F).

Accordingly, there is a large gap between the scaled ${\mathcal{H}}_2$-norm and the ${\mathcal{H}}_{\infty }$-norm of the discrete-time system $F_T$:

$${\displaystyle \frac{1}{\sqrt{m}}}\parallel F_T{\parallel }_2=0.4701,{\parallel F_T\parallel }_{\infty }=3.8617.$$

These norms are the endpoints of the range of the $(T,a)$-anisotropic norm $\left|\right||F{\left|\right||}_{T,a}$ of the system (as the mean anisotropy level a varies from 0 to $+\infty$), computed using Theorem 4 (with Newton’s iterations [60, Section 4], [14, Section 9]) and shown in Figure 9.

For example, at

$$a=0.5000,$$

(215)

the norm being considered is

$$\left|\right||F{\left|\right||}_{T,a}=2.4650,$$

and the matrices (138) of the worst-case noise shaping filter (149) are given by

$$L=\left[\begin{array}{cccc}0.0697& 0.0861& 0.2543& 0.2487\\ -0.0422& 0.0009& 0.1163& 0.2334\end{array}\right],M=\left[\begin{array}{cc}1.0000& -0.0002\\ 0.0002& 1.0000\end{array}\right].$$

(216)

In comparison with (213), the spectrum of the corresponding matrix $\alpha$ in (151) is closer to the imaginary axis:

$$\{-0.2528\pm 1.9485i,-0.0011\pm 1.0659i\},$$

so that the parasitic feedback loop in Figure 2 leads to a less stable system (20). While the matrix L is relatively small, M is close to the identity matrix $I_2$. Moreover, since $D=0$ in this example as indicated in (212), the matrix M is orthogonal in view of (155), although its orthogonality is slightly distorted in (216) by the finite accuracy of digital representation. The resulting martingale part $MV$ of the worst-case noise W has the identity diffusion matrix

$$\Delta =I_2$$

and thus inherits from V the property of being a standard Wiener process in ${\mathbb{R}}^2$. In the worst-case noise scenario at the mean anisotropy level (215), the spatial and temporal components in (156)–(158) are

$$a_{\mathrm{spat}}=0.0752,a_{\mathrm{temp}}=0.4248.$$

(217)

The spatial component comes from the covariance matrix (146) of the auxiliary stationary Gaussian sequence (91), which is also the invariant covariance matrix of the filtered version $W_T$ of W:

$$L_T{P}_T{L}_T^{\mathrm{T}}+M_T^2=\underset{t\to +\infty }{lim}\mathbf{cov}\left(W_T\left(t\right)\right)=\left[\begin{array}{cc}1.9948& 0.5508\\ 0.5508& 1.3801\end{array}\right]$$

(218)

(its eigenvalues are $2.3182$ and $1.0567$). The KL relative entropy rate (171) of the worst-case noise W with respect to its martingale part $MV$, and the characteristic time scale (185) for their discrepancy to manifest itself are

$${\displaystyle \frac{1}{2}}{\parallel \Xi \parallel }_{\mathrm{F}}^2=4.1316,{\tau }_{*}=0.2420.$$

(219)

The geometric factor in (189) is

$$\sigma =0.9873,$$

and its upper bound in (202) is $1.0000$. In addition to (217), the intermediate term in (190) and the right-hand side of (188) are

$${\displaystyle \frac{1}{2}}{\parallel {\Xi }_T\parallel }_{\mathrm{F}}^2=0.6431,2\sigma {\displaystyle \frac{T}{{\tau }_{*}}}=0.6607.$$

The time scale ${\tau }_{*}$ in (219) noticeably (yet insignificantly) exceeds the Tustin transform parameter T in (214), with ${\displaystyle \frac{{\tau }_{*}}{T}}=2.9887$. In this sense, the worst-case noise at the mean anisotropy level (215) can informally be qualified as moderately coloured, while comparison of ${\tau }_{*}$ with other time scales can also be considered. Typical sample paths of the filtered versions

$$W_T=L{X}_T+M{V}_T,Z_T=(C+DL)X_T+DM{V}_T$$

of the noise process W and the output signal Z of the RLC circuit are provided in Figure 10, Figure 11 along with those of $L{X}_T$.

They are obtained by the numerical integration of the SDE

$$\mathrm{d}\left[\begin{array}{c}{X}_T\\ V_T\end{array}\right]=\left[\begin{array}{cc}\alpha & \beta \\ 0& -\Omega I_m\end{array}\right]\left[\begin{array}{c}{X}_T\\ V_T\end{array}\right]\mathrm{d}t+\left[\begin{array}{c}0\\ \sqrt{2\Omega }{I}_m\end{array}\right]\mathrm{d}V$$

(which combines the ODE (193) with the SDE (22) and uses the matrices (151)) in such a way that the probability law of the jointly Gaussian processes $X_T$, $V_T$ in their stationary regime is exactly reproduced. Note that the elliptical shape and tilt of the scatter plot in Figure 10 is in accordance with the eigenstructure of the invariant covariance matrix (218), including the positive correlation between the components of $W_T$ considered at the same moment of time.

12. Concluding Remarks

We have outlined a particular way of extending the discrete-time anisotropy-based criteria to LCTI systems governed by SDEs with statistically uncertain Ito processes as external noise inputs. This approach uses the RMS gain of the system in terms of filtered versions of the input and output processes. The time scale T of the low-pass filter parameterizes Tustin’s transform of the original LCTI system F to its LDTI counterpart $F_T$ with the same RMS gain, though understood in the usual sense for fragments (or steady-state RMS values) of stationary Gaussian random sequences.

The resulting two-parameter norm $\left|\right||F{\left|\right||}_{T,a}$ is defined as the a-anisotropic norm $\left|\right||F_T{\left|\right||}_a$ of the effective discrete-time system, with the input mean anisotropy threshold a quantifying indirectly the deviation of the spectral density of the Ito process at the input of the original system from constant scalar matrices. The computation of this norm reduces to the numerical solution of a set of matrix algebraic Riccati, Lyapunov and log-determinant equations developed 30 years ago in the discrete-time anisotropy-based control theoretic framework. Under Tustin’s inverse transform, the worst-case disturbance inherits the general feedback structure, whereby its drift depends linearly on the internal state of the system.

Therefore, in combination with Tustin’s direct and inverse transforms, the methods of anisotropy-based robust performance analysis and control design, developed earlier for discrete-time systems, are applicable to continuous-time settings. In particular, this allows the state-space solutions [61,63] of discrete-time anisotropy-based control and filtering problems to be converted to their continuous-time counterparts. However, such extension is limited to infinite-horizon settings for time invariant systems since it substantially employs the commutativity between two LCTI systems as input-output operators, provided one of them has a scalar transfer function. Accordingly, an extension of finite-horizon time-varying formulations of the discrete-time anisotropy-based theory to their continuous-time counterparts requires the development of alternative approaches, which will be reported elsewhere.