Optimal Control of Discrete Time-varying System with Multiple Delays and Multiplicative Noises

1. Introduction

Linear quadratic Gaussian (LQG) control problem stems from the optimal stochastic control theory of the systems with additive Gaussian white noises and state/control-dependent, which combines the concept of linear quadratic regulators for full state feedback and Kalman filters for state estimation [1,2,3]. Recently, the optimal LQG control has been applied in various fields, such as the robots of power substation, all-electric vehicles, electrical safety engineering networked control systems (NCSs) [5,6,7,9]. Specificly, for mobile monitoring robot in a ultrahigh-voltage power substation, the LQG conotrol is proposed to minimize the difference between the actual SNIR and its expectation and the change in transmitting power[4]. These motivate us to study the more complicated LQG control systems with multiple input delays and multiplicative noises.

It is generally known that random time delay and packet dropout always occur in the data transmission of NCSs. Many literatures have been investigated on LQG control problems with input delays and packet loss [8,10,11,12,13,14]. Basin [8] presented an optimal linear regulator (LQR) with input delay by using the duality principle. Cacace[10] studied the LQG problems for linear system with single input delay. Matni[11] presented an explicit solution to a two-player distributed LQG problem in which communication between controllers occurs across a communication link with varying delay. Basin[12] further established a necessary and sufficient condition of the optimal LQR control for the linear system with multiple input delays. Zhang[14] studied the classical LQR problem with multiple input delays for both continuous-time and discrete-time cases.

On the other hand, packet dropout is generally described as the multiplicative noises. Many references have focused on the LQG system with multiplicative noises [15,16,17]. Gupta[15] solved the optimal LQG problem with packet-dropping links by decomposing the problem into a standard LQR state-feedback controller designing. Liang[16] studied the optimal control and stabilization problems for NCSs with remote controller and local controller subject to packet dropout. For systems with both input delay and packet dropout, Liang[18] presented the optimal LQR controller, and derived the necessary and sufficient condition for the mean-square stabilization. Liang[19] considered the discrete-time LQG system with input delay and multiplicative noises, and obtained both optimal state feedback controller and suboptimal output feedback controller.

The aforementioned literatures are mainly focused on single delay and packet dropout. To our best knowledge, little progress has been made on the optimal LQG control for time-varying systems with multiple input delays and multiplicative noises.

Motivated by the work of [18,19,20], this paper studies the optimal LQG control for discrete time-varying system involving with multiple delays and multiplicative noises. The main contributions of this paper are summarized as follows: 1) When the state variables can be observed exactly, by introducing the stochastic maximum principle for system with multiple delays and multiplicative noises, a solution to the forward backward differential equations (FBSDEs) is obtained based on the coupled Riccati equations. 2) In terms of the solution to the FBSDEs, a necessary and sufficient condition is given for the optimal LQG control. 3) When the state variables are partially observed, we derive a suboptimal linear output feedback controller by linearizing the optimal estimator and neglecting higher order terms.

The rest of the paper is organized as follows. In Section 2, we give the results of optimal state feedback control problem. In Section 3, we derive a suboptimal linear output feedback controller for the LQG systems involving multiple input delays and multiplicative noises. Numerical examples are provided in Section 4. Conclusions are given in Section 5.

Notation:$R^n$ denotes the n-dimensional real Euclidean space. I presents the unit matrix of appropriate dimension. The superscript ${}^{\prime }$ denotes the transpose of the matrix. $\{\Omega ,\mathcal{F},\mathcal{P},{\left\{{\mathcal{F}}_k\right\}}_{k\ge 0}\}$ denotes a complete probability space on which random variable $w_k$ are defined such that ${\left\{{\mathcal{F}}_k\right\}}_{k\ge 0}$ is the natural filtration generated by $w_k$ and ${\nu }_k$, i.e., ${\mathcal{F}}_k=\sigma \{w_0,\dots ,w_k,{\nu }_0,\dots ,{\nu }_k\}$, augmented by all the $\mathcal{P}-$null sets in $\mathcal{F}$. A symmetric $A>0(\ge 0)$ means that it is a positive definite (positive semi-definite) matrix. $Tr\left(A\right)$ represents the trace of matrix A.

2. State Feedback Controller

When the state variable $x_k$ can be observed exactly, we consider the following discrete time-varying LQG system with multiple input delays and multiplicative noises

$$\begin{array}{cc}\hfill x_{k+1}& =\left[C\left(k\right)+{\nu }_k\overline{C}\left(k\right)\right]x_k+\left[D_0\left(k\right)+{\nu }_k{\overline{D}}_0\left(k\right)\right]u_k\hfill \\ \hfill & +\left[D_d\left(k\right)+{\nu }_k{\overline{D}}_d\left(k\right)\right]u_{k-d}+w_k,\hfill \end{array}$$

(1)

where $x_k\in R^n$ is the state, $u_k\in R^m$ is the input control with the delay $d>0$, ${\nu }_k$ is the scalar white noise with zero mean and variance ${\varphi }^2$, $w_k\in R^n$ is the random variables satisfying $E\left[w_k\right|{\mathcal{F}}_{k-1}]={\overline{w}}_k$ and $E\left[w_k{w}_k^{\prime }\right]=Q_{w_k}$. $C\left(k\right),\overline{C}\left(k\right),D_i\left(k\right)$ and ${\overline{D}}_i\left(k\right)$ with $i=0,d$ are coefficient matrices with compatible dimensions. ${\nu }_k$ and $w_k$ are correlated with $E\left[v_k{w}_k^{\prime }\right|{\mathcal{F}}_{k-1}]=\rho$, $E\left[v_k{w}_l^{\prime }\right]=0$, $k\ne l$. The initial state $x_0$, $u_i$ for $i=-d,\dots ,-1$ are known.

The associated cost function for system (1) is given by

$$\begin{array}{c}\hfill J_N=E\left\{\sum _{k=0}^N{x_k}^{\prime }{Q}_k{x}_k+{u_k}^{\prime }{R}_k{u}_k+{x_{N+1}}^{\prime }{\mathcal{P}}_{N+1}{x}_{N+1}\right\},\end{array}$$

(2)

where $Q_k$ and ${\mathcal{P}}_{N+1}$ are positive semi-definite constant matrices with appropriate dimensions, control cost matrix $R_k$ should be positive definite matrix, and N is the horizon length.

Problem 1. 

Find the unique ${\mathcal{F}}_{k-1}$-measurable state feedback controller $u_k$, for $k=0,\dots ,N$, to minimize (2) subject to (1).

For simplicity, we make the following definitions

$$\begin{array}{c}\hfill C_k\left(k\right)=C\left(k\right)+{\nu }_k\overline{C}\left(k\right),D_k^i\left(k\right)=D_i\left(k\right)+{\nu }_k{\overline{D}}_i\left(k\right),\end{array}$$

for $i=0,d$. Then the system (1) becomes

$$\begin{array}{c}\hfill x_{k+1}=C_k\left(k\right)x_k+D_k^0\left(k\right)u_k+D_k^d\left(k\right)u_{k-d}+w_k.\end{array}$$

(3)

Following the similar discussion of [19], in virtue of the Pontryagin’s maximum principle for (3) and (2), we have

$$\begin{array}{cc}\hfill {\zeta }_N=& {\mathcal{P}}_{N+1}{x}_{N+1},\hfill \end{array}$$

(4)

$$\begin{array}{cc}\hfill {\zeta }_{k-1}=& E\left[C_k^{\prime }\left(k\right){\zeta }_k\right|{\mathcal{F}}_{k-1}]+Q_k{x}_k,\hfill \end{array}$$

(5)

$$\begin{array}{cc}\hfill 0=& E[{\left(D_k^0\left(k\right)\right)}^{\prime }{\zeta }_k+{\left(D_{k+d}^d(k+d)\right)}^{\prime }{\zeta }_{k+d}|{\mathcal{F}}_{k-1}]+R_k{u}_k,\hfill \end{array}$$

(6)

for $k=0,\dots ,N$, where ${\zeta }_k$ is the costate with ${\zeta }_k=0$ for $k>N$.

For further study, the following coupled Riccati difference equations are given:

$$\begin{array}{cc}\hfill {\mathcal{P}}_k=& C^{\prime }\left(k\right){\mathcal{P}}_{k+1}C\left(k\right)+{\varphi }^2{\overline{C}}^{\prime }\left(k\right){\mathcal{P}}_{k+1}\overline{C}\left(k\right)\hfill \\ \hfill & -M_k^{\prime }{\Omega }_k^{-1}{M}_k+Q_k,\hfill \end{array}$$

(7)

where

$$\begin{array}{cc}\hfill {\Omega }_k& =R_k+D_0^{\prime }\left(k\right){\mathcal{P}}_{k+1}{D}_0\left(k\right)+{\varphi }^2{\overline{D}}_0^{\prime }\left(k\right){\mathcal{P}}_{k+1}{\overline{D}}_0\left(k\right)\hfill \\ \hfill & +D_d^{\prime }(k+d){\mathcal{P}}_{k+d+1}{D}_d(k+d)+{\varphi }^2{\overline{D}}_d^{\prime }(k+d)\hfill \\ \hfill & \times {\mathcal{P}}_{k+d+1}{\overline{D}}_d(k+d)+D_0^{\prime }\left(k\right){\mathcal{P}}_{k+1}^{d-1}+{\left({\mathcal{P}}_{k+1}^{d-1}\right)}^{\prime }\hfill \\ \hfill & \times D_0\left(k\right)-\sum _{i=1}^d{\left(M_{k+i}^{d-i}\right)}^{\prime }{\Omega }_{k+i}^{-1}{M}_{k+i}^{d-i},\hfill \end{array}$$

(8)

$$\begin{array}{cc}\hfill M_k& =D_0^{\prime }\left(k\right){\mathcal{P}}_{k+1}C\left(k\right)+{\varphi }^2{\overline{D}}_0^{\prime }\left(k\right){\mathcal{P}}_{k+1}\overline{C}\left(k\right)\hfill \\ \hfill & +{\left({\mathcal{P}}_{k+1}^{d-1}\right)}^{\prime }C\left(k\right),\hfill \end{array}$$

(9)

with

$$\begin{array}{cc}\hfill M_k^0& =D_0^{\prime }\left(k\right){\mathcal{P}}_{k+1}{D}_d\left(k\right)+{\varphi }^2{\overline{D}}_0^{\prime }\left(k\right){\mathcal{P}}_{k+1}{\overline{D}}_d\left(k\right)\hfill \\ \hfill & +{\left({\mathcal{P}}_{k+1}^{d-1}\right)}^{\prime }{D}_d\left(k\right),\hfill \end{array}$$

(10)

$$\begin{array}{cc}\hfill M_k^j& =D_0^{\prime }\left(k\right){\mathcal{P}}_{k+1}^{j-1}+{\left({\mathcal{P}}_{k+j+1}^{d-j-1}\right)}^{\prime }{D}_d(k+j)\hfill \\ \hfill & -\sum _{i=1}^j{\left(M_{k+i}^{d-i}\right)}^{\prime }{\Omega }_{k+i}^{-1}{M}_{k+i}^{j-i},\hfill \end{array}$$

(11)

$$\begin{array}{cc}\hfill {\mathcal{P}}_k^0& =C^{\prime }\left(k\right){\mathcal{P}}_{k+1}{D}_d\left(k\right)+{\varphi }^2{\overline{C}}^{\prime }\left(k\right){\mathcal{P}}_{k+1}{\overline{D}}_d\left(k\right)\hfill \\ \hfill & -M_k^{\prime }{\Omega }_k^{-1}{M}_k^0,\hfill \end{array}$$

(12)

$$\begin{array}{cc}\hfill {\mathcal{P}}_k^j& =C^{\prime }\left(k\right){\mathcal{P}}_{k+1}^{j-1}-M_k^{\prime }{\Omega }_k^{-1}{M}_k^j,j=1,\dots ,d-1.\hfill \end{array}$$

(13)

The terminal values are given by

$$\begin{array}{cc}\hfill & {\mathcal{P}}_{N+1},{\mathcal{P}}_{N+i+1}=0,{\mathcal{P}}_{N+i}^j=0,\hfill \\ \hfill & M_{N+i}^j=0,{\Omega }_{N+i}=I,i\ge 1,j=0,\dots ,d-1.\hfill \end{array}$$

(14)

Remark 1. 

As can be seen that the costate equations (4)-(6) are quite different from those of Liang [19] and Zhang [20]. What’s more, the coupled Riccati equations (7)-(13) are more complicated than those in Liang [19] and Zhang [20].

It is stressed that the key to solve the optimal LQG control problem is to obtain the solution to the FBSDEs (3) and (4)-(6). We now show the solution to the FBSDEs in the following lemma.

Lemma 1. 

Supposing that ${\Omega }_k$ are positive definite for $k=0,\dots ,N$, the following equation

$$\begin{array}{c}\hfill {\zeta }_{k-1}={\mathcal{P}}_k{x}_k+\sum _{j=0}^{d-1}{\mathcal{P}}_k^j{u}_{j+k-d}+{\Phi }_k,\end{array}$$

(15)

is the solution to FBSDEs (3) and (4)-(6), with

$$\begin{array}{cc}\hfill {\Phi }_k=& C^{\prime }\left(k\right)({\mathcal{P}}_{k+1}{\overline{w}}_k-M_k^{\prime }{\Omega }_k^{-1}{\Sigma }_k+{\Phi }_{k+1})+{\overline{C}}^{\prime }\left(k\right){\mathcal{P}}_{k+1}\rho ,\hfill \\ \hfill {\Sigma }_k=& D_0^{\prime }\left(k\right)({\mathcal{P}}_{k+1}{\overline{w}}_k+{\Phi }_{k+1})+{\overline{D}}_0^{\prime }\left(k\right){\mathcal{P}}_{k+1}\rho +D_d^{\prime }\left(k\right)\hfill \\ \hfill & \times ({\mathcal{P}}_{k+d+1}{\overline{w}}_{k+d}+{\Phi }_{k+d+1})+{\overline{D}}_d^{\prime }\left(k\right){\mathcal{P}}_{k+d+1}\rho \hfill \end{array}$$

(16)

$$\begin{array}{cc}\hfill & +\sum _{j=0}^{d-1}{\left({\mathcal{P}}_{k+j+1}^{d-j-1}\right)}^{\prime }{\overline{w}}_{k+j}-\sum _{i=0}^d{\left(M_{k+i}^{d-i}\right)}^{\prime }{\Omega }_{k+i}^{-1}{\Sigma }_{k+i},\hfill \end{array}$$

(17)

where ${\Phi }_{k+1}=0$ and ${\Sigma }_{k+1}=0$ for $k\ge N$. Besides, ${\mathcal{P}}_k$, ${\mathcal{P}}_k^j$ satisfy the coupled equations (7), (12), (13).

Proof. 

The proof of Lemma 1 is put into Appendix A. □

Now we are ready to present the solution to Problem 1.

Theorem 1. 

There exists the unique ${\mathcal{F}}_{k-1}$-measurable $u_k$ for Problem 1 if and only if ${\Omega }_k$, for $k=0,\dots ,N$, are positive definite. In this case, the optimal controller $u_k$ is given by

$$\begin{array}{c}\hfill u_k=-{\Omega }_k^{-1}{M}_k{x}_k-{\Omega }_k^{-1}\sum _{j=0}^{d-1}{M}_k^j{u}_{j+k-d}-{\Omega }_k^{-1}{\Sigma }_k.\end{array}$$

(18)

The associated optimal performance index is as

$$\begin{array}{cc}\hfill J_N^{*}=& x_0^{\prime }{\mathcal{P}}_0{x}_0+2x_0^{\prime }\sum _{j=0}^{d-1}{\mathcal{P}}_0^j{u}_{j-d}+\sum _{j=0}^{d-1}{u}_{j-d}^{\prime }(D_d^{\prime }\left(j\right){\mathcal{P}}_{j+1}{D}_d\left(j\right)\hfill \\ \hfill & +{\varphi }^2{\overline{D}}_d^{\prime }\left(j\right){\mathcal{P}}_{j+1}{\overline{D}}_d\left(j\right))u_{j-d}+2\sum _{j=0}^{d-1}\sum _{i=0}^{d-1}{u}_{j-d}^{\prime }{D}_d^{\prime }\left(j\right)\hfill \\ \hfill & \times {\mathcal{P}}_{j+1}^{i-j-1}{u}_{i-d}-\sum _{j=0}^{d-1}\sum _{i=0}^{d-1}\sum _{m=0}^{d-1}{u}_{j-d}^{\prime }{\left(M_m^{j-m}\right)}^{\prime }{\Omega }_m^{-1}{M}_m^{i-m}\hfill \\ \hfill & \times u_{i-d}+2x_0^{\prime }{\Phi }_0+2\sum _{k=0}^N{\overline{w}}_k^{\prime }{\Phi }_{k+1}-\sum _{k=0}^N{\Sigma }_k^{\prime }{\Omega }_k^{-1}{\Sigma }_k\hfill \\ \hfill & +\sum _{k=0}^{N}Tr\left[{\mathcal{P}}_{k+1}{Q}_{w_k}\right],\hfill \end{array}$$

(19)

where ${\Omega }_k$, $M_k$, $M_k^j$, ${\mathcal{P}}_k$, ${\mathcal{P}}_k^j$, ${\Phi }_k$, ${\Sigma }_k$ satisfy the coupled equations (7)-(13),(16),(17) and ${\mathcal{P}}_k^j=0$, $M_k^j=0$ for $j<0$.

Proof. 

The proof of Theorem 1 is put into Appendix B. □

Remark 2. 

We make the coefficients of the system (3) and the cost function (2) to be time-invariant. When there is no time delay in system (3), i.e., $d=0$, we have that $D_d={\overline{D}}_d=0$. Considering the noise-uncorrelated case with ${\overline{w}}_k=0$, it is obviously obtained that the coupled equations (10) and (12) can be rewritten as

$$\begin{array}{c}\hfill M_k^0=0,{\mathcal{P}}_k^0=-M_k^{\prime }{\Omega }_k^{-1}{M}_k^0=0.\end{array}$$

Substituting $M_k^0$ and ${\mathcal{P}}_k^0$ into (11) and (13), it can be derived that $M_k^j=0,{\mathcal{P}}_k^j=0$ for $j=0,\dots ,d-1$. Then the difference equations (8) and (9) yield to

$$\begin{array}{cc}\hfill {\Omega }_k& =R+D_0^{\prime }{\mathcal{P}}_{k+1}{D}_0+{\varphi }^2{\overline{D}}_0^{\prime }{\mathcal{P}}_{k+1}{\overline{D}}_0,\hfill \\ \hfill M_k& =D_0^{\prime }{\mathcal{P}}_{k+1}C+{\varphi }^2{\overline{D}}_0^{\prime }{\mathcal{P}}_{k+1}\overline{C}.\hfill \end{array}$$

The optimal controller reduces to

$$\begin{array}{c}\hfill u_k=-{\Omega }_k^{-1}{M}_k{x}_k,\end{array}$$

which is exactly the result of Moore[2].

Remark 3. 

When the system (3) is a time-invariant system, (3) can be rewritten as

$$\begin{array}{c}\hfill x_{k+1}=C\left(k\right)x_k+D_0\left(k\right)u_k+D_d\left(k\right)u_{k-d}+w_k\end{array}$$

with $C\left(k\right)=C+{\nu }_k\overline{C}$, $D_0\left(k\right)=D_0+{\nu }_k\overline{D_0}$, $D_d\left(k\right)=D_d+{\nu }_k\overline{D_d}$. The performance index becomes

$$\begin{array}{c}\hfill J_N=E\left\{\sum _{k=0}^N{x_k}^{\prime }Q{x}_k+{u_k}^{\prime }R{u}_k+{x_{N+1}}^{\prime }{\mathcal{P}}_{N+1}{x}_{N+1}\right\}.\end{array}$$

By using the results of Theorem 1, the optimal time-invariant LQG controller yields that

$$\begin{array}{c}\hfill u_k=-{\Omega }_k^{-1}{M}_k{x}_k-{\Omega }_k^{-1}\sum _{j=0}^{d-1}{M}_k^j{u}_{j+k-d}-{\Omega }_k^{-1}{\Sigma }_k,\end{array}$$

and the minimal cost function is as (19) where the coefficient matrices in ${\Omega }_k$, $M_k$, $M_k^j$, $P_k$, $P_k^j$ are time-invariant.

In view of obtaining the special case of optimal LQG control for system (3), now we shall show the results for the general system with multiple delays and multiplicative noises.

Consider the following general discrete time-varying system

$$\begin{array}{cc}\hfill x_{k+1}& =C_k\left(k\right)x_k+\sum _{i=0}^d{D}_k^i\left(k\right)u_{k-i}+w_k,\hfill \end{array}$$

(20)

and the cost function is as (2).

Problem 2. 

Find the unique ${\mathcal{F}}_{k-1}$-measurable state feedback controller $u_k$, for $k=0,\dots ,N$, to minimize the cost function (2) subject to the system (20).

Combining the system (20) and the cost function (2), we apply the Pontryagin’s maximum principle to yield the following costate equations:

$$\begin{array}{cc}\hfill {\zeta }_N=& {\mathcal{P}}_{N+1}{x}_{N+1},\hfill \end{array}$$

(21)

$$\begin{array}{cc}\hfill {\zeta }_{k-1}=& E\left[C_k^{\prime }\left(k\right){\zeta }_k\right|{\mathcal{F}}_{k-1}]+Q_k{x}_k,\hfill \end{array}$$

(22)

$$\begin{array}{cc}\hfill 0=& E\left[\sum _{i=0}^d{\left(D_{k+i}^i(k+i)\right)}^{\prime }{\zeta }_{k+i}\right|{\mathcal{F}}_{k-1}]+R_k{u}_k,\hfill \end{array}$$

(23)

with $i=0,\dots ,d$ for $k=0,\dots ,N$, and ${\zeta }_k=0$ for $k>N$.

We introduce the following coupled Riccati equations subject to the system with multiple deleys:

$$\begin{array}{cc}\hfill {\Omega }_k=& R_k+\sum _{i=0}^d(D_i^{\prime }(k+i){\mathcal{P}}_{k+i+1}{D}_i(k+i)+{\varphi }^2{\overline{D}}_i^{\prime }(k+i)\hfill \\ \hfill & \times {\mathcal{P}}_{k+i+1}{\overline{D}}_i(k+i))+\sum _{i=0}^{d-1}{D}_i^{\prime }(k+i){\mathcal{P}}_{k+i+1}^{d-i-1}\hfill \\ \hfill & +\sum _{i=0}^{d-1}{\left({\mathcal{P}}_{k+i+1}^{d-i-1}\right)}^{\prime }{D}_i(k+i)-\sum _{i=1}^d{\left(M_{k+i}^{d-i}\right)}^{\prime }{\Omega }_{k+i}^{-1}{M}_{k+i}^{d-i},\hfill \end{array}$$

(24)

$$\begin{array}{cc}\hfill M_k^j=& \sum _{i=0}^j(D_i^{\prime }(k+i){\mathcal{P}}_{k+i+1}{D}_{i-j+d}(k+i)+{\varphi }^2{\overline{D}}_i^{\prime }(k+i)\hfill \\ \hfill & \times {\mathcal{P}}_{k+i+1}{\overline{D}}_{i-j+d}(k+i))+\sum _{i=0}^{j-1}{D}_i^{\prime }(k+i){\mathcal{P}}_{k+i+1}^{j-i-1}\hfill \\ \hfill & +\sum _{i=0}^j{\left({\mathcal{P}}_{k+i+1}^{d-i-1}\right)}^{\prime }{D}_{i-j+d}(k+i)-\sum _{i=1}^j{\left(M_{k+i}^{d-i}\right)}^{\prime }{\Omega }_{k+i}^{-1}{M}_{k+i}^{j-i},\hfill \end{array}$$

(25)

$$\begin{array}{cc}\hfill {\mathcal{P}}_k^j=& C^{\prime }\left(k\right){\mathcal{P}}_{k+1}{D}_{d-j}\left(k\right)+{\varphi }^2{\overline{C}}^{\prime }\left(k\right){\mathcal{P}}_{k+1}{\overline{D}}_{d-j}\left(k\right)\hfill \\ \hfill & +C^{\prime }\left(k\right){\mathcal{P}}_{k+1}^{j-1}-M_k^{\prime }{\Omega }_k^{-1}{M}_k^j,\hfill \end{array}$$

(26)

for $j=0,\dots ,d-1$, where the terminal value is as (14).

Now we give the main results for Problem 2 in the following theorem.

Theorem 2. 

There exists the unique ${\mathcal{F}}_{k-1}$-measurable $u_k$ for Problem 2 if and only if ${\Omega }_k$, for $k=0,\dots ,N$, are positive definite. In this case, the optimal controller $u_k$ is calculated by

$$\begin{array}{c}\hfill u_k=-{\Omega }_k^{-1}{M}_k{x}_k-{\Omega }_k^{-1}\sum _{j=0}^{d-1}{M}_k^j{u}_{j+k-d}-{\Omega }_k^{-1}{\Sigma }_k,\end{array}$$

(27)

where

$$\begin{array}{cc}\hfill {\Sigma }_k=& \sum _{i=0}^d[D_i^{\prime }(k+i)({\mathcal{P}}_{k+i+1}{\overline{w}}_{k+i}+{\Phi }_{k+i+1})+{\overline{D}}_i^{\prime }(k+i){\mathcal{P}}_{k+i+1}\hfill \\ \hfill & \times \rho ]+\sum _{j=0}^{d-1}{\left({\mathcal{P}}_{k+j+1}^{d-j-1}\right)}^{\prime }{\overline{w}}_{k+j}-\sum _{i=1}^d{\left(M_{k+i}^{d-i}\right)}^{\prime }{\Omega }_{k+i}^{-1}{\Sigma }_{k+i}\hfill \end{array}$$

and the optimal cost is as

$$\begin{array}{cc}\hfill J_N^{*}=& x_0^{\prime }{\mathcal{P}}_0{x}_0+2x_0^{\prime }\sum _{j=0}^{d-1}{\mathcal{P}}_0^j{u}_{j-d}+\sum _{j=0}^{d-1}\sum _{i=0}^{d-1}\sum _{m=0}^{d-1}{u}_{j-d}^{\prime }[D_{m+d-j}^{\prime }\left(m\right)\hfill \\ \hfill & \times {\mathcal{P}}_{m+1}{D}_{m+d-i}\left(m\right)+{\varphi }^2{\overline{D}}_{m+d-j}^{\prime }\left(m\right){\mathcal{P}}_{m+1}{\overline{D}}_{m+d-i}\left(m\right)\hfill \\ \hfill & +D_{m+d-j}^{\prime }\left(m\right){\mathcal{P}}_{m+1}^{i-m-1}+{\left({\mathcal{P}}_{m+1}^{i-m-1}\right)}^{\prime }{D}_{m+d-i}\left(m\right)-{\left(M_m^{j-m}\right)}^{\prime }\hfill \\ \hfill & \times {\Omega }_m^{-1}{M}_m^{i-m}]u_{i-d}+\sum _{k=0}^{d}Tr\left[{\mathcal{P}}_{k+1}{Q}_{w_k}\right]2x_0^{\prime }{\Phi }_0\hfill \\ \hfill & +2\sum _{k=0}^N{\overline{w}}_k^{\prime }{\Phi }_{k+1}-\sum _{k=0}^N{\Sigma }_k^{\prime }{\Omega }_k^{-1}{\Sigma }_k\hfill \end{array}$$

(28)

where $D_i=0$ for $i>d$.

In addition, the relationship of the optimal costate ${\zeta }_{k-1}$ and state $x_k$ is as (16) in Lemma 1.

Proof. 

The proof is similar to that of Theorem 1, and to save the space of the paper, we omit it here. □

3. Output Feedback Controller

When the state variable $x_k$ are partially observed, we study the following discrete-time stochastic system:

$$\begin{array}{cc}\hfill x_{k+1}& =[C\left(k\right)+{\nu }_k\overline{C}\left(k\right)]x_k+[D_0\left(k\right)+{\nu }_k{\overline{D}}_0\left(k\right)]u_k+[D_d\left(k\right)\hfill \\ \hfill & +{\nu }_k{\overline{D}}_d\left(k\right)]u_{k-d}+w_k\hfill \\ \hfill z_k& =[H\left(k\right)+g_k\overline{H}\left(k\right)]x_k+e_k\hfill \end{array}$$

(29)

where $z_k\in R^q$ is the measurement, $g_k$ is the scalar white noise with zero mean and variance $Q_{g_k}$, $w_k$ and $e_k$ are Gaussian zero-mean white noises with covariance $Q_{w_k}$ and $Q_{e_k}$. $H\left(k\right)$ and $\overline{H}\left(k\right)$ are deterministic matrices with compatible dimensions. In this case, the initial value $x_0$ is known, ${\nu }_k$, $w_k$, $g_k$ and $e_k$ are independent of each other.

Obviously, there exist multiplicative noises ${\nu }_k$ and $g_k$ in system (29). As we can not obtain the exact information of the state by (29), we introduce the state estimation to design the controller instead. We first obtain the linear optimal state estimator for by applying standard filtering results in [3]. Then, we will derive the suboptimal linear state estimate feedback controller through the following linearizations.

The aim of this section is to find the suboptimal linear state estimate feedback controller for system (29) in order to minimize the cost function (2).

First, we introduce the linear optimal state estimator in Lemma 2.

Lemma 2. 

Based on the system (29) with input delays and multiplicative noises, the linear optimal estimator is given by

$$\begin{array}{cc}\hfill {\widehat{x}}_{k+1|k}=& E\left[x_{k+1}\right|z_0,\dots ,z_k]\hfill \\ \hfill =& C\left(k\right){\widehat{x}}_{k|k-1}+D_0\left(k\right)u_k+D_d\left(k\right)u_{k-d}+K_k{\tilde{z}}_k,\hfill \end{array}$$

(30)

where

$$\begin{array}{cc}\hfill {\tilde{z}}_k=& z_k-C\left(k\right){\widehat{x}}_{k|k-1},\hfill \\ \hfill K_k=& C\left(k\right){\Sigma }_{k|k-1}{H}^{\prime }\left(k\right)(H\left(k\right){\Sigma }_{k|k-1}{H}^{\prime }\left(k\right)+Q_{g_k}\overline{H}\left(k\right)\hfill \\ \hfill & \times ({\widehat{x}}_{k|k-1}{\widehat{x}}_{k|k-1}^{\prime }+{\Sigma }_{k|k-1}){\overline{H}}^{\prime }\left(k\right)+Q_{e_k}{)}^{-1}\hfill \end{array}$$

Besides, the estimator error covariance matrix is

$$\begin{array}{cc}\hfill {\Sigma }_{k+1|k}=& E\left[(x_{k+1}-{\widehat{x}}_{k+1|k}){(x_{k+1}-{\widehat{x}}_{k+1|k})}^{\prime }\right|z_0,\dots ,z_k]\hfill \\ \hfill =& C\left(k\right){\Sigma }_{k|k-1}{C}^{\prime }\left(k\right)-K_k\left(H\left(k\right){\Sigma }_{k|k-1}^{\prime }{C}^{\prime }\left(k\right)\right)+{\varphi }^2\hfill \\ \hfill & \times [\overline{C}\left(k\right)({\Sigma }_{k|k-1}+{\widehat{x}}_{k|k-1}{\widehat{x}}_{k|k-1}^{\prime }){\overline{C}}^{\prime }\left(k\right)+{\overline{D}}_0\left(k\right)u_k\hfill \\ \hfill & \times u_k^{\prime }{\overline{D}}_0^{\prime }\left(k\right)+{\overline{D}}_d\left(k\right)u_{k-d}{u}_{k-d}^{\prime }{\overline{D}}_d^{\prime }\left(k\right)]+Q_{w_k}.\hfill \end{array}$$

The initial values ${\widehat{x}}_{0|-1}={\overline{x}}_0$ and ${\Sigma }_{0|-1}={\overline{\mathcal{P}}}_0^e$.

Proof. 

The proof of Lemma 2 is put into Appendix C. □

Now, the state estimation is obtained, and we can consider (30) as the state instead of the unavailable exact state information. Observing (3) and (30), we know that the filter gain $K_k$ on ${\widehat{x}}_{k|k-1}$ should be affine, so that we can apply the results of Theorem 1 in this section. Then, we will linearize the filter gain $K_k$.

Applying first order of Taylor expansion on $K_k$ through the fixed point ${\widehat{x}}_{k|k-1}={\overline{x}}_0$, the linearization of $K_k$ yields

$$\begin{array}{c}\hfill K_k=K_k^0+K_k^1({\widehat{x}}_{k|k-1}-{\overline{x}}_0)+o\left(∥{\widehat{x}}_{k|k-1}∥\right).\end{array}$$

(31)

Ignoring the quadratic and higher order terms in (31), and plug (30) into it, the approximation of ${\widehat{x}}_{k+1|k}$ becomes

$$\begin{array}{cc}\hfill {\widehat{x}}_{k+1|k}\approx & C\left(k\right){\widehat{x}}_{k|k-1}+D_0\left(k\right)u_k+D_d\left(k\right)u_{k-d}+(K_k^0\hfill \\ \hfill & +K_k^1({\widehat{x}}_{k|k-1}-{\overline{x}}_0)){\tilde{z}}_k\hfill \\ \hfill =& (C\left(k\right)+K_k^1{\tilde{z}}_k){\widehat{x}}_{k|k-1}+D_0\left(k\right)u_k+D_d\left(k\right)u_{k-d}\hfill \\ \hfill & +(K_k^0-K_k^1{\overline{x}}_0){\tilde{z}}_k.\hfill \end{array}$$

(32)

With (30)-(32), the cost function (2) can be reorganized as

$$\begin{array}{cc}\hfill J_N^e\approx & E[\sum _{k=0}^N\left({\widehat{x}}_{k|k-1}^{\prime }{Q}_k{\widehat{x}}_{k|k-1}+u_k^{\prime }{R}_k{u}_k+Tr\left[Q_k{\Sigma }_{k|k-1}\right]\right)\hfill \\ \hfill & +{\widehat{x}}_{N+1|N}^{\prime }{\mathcal{P}}_{N+1}^e{\widehat{x}}_{N+1|N}+Tr\left[{\mathcal{P}}_{N+1}^e{\Sigma }_{N+1|N}\right]].\hfill \end{array}$$

(33)

In this case, the coupled Riccati equations can be derived as

$$\begin{array}{c}\hfill {\mathcal{P}}_k^e=C^{\prime }\left(k\right){\mathcal{P}}_{k+1}^{e}C\left(k\right)+{\eta }_k{\left(K_k^1\right)}^{\prime }{\mathcal{P}}_{k+1}^e\left(K_k^1\right)-{\left(M_k^e\right)}^{\prime }{\left({\Omega }_k^e\right)}^{-1}{M}_k^e+Q_k,\end{array}$$

where

$$\begin{array}{cc}\hfill {\Omega }_k^e& =R_k+D_0^{\prime }\left(k\right){\mathcal{P}}_{k+1}^e{D}_0\left(k\right)+D_d^{\prime }(k+d){\mathcal{P}}_{k+d+1}^e{D}_d(k+d)\hfill \\ \hfill & +D_0^{\prime }\left(k\right){\mathcal{P}}_{k+1}^{d-1}+{\left({\mathcal{P}}_{k+1}^{d-1}\right)}^{\prime }{D}_0\left(k\right)-\sum _{i=1}^d{\left(M_{k+i}^{d-i}\right)}^{\prime }{\Omega }_{k+i}^{-1}{M}_{k+i}^{d-i},\hfill \\ \hfill M_k^e& =D_0^{\prime }\left(k\right){\mathcal{P}}_{k+1}^{e}C\left(k\right)+{\left({\mathcal{P}}_{k+1}^{d-1}\right)}^{\prime }C\left(k\right),\hfill \end{array}$$

with

$$\begin{array}{cc}\hfill {\left(M_k^0\right)}^e& =D_0^{\prime }\left(k\right){\mathcal{P}}_{k+1}^e{D}_d\left(k\right)+{\left({\left({\mathcal{P}}_{k+1}^{d-1}\right)}^e\right)}^{\prime }{D}_d\left(k\right),\hfill \\ \hfill {\left(M_k^j\right)}^e& =D_0^{\prime }\left(k\right){\left({\mathcal{P}}_{k+1}^{j-1}\right)}^e+{\left({\left({\mathcal{P}}_{k+j+1}^{d-j-1}\right)}^e\right)}^{\prime }{D}_d(k+j)\hfill \\ \hfill & -\sum _{i=1}^j{\left({\left(M_{k+i}^{d-i}\right)}^e\right)}^{\prime }{\left({\Omega }_{k+i}^e\right)}^{-1}{\left(M_{k+i}^{j-i}\right)}^e,\hfill \\ \hfill {\left({\mathcal{P}}_k^0\right)}^e& =C^{\prime }\left(k\right){\mathcal{P}}_{k+1}^e{D}_d\left(k\right)-{\left(M_k^e\right)}^{\prime }{\left({\Omega }_k^e\right)}^{-1}{\left(M_k^0\right)}^e,\hfill \\ \hfill {\left({\mathcal{P}}_k^j\right)}^e& =C^{\prime }\left(k\right){\left({\mathcal{P}}_{k+1}^{j-1}\right)}^e-{\left(M_k^e\right)}^{\prime }{\left({\Omega }_k^e\right)}^{-1}{\left(M_k^j\right)}^e,\hfill \end{array}$$

$j=1,\dots ,d-1$, with the terminal values

$$\begin{array}{cc}\hfill & {\mathcal{P}}_{N+1}^e,{\mathcal{P}}_{N+i+1}^e=0,{\left({\mathcal{P}}_{N+i}^j\right)}^e=0,\hfill \\ \hfill & {\left(M_{N+i}^j\right)}^e=0,{\Omega }_{N+i}^e=I,i\ge 1,j=0,\dots ,d-1,\hfill \end{array}$$

and

$$\begin{array}{cc}\hfill {\eta }_k=& E\left[{\tilde{z}}_k{\tilde{z}}_k^{\prime }\right|z_0,\dots ,z_{k-1}]\hfill \\ \hfill =& H\left(k\right){\Sigma }_{k|k-1}{H}^{\prime }\left(k\right)+Q_{g_k}{\overline{H}}^{\prime }\left(k\right)E\left[x_k{x}_k^{\prime }\right|z_0,\dots ,z_{k-1}]\hfill \\ \hfill & \times {\overline{H}}^{\prime }\left(k\right)+Q_{e_k},\hfill \end{array}$$

where

$$\begin{array}{cc}\hfill & E\left[x_{k+1}{x}_{k+1}^{\prime }\right|z_0,\dots ,z_k]\hfill \\ \hfill =& C\left(k\right)E\left[x_k{x}_k^{\prime }\right|z_0,\dots ,z_{k-1}]C^{\prime }\left(k\right)+{\varphi }^2{\overline{C}}^{\prime }\left(k\right)E\left[x_k{x}_k^{\prime }\right|z_0,\dots ,z_{k-1}]\hfill \\ \hfill & \times \overline{C}\left(k\right)+D_0\left(k\right)u_k{u}_k^{\prime }{D}_0^{\prime }\left(k\right)+{\overline{D}}_0\left(k\right)u_k{u}_k^{\prime }{\overline{D}}_0^{\prime }\left(k\right)+D_d\left(k\right)u_{k-d}\hfill \\ \hfill & \times u_{k-d}^{\prime }{D}_d^{\prime }\left(k\right)+{\overline{D}}_d\left(k\right)u_{k-d}{u}_{k-d}^{\prime }{\overline{D}}_d^{\prime }\left(k\right)Q_{w_k}+C\left(k\right){\widehat{x}}_{k|k-1}{u}_k^{\prime }{D}_0^{\prime }\left(k\right)\hfill \\ \hfill & +{\varphi }^2\overline{C}\left(k\right){\widehat{x}}_{k|k-1}{u}_k^{\prime }{\overline{D}}_0^{\prime }\left(k\right)+C\left(k\right){\widehat{x}}_{k|k-1}{u}_{k-d}^{\prime }{D}_d^{\prime }\left(k\right)+{\varphi }^2\overline{C}\left(k\right)\hfill \\ \hfill & \times {\widehat{x}}_{k|k-1}{u}_{k-d}^{\prime }{\overline{D}}_d^{\prime }\left(k\right)+D_0\left(k\right)u_k{\widehat{x}}_{k|k-1}^{\prime }{C}^{\prime }\left(k\right)+{\varphi }^2{\overline{D}}_0\left(k\right)u_k{\widehat{x}}_{k|k-1}^{\prime }\hfill \\ \hfill & \times {\overline{C}}^{\prime }\left(k\right)+D_0\left(k\right)u_k{u}_{k-d}^{\prime }{D}_d^{\prime }\left(k\right)+{\varphi }^2{\overline{D}}_0\left(k\right)u_k{u}_{k-d}^{\prime }{\overline{D}}_d^{\prime }\left(k\right)+D_d\left(k\right)\hfill \\ \hfill & \times u_{k-d}{\widehat{x}}_{k|k-1}^{\prime }{C}^{\prime }\left(k\right)+{\varphi }^2{\overline{D}}_d\left(k\right)u_{k-d}{\widehat{x}}_{k|k-1}^{\prime }{\overline{C}}^{\prime }\left(k\right)+D_d\left(k\right)u_{k-d}\hfill \\ \hfill & \times u_k^{\prime }{D}_0^{\prime }\left(k\right)+{\varphi }^2{\overline{D}}_d\left(k\right)u_{k-d}{u}_k^{\prime }{\overline{D}}_0^{\prime }\left(k\right).\hfill \end{array}$$

with the initial value $E\left[x_0{x}_0^{\prime }\right]={\overline{x}}_0{\overline{x}}_0^{\prime }+{\overline{\mathcal{P}}}_0^e$.

Now, we can find the suboptimal controller to minimize the cost function (33) subject to (29), by the results of Theorem 1.

Theorem 3. 

The suboptimal linear state estimate feedback controller for system (29) that minimizes the cost function (33) is given by

$$\begin{array}{c}\hfill u_k^e=-{\left({\Omega }_k^e\right)}^{-1}{M}_k^e{\widehat{x}}_{k|k-1}-{\left({\Omega }_k^e\right)}^{-1}\sum _{j=0}^{d-1}{\left(M_k^j\right)}^e{u}_{j+k-d}^e.\end{array}$$

(34)

The minimized cost function is given by

$$\begin{array}{cc}\hfill {\left(J_N^e\right)}^{*}=& {\overline{x}}_0^{\prime }{\mathcal{P}}_0^e{\overline{x}}_0+2{\overline{x}}_0^{\prime }\sum _{j=0}^{d-1}{\left({\mathcal{P}}_0^j\right)}^e{u}_{j-d}+\sum _{j=0}^{d-1}{u}_{j-d}^{\prime }{D}_d^{\prime }\left(j\right){\mathcal{P}}_{j+1}^e\hfill \\ \hfill & \times D_d\left(j\right)u_{j-d}+2\sum _{j=0}^{d-1}\sum _{i=0}^{d-1}{u}_{j-d}^{\prime }{D}_d^{\prime }\left(j\right){\left({\mathcal{P}}_{j+1}^{i-j-1}\right)}^e{u}_{i-d}^e\hfill \\ \hfill & -\sum _{j=0}^{d-1}\sum _{i=0}^{d-1}\sum _{m=0}^{d-1}{\left(u_{j-d}^e\right)}^{\prime }{\left({\left(M_m^{j-m}\right)}^e\right)}^{\prime }{\left({\Omega }_m^e\right)}^{-1}{\left(M_m^{i-m}\right)}^e\hfill \\ \hfill & \times u_{i-d}^e+\sum _{k=0}^{N}Tr\left[{\mathcal{P}}_{k+1}^e{Q}_{w_k}\right],\hfill \end{array}$$

(35)

4. Numerical examples

Example 1 Consider the scalar case of time-invariant LQG control system (3) in Remark 3. We consider the case that the additive noise $w_k$ is the zero-mean white noise. The associate parameters are as

$$\begin{array}{cc}\hfill & C=2,\overline{C}=1,D_0=3,{\overline{D}}_0=1,D_d=2,{\overline{D}}_d=2,\hfill \\ \hfill & d=5,\varphi =1,Q_w=1,\rho =0.6,\overline{w}=0.2,\hfill \end{array}$$

with the initial values

$$\begin{array}{cc}\hfill & x_0=1,u_{-5}=-0.5,u_{-4}=0.8,u_{-3}=-1.2,\hfill \\ \hfill & u_{-2}=-1,u_{-1}=-0.6,\hfill \end{array}$$

and the cost function (2) with $Q=1,R=1,{\mathcal{P}}_{N+1}=1$. When the delay $d=5$, and $N=30$, by applying Theorem 1 and the equations (7)-(13), direct calculations yield that ${\mathcal{P}}_k,{\Omega }_k,M_k,{\mathcal{P}}_k^j,M_k^j$ for $k=0,\dots ,N$. It can be obviously known that ${\Omega }_k$ is positive definite for $k=0,\dots ,N$. Thus, there exists a unique $u_k$ from Theorem 1, and the optimal controller can be calculated with (18), which is shown in Figure 1.

Accordingly, the associated optimal value of (17) is $J_N^{*}$ =107.5150.

In order to illustrate that the proposed LQG controller can minimize performance index, let us consider the time-invariant standard state feedback controller $u_k=-{\Omega }_k^{-1}{M}_k{x}_k$. Based on the above parameters and by substituting into cost function, the controller $u_k$ are shown in Figure 2, and the associated value is $J_N^{*}$ =255.0603, which confirmed the effectiveness of the algorithm.

Example 2 Consider the discrete time-varying LQG control system with multiple delays and multiplicative noises with $x_k\in R^2$, $u_k\in R^2$, and the cost function (2). The associate coefficients are:

$$\begin{array}{cc}\hfill & C\left(1\right)=\left[\begin{array}{cc}-0.2& -0.1\\ -0.8& 1.1\end{array}\right],C\left(2\right)=\left[\begin{array}{cc}-1.9& 1.1\\ -1.4& -1.9\end{array}\right],\hfill \\ \hfill & C\left(3\right)=\left[\begin{array}{cc}-1.4& -0.3\\ 0& 0.4\end{array}\right],C\left(4\right)=\left[\begin{array}{cc}-1.6& 0.6\\ 1.9& -0.8\end{array}\right],\hfill \\ \hfill & \overline{C}\left(1\right)=\left[\begin{array}{cc}-1.3& 0.8\\ -1.9& 0.4\end{array}\right],\overline{C}\left(2\right)=\left[\begin{array}{cc}-1.4& 1.6\\ -0.7& 0.4\end{array}\right],\hfill \\ \hfill & \overline{C}\left(3\right)=\left[\begin{array}{cc}-0.9& 0.3\\ 0.3& -1.5\end{array}\right],\overline{C}\left(4\right)=\left[\begin{array}{cc}1.4& 1\\ -1.8& 1.5\end{array}\right],\hfill \\ \hfill & D_0\left(1\right)=\left[\begin{array}{cc}-0.3& 0\\ 1.6& 1.6\end{array}\right],D_0\left(2\right)=\left[\begin{array}{cc}-1.9& -1.2\\ -0.7& -0.9\end{array}\right],\hfill \\ \hfill & D_0\left(3\right)=\left[\begin{array}{cc}-1.7& -1.3\\ -1.8& 0.1\end{array}\right],D_0\left(4\right)=\left[\begin{array}{cc}0.5& -0.2\\ 0.8& 1\end{array}\right],\hfill \\ \hfill & {\overline{D}}_0\left(1\right)=\left[\begin{array}{cc}0& 0.8\\ 1.7& 0.7\end{array}\right],{\overline{D}}_0\left(2\right)=\left[\begin{array}{cc}-1.9& 1\\ 1.3& 0.8\end{array}\right],\hfill \\ \hfill & {\overline{D}}_0\left(3\right)=\left[\begin{array}{cc}1.3& -1.2\\ -0.8& -1.7\end{array}\right],{\overline{D}}_0\left(4\right)=\left[\begin{array}{cc}-1.9& -1.6\\ -1.2& -1.3\end{array}\right],\hfill \\ \hfill & D_d\left(1\right)=\left[\begin{array}{cc}-1.9& -0.3\\ 0.1& 0.5\end{array}\right],D_d\left(2\right)=\left[\begin{array}{cc}1.3& -1.5\\ 1& -0.5\end{array}\right],\hfill \\ \hfill & D_d\left(3\right)=\left[\begin{array}{cc}1.7& 0.5\\ -1& -1.1\end{array}\right],D_d\left(4\right)=\left[\begin{array}{cc}1.7& -1.5\\ 1.4& 0.6\end{array}\right],\hfill \\ \hfill & D_d\left(5\right)=\left[\begin{array}{cc}1.8& -1.9\\ 0& -0.3\end{array}\right],D_d\left(6\right)=\left[\begin{array}{cc}-0.3& -0.2\\ 0.4& -1.9\end{array}\right],\hfill \\ \hfill & D_d\left(7\right)=\left[\begin{array}{cc}-0.9& -1\\ 0.3& 1.8\end{array}\right],{\overline{D}}_d\left(1\right)=\left[\begin{array}{cc}0.2& 0.2\\ -1.1& 1.4\end{array}\right],\hfill \\ \hfill & {\overline{D}}_d\left(2\right)=\left[\begin{array}{cc}-1.9& -2\\ -2& -1.3\end{array}\right],{\overline{D}}_d\left(3\right)=\left[\begin{array}{cc}-1& -1.4\\ 1& 1.3\end{array}\right],\hfill \\ \hfill & {\overline{D}}_d\left(4\right)=\left[\begin{array}{cc}-1.8& -1.2\\ -1.4& 0.6\end{array}\right],{\overline{D}}_d\left(5\right)=\left[\begin{array}{cc}-1.5& -0.9\\ 1.8& 1.5\end{array}\right],\hfill \\ \hfill & {\overline{D}}_d\left(6\right)=\left[\begin{array}{cc}-1.8& 1.9\\ -0.4& 0.1\end{array}\right],{\overline{D}}_d\left(7\right)=\left[\begin{array}{cc}-1.7& 0.5\\ -1.2& -0.8\end{array}\right],\hfill \\ \hfill & {\mathcal{P}}_{N+1}=\left[\begin{array}{cc}0& 0\\ 0& 0\end{array}\right],Q_1=\left[\begin{array}{cc}1& 0\\ 0& 1\end{array}\right],Q_2=\left[\begin{array}{cc}0& 1\\ 1& 0\end{array}\right],\hfill \\ \hfill & Q_3=\left[\begin{array}{cc}0.8& 1\\ 1& 0.8\end{array}\right],Q_4=\left[\begin{array}{cc}1& 0\\ 0& 1\end{array}\right],R_1=\left[\begin{array}{cc}2& 0\\ 0& 2\end{array}\right],\hfill \\ \hfill & R_2=\left[\begin{array}{cc}0& 1\\ 1& 0\end{array}\right],R_3=\left[\begin{array}{cc}1& 0\\ 0& 1\end{array}\right],R_4=\left[\begin{array}{cc}1& 0\\ 0& 1\end{array}\right],\hfill \end{array}$$

When the delay time is $d=3$ and the final time is $N=4$, given the initial value

$$\begin{array}{cc}\hfill & x_0=\left[\begin{array}{c}1\\ 1\end{array}\right],u_{-3}=\left[\begin{array}{c}1.7\\ -1.9\end{array}\right],u_{-2}=\left[\begin{array}{c}1\\ 1\end{array}\right],u_{-1}=\left[\begin{array}{c}0\\ 1.6\end{array}\right],\hfill \\ \hfill & C\left(0\right)=\left[\begin{array}{cc}2& 0\\ 1& 2\end{array}\right],\overline{C}\left(0\right)=\left[\begin{array}{cc}1& 1\\ 2& 1\end{array}\right],D_0\left(0\right)=\left[\begin{array}{cc}1& 2\\ 1& 0\end{array}\right],\hfill \\ \hfill & {\overline{D}}_0\left(0\right)=\left[\begin{array}{cc}0& 1\\ 2& 1\end{array}\right],D_d\left(0\right)=\left[\begin{array}{cc}2& 1\\ 1& 2\end{array}\right],{\overline{D}}_d\left(0\right)=\left[\begin{array}{cc}1& 2\\ 1& 0\end{array}\right],\hfill \end{array}$$

by applying Theorem 1 and (8)-(13), it yields that

$$\begin{array}{cc}\hfill & {\mathcal{P}}_1=\left[\begin{array}{cc}25.33& -2.03\\ -2.03& 2.60\end{array}\right],{\mathcal{P}}_2=\left[\begin{array}{cc}9.10& 1.45\\ 1.45& 12.47\end{array}\right],\hfill \\ \hfill & {\mathcal{P}}_3=\left[\begin{array}{cc}2.83& -0.76\\ -0.76& 2.90\end{array}\right],{\mathcal{P}}_4=\left[\begin{array}{cc}1& 0\\ 0& 1\end{array}\right],\hfill \\ \hfill & {\Omega }_1=\left[\begin{array}{cc}69.41& 48.05\\ 48.05& 47.49\end{array}\right],{\Omega }_2=\left[\begin{array}{cc}28.53& 5.16\\ 5.16& 8.26\end{array}\right],\hfill \\ \hfill & {\Omega }_3=\left[\begin{array}{cc}9.46& 1.83\\ 1.83& 7.03\end{array}\right],{\Omega }_4=\left[\begin{array}{cc}1& 0\\ 0& 1\end{array}\right],\hfill \\ \hfill & M_1=\left[\begin{array}{cc}-59.03& 31.96\\ -46.01& 32.31\end{array}\right],M_2=\left[\begin{array}{cc}15.30& -12.32\\ 3.33& 4.43\end{array}\right],\hfill \\ \hfill & M_3=\left[\begin{array}{cc}0.97& 0.36\\ 2.39& 1.84\end{array}\right],M_4=\left[\begin{array}{cc}0& 0\\ 0& 0\end{array}\right].\hfill \end{array}$$

For $i=1,2,3,4$, ${\Omega }_i>0$, thus, there is an optimal solution to the LQG system with multiple delays and state/control noises from Theorem 1. Based on the above data, the optimal controller can be calculated as

$$\begin{array}{cc}\hfill & u_0=\left[\begin{array}{c}0.59\\ -1.35\end{array}\right],u_1=\left[\begin{array}{c}-0.96\\ -0.46\end{array}\right],u_2=\left[\begin{array}{c}1.38\\ -2.34\end{array}\right],\hfill \\ \hfill & u_3=\left[\begin{array}{c}-0.10\\ 1.34\end{array}\right],u_4=\left[\begin{array}{c}0\\ 0\end{array}\right].\hfill \end{array}$$

According to (17), the optimal performance index of system (3) is $J_N^{*}=51.1915$.

5. Conclusions

In this paper, the discrete time-varying optimal linear quadratic Gaussian (LQG) control problem involving multiple delays and state/control- dependent noises has been studied. A necessary and sufficient condition for the existence of unique optimal controller to the problem is given, which is based on the obtained maximum principle and the relationship between the state and costate. Under this context, the optimal controller and the minimized performance index are represented. What’s more, as the state variables observed partially, the suboptimal linear state estimate feedback controllers for the LQG models with input delays and multiplicative noises are derived.