Preprint
Article

Optimal Control of Discrete Time-varying System with Multiple Delays and Multiplicative Noises

Altmetrics

Downloads

120

Views

34

Comments

0

This version is not peer-reviewed

Submitted:

16 May 2023

Posted:

18 May 2023

You are already at the latest version

Alerts
Abstract
This paper is concerned with the optimal linear quadratic Gaussian (LQG) control problem for discrete time-varying system with multiple input delays and multiplicative noises. The main contributions are two-fold. Firstly, when the state variables can be observed exactly, we obtain a necessary and suffcient condition for the multiple-delays system in terms of the non-homogeneous relationship between the state and costate, which is the solution to the coupled forward and backward stochastic difference equations. Secondly, when the state variables are partially observed, we derive a suboptimal linear output feedback controller for the discrete-time system based on the obtained results of the optimal LQG control. Numerical examples are shown to illustrate the proposed algorithm.
Keywords: 
Subject: Computer Science and Mathematics  -   Applied Mathematics

1. Introduction

Linear quadratic Gaussian (LQG) control problem stems from the optimal stochastic control theory of the systems with additive Gaussian white noises and state/control-dependent, which combines the concept of linear quadratic regulators for full state feedback and Kalman filters for state estimation [1,2,3]. Recently, the optimal LQG control has been applied in various fields, such as the robots of power substation, all-electric vehicles, electrical safety engineering networked control systems (NCSs) [5,6,7,9]. Specificly, for mobile monitoring robot in a ultrahigh-voltage power substation, the LQG conotrol is proposed to minimize the difference between the actual SNIR and its expectation and the change in transmitting power[4]. These motivate us to study the more complicated LQG control systems with multiple input delays and multiplicative noises.
It is generally known that random time delay and packet dropout always occur in the data transmission of NCSs. Many literatures have been investigated on LQG control problems with input delays and packet loss [8,10,11,12,13,14]. Basin [8] presented an optimal linear regulator (LQR) with input delay by using the duality principle. Cacace[10] studied the LQG problems for linear system with single input delay. Matni[11] presented an explicit solution to a two-player distributed LQG problem in which communication between controllers occurs across a communication link with varying delay. Basin[12] further established a necessary and sufficient condition of the optimal LQR control for the linear system with multiple input delays. Zhang[14] studied the classical LQR problem with multiple input delays for both continuous-time and discrete-time cases.
On the other hand, packet dropout is generally described as the multiplicative noises. Many references have focused on the LQG system with multiplicative noises [15,16,17]. Gupta[15] solved the optimal LQG problem with packet-dropping links by decomposing the problem into a standard LQR state-feedback controller designing. Liang[16] studied the optimal control and stabilization problems for NCSs with remote controller and local controller subject to packet dropout. For systems with both input delay and packet dropout, Liang[18] presented the optimal LQR controller, and derived the necessary and sufficient condition for the mean-square stabilization. Liang[19] considered the discrete-time LQG system with input delay and multiplicative noises, and obtained both optimal state feedback controller and suboptimal output feedback controller.
The aforementioned literatures are mainly focused on single delay and packet dropout. To our best knowledge, little progress has been made on the optimal LQG control for time-varying systems with multiple input delays and multiplicative noises.
Motivated by the work of [18,19,20], this paper studies the optimal LQG control for discrete time-varying system involving with multiple delays and multiplicative noises. The main contributions of this paper are summarized as follows: 1) When the state variables can be observed exactly, by introducing the stochastic maximum principle for system with multiple delays and multiplicative noises, a solution to the forward backward differential equations (FBSDEs) is obtained based on the coupled Riccati equations. 2) In terms of the solution to the FBSDEs, a necessary and sufficient condition is given for the optimal LQG control. 3) When the state variables are partially observed, we derive a suboptimal linear output feedback controller by linearizing the optimal estimator and neglecting higher order terms.
The rest of the paper is organized as follows. In Section 2, we give the results of optimal state feedback control problem. In Section 3, we derive a suboptimal linear output feedback controller for the LQG systems involving multiple input delays and multiplicative noises. Numerical examples are provided in Section 4. Conclusions are given in Section 5.
Notation: R n denotes the n-dimensional real Euclidean space. I presents the unit matrix of appropriate dimension. The superscript denotes the transpose of the matrix. { Ω , F , P , { F k } k 0 } denotes a complete probability space on which random variable w k are defined such that { F k } k 0 is the natural filtration generated by w k and ν k , i.e., F k = σ { w 0 , , w k , ν 0 , , ν k } , augmented by all the P null sets in F . A symmetric A > 0 ( 0 ) means that it is a positive definite (positive semi-definite) matrix. T r ( A ) represents the trace of matrix A.

2. State Feedback Controller

When the state variable x k can be observed exactly, we consider the following discrete time-varying LQG system with multiple input delays and multiplicative noises
x k + 1 = C ( k ) + ν k C ¯ ( k ) x k + D 0 ( k ) + ν k D ¯ 0 ( k ) u k + D d ( k ) + ν k D ¯ d ( k ) u k d + w k ,
where x k R n is the state, u k R m is the input control with the delay d > 0 , ν k is the scalar white noise with zero mean and variance ϕ 2 , w k R n is the random variables satisfying E [ w k | F k 1 ] = w ¯ k and E [ w k w k ] = Q w k . C ( k ) , C ¯ ( k ) , D i ( k ) and D ¯ i ( k ) with i = 0 , d are coefficient matrices with compatible dimensions. ν k and w k are correlated with E [ v k w k | F k 1 ] = ρ , E [ v k w l ] = 0 , k l . The initial state x 0 , u i for i = d , , 1 are known.
The associated cost function for system (1) is given by
J N = E k = 0 N x k Q k x k + u k R k u k + x N + 1 P N + 1 x N + 1 ,
where Q k and P N + 1 are positive semi-definite constant matrices with appropriate dimensions, control cost matrix R k should be positive definite matrix, and N is the horizon length.
Problem 1. 
Find the unique F k 1 -measurable state feedback controller u k , for k = 0 , , N , to minimize (2) subject to (1).
For simplicity, we make the following definitions
C k ( k ) = C ( k ) + ν k C ¯ ( k ) , D k i ( k ) = D i ( k ) + ν k D ¯ i ( k ) ,
for i = 0 , d . Then the system (1) becomes
x k + 1 = C k ( k ) x k + D k 0 ( k ) u k + D k d ( k ) u k d + w k .
Following the similar discussion of [19], in virtue of the Pontryagin’s maximum principle for (3) and (2), we have
ζ N = P N + 1 x N + 1 ,
ζ k 1 = E [ C k ( k ) ζ k | F k 1 ] + Q k x k ,
0 = E [ ( D k 0 ( k ) ) ζ k + ( D k + d d ( k + d ) ) ζ k + d | F k 1 ] + R k u k ,
for k = 0 , , N , where ζ k is the costate with ζ k = 0 for k > N .
For further study, the following coupled Riccati difference equations are given:
P k = C ( k ) P k + 1 C ( k ) + ϕ 2 C ¯ ( k ) P k + 1 C ¯ ( k ) M k Ω k 1 M k + Q k ,
where
Ω k = R k + D 0 ( k ) P k + 1 D 0 ( k ) + ϕ 2 D ¯ 0 ( k ) P k + 1 D ¯ 0 ( k ) + D d ( k + d ) P k + d + 1 D d ( k + d ) + ϕ 2 D ¯ d ( k + d ) × P k + d + 1 D ¯ d ( k + d ) + D 0 ( k ) P k + 1 d 1 + ( P k + 1 d 1 ) × D 0 ( k ) i = 1 d ( M k + i d i ) Ω k + i 1 M k + i d i ,
M k = D 0 ( k ) P k + 1 C ( k ) + ϕ 2 D ¯ 0 ( k ) P k + 1 C ¯ ( k ) + ( P k + 1 d 1 ) C ( k ) ,
with
M k 0 = D 0 ( k ) P k + 1 D d ( k ) + ϕ 2 D ¯ 0 ( k ) P k + 1 D ¯ d ( k ) + ( P k + 1 d 1 ) D d ( k ) ,
M k j = D 0 ( k ) P k + 1 j 1 + ( P k + j + 1 d j 1 ) D d ( k + j ) i = 1 j ( M k + i d i ) Ω k + i 1 M k + i j i ,
P k 0 = C ( k ) P k + 1 D d ( k ) + ϕ 2 C ¯ ( k ) P k + 1 D ¯ d ( k ) M k Ω k 1 M k 0 ,
P k j = C ( k ) P k + 1 j 1 M k Ω k 1 M k j , j = 1 , , d 1 .
The terminal values are given by
P N + 1 , P N + i + 1 = 0 , P N + i j = 0 , M N + i j = 0 , Ω N + i = I , i 1 , j = 0 , , d 1 .
Remark 1. 
As can be seen that the costate equations (4)-(6) are quite different from those of Liang [19] and Zhang [20]. What’s more, the coupled Riccati equations (7)-(13) are more complicated than those in Liang [19] and Zhang [20].
It is stressed that the key to solve the optimal LQG control problem is to obtain the solution to the FBSDEs (3) and (4)-(6). We now show the solution to the FBSDEs in the following lemma.
Lemma 1. 
Supposing that Ω k are positive definite for k = 0 , , N , the following equation
ζ k 1 = P k x k + j = 0 d 1 P k j u j + k d + Φ k ,
is the solution to FBSDEs (3) and (4)-(6), with
Φ k = C ( k ) ( P k + 1 w ¯ k M k Ω k 1 Σ k + Φ k + 1 ) + C ¯ ( k ) P k + 1 ρ , Σ k = D 0 ( k ) ( P k + 1 w ¯ k + Φ k + 1 ) + D ¯ 0 ( k ) P k + 1 ρ + D d ( k ) × ( P k + d + 1 w ¯ k + d + Φ k + d + 1 ) + D ¯ d ( k ) P k + d + 1 ρ
+ j = 0 d 1 ( P k + j + 1 d j 1 ) w ¯ k + j i = 0 d ( M k + i d i ) Ω k + i 1 Σ k + i ,
where Φ k + 1 = 0 and Σ k + 1 = 0 for k N . Besides, P k , P k j satisfy the coupled equations (7), (12), (13).
Proof. 
The proof of Lemma 1 is put into Appendix A. □
Now we are ready to present the solution to Problem 1.
Theorem 1. 
There exists the unique F k 1 -measurable u k for Problem 1 if and only if Ω k , for k = 0 , , N , are positive definite. In this case, the optimal controller u k is given by
u k = Ω k 1 M k x k Ω k 1 j = 0 d 1 M k j u j + k d Ω k 1 Σ k .
The associated optimal performance index is as
J N * = x 0 P 0 x 0 + 2 x 0 j = 0 d 1 P 0 j u j d + j = 0 d 1 u j d ( D d ( j ) P j + 1 D d ( j ) + ϕ 2 D ¯ d ( j ) P j + 1 D ¯ d ( j ) ) u j d + 2 j = 0 d 1 i = 0 d 1 u j d D d ( j ) × P j + 1 i j 1 u i d j = 0 d 1 i = 0 d 1 m = 0 d 1 u j d ( M m j m ) Ω m 1 M m i m × u i d + 2 x 0 Φ 0 + 2 k = 0 N w ¯ k Φ k + 1 k = 0 N Σ k Ω k 1 Σ k + k = 0 N T r [ P k + 1 Q w k ] ,
where Ω k , M k , M k j , P k , P k j , Φ k , Σ k satisfy the coupled equations (7)-(13),(16),(17) and P k j = 0 , M k j = 0 for j < 0 .
Proof. 
The proof of Theorem 1 is put into Appendix B. □
Remark 2. 
We make the coefficients of the system (3) and the cost function (2) to be time-invariant. When there is no time delay in system (3), i.e., d = 0 , we have that D d = D ¯ d = 0 . Considering the noise-uncorrelated case with w ¯ k = 0 , it is obviously obtained that the coupled equations (10) and (12) can be rewritten as
M k 0 = 0 , P k 0 = M k Ω k 1 M k 0 = 0 .
Substituting M k 0 and P k 0 into (11) and (13), it can be derived that M k j = 0 , P k j = 0 for j = 0 , , d 1 . Then the difference equations (8) and (9) yield to
Ω k = R + D 0 P k + 1 D 0 + ϕ 2 D ¯ 0 P k + 1 D ¯ 0 , M k = D 0 P k + 1 C + ϕ 2 D ¯ 0 P k + 1 C ¯ .
The optimal controller reduces to
u k = Ω k 1 M k x k ,
which is exactly the result of Moore[2].
Remark 3. 
When the system (3) is a time-invariant system, (3) can be rewritten as
x k + 1 = C ( k ) x k + D 0 ( k ) u k + D d ( k ) u k d + w k
with C ( k ) = C + ν k C ¯ , D 0 ( k ) = D 0 + ν k D 0 ¯ , D d ( k ) = D d + ν k D d ¯ . The performance index becomes
J N = E k = 0 N x k Q x k + u k R u k + x N + 1 P N + 1 x N + 1 .
By using the results of Theorem 1, the optimal time-invariant LQG controller yields that
u k = Ω k 1 M k x k Ω k 1 j = 0 d 1 M k j u j + k d Ω k 1 Σ k ,
and the minimal cost function is as (19) where the coefficient matrices in Ω k , M k , M k j , P k , P k j are time-invariant.
In view of obtaining the special case of optimal LQG control for system (3), now we shall show the results for the general system with multiple delays and multiplicative noises.
Consider the following general discrete time-varying system
x k + 1 = C k ( k ) x k + i = 0 d D k i ( k ) u k i + w k ,
and the cost function is as (2).
Problem 2. 
Find the unique F k 1 -measurable state feedback controller u k , for k = 0 , , N , to minimize the cost function (2) subject to the system (20).
Combining the system (20) and the cost function (2), we apply the Pontryagin’s maximum principle to yield the following costate equations:
ζ N = P N + 1 x N + 1 ,
ζ k 1 = E [ C k ( k ) ζ k | F k 1 ] + Q k x k ,
0 = E [ i = 0 d ( D k + i i ( k + i ) ) ζ k + i | F k 1 ] + R k u k ,
with i = 0 , , d for k = 0 , , N , and ζ k = 0 for k > N .
We introduce the following coupled Riccati equations subject to the system with multiple deleys:
Ω k = R k + i = 0 d ( D i ( k + i ) P k + i + 1 D i ( k + i ) + ϕ 2 D ¯ i ( k + i ) × P k + i + 1 D ¯ i ( k + i ) ) + i = 0 d 1 D i ( k + i ) P k + i + 1 d i 1 + i = 0 d 1 ( P k + i + 1 d i 1 ) D i ( k + i ) i = 1 d ( M k + i d i ) Ω k + i 1 M k + i d i ,
M k j = i = 0 j ( D i ( k + i ) P k + i + 1 D i j + d ( k + i ) + ϕ 2 D ¯ i ( k + i ) × P k + i + 1 D ¯ i j + d ( k + i ) ) + i = 0 j 1 D i ( k + i ) P k + i + 1 j i 1 + i = 0 j ( P k + i + 1 d i 1 ) D i j + d ( k + i ) i = 1 j ( M k + i d i ) Ω k + i 1 M k + i j i ,
P k j = C ( k ) P k + 1 D d j ( k ) + ϕ 2 C ¯ ( k ) P k + 1 D ¯ d j ( k ) + C ( k ) P k + 1 j 1 M k Ω k 1 M k j ,
for j = 0 , , d 1 , where the terminal value is as (14).
Now we give the main results for Problem 2 in the following theorem.
Theorem 2. 
There exists the unique F k 1 -measurable u k for Problem 2 if and only if Ω k , for k = 0 , , N , are positive definite. In this case, the optimal controller u k is calculated by
u k = Ω k 1 M k x k Ω k 1 j = 0 d 1 M k j u j + k d Ω k 1 Σ k ,
where
Σ k = i = 0 d [ D i ( k + i ) ( P k + i + 1 w ¯ k + i + Φ k + i + 1 ) + D ¯ i ( k + i ) P k + i + 1 × ρ ] + j = 0 d 1 ( P k + j + 1 d j 1 ) w ¯ k + j i = 1 d ( M k + i d i ) Ω k + i 1 Σ k + i
and the optimal cost is as
J N * = x 0 P 0 x 0 + 2 x 0 j = 0 d 1 P 0 j u j d + j = 0 d 1 i = 0 d 1 m = 0 d 1 u j d [ D m + d j ( m ) × P m + 1 D m + d i ( m ) + ϕ 2 D ¯ m + d j ( m ) P m + 1 D ¯ m + d i ( m ) + D m + d j ( m ) P m + 1 i m 1 + ( P m + 1 i m 1 ) D m + d i ( m ) ( M m j m ) × Ω m 1 M m i m ] u i d + k = 0 d T r [ P k + 1 Q w k ] 2 x 0 Φ 0 + 2 k = 0 N w ¯ k Φ k + 1 k = 0 N Σ k Ω k 1 Σ k
where D i = 0 for i > d .
In addition, the relationship of the optimal costate ζ k 1 and state x k is as (16) in Lemma 1.
Proof. 
The proof is similar to that of Theorem 1, and to save the space of the paper, we omit it here. □

3. Output Feedback Controller

When the state variable x k are partially observed, we study the following discrete-time stochastic system:
x k + 1 = [ C ( k ) + ν k C ¯ ( k ) ] x k + [ D 0 ( k ) + ν k D ¯ 0 ( k ) ] u k + [ D d ( k ) + ν k D ¯ d ( k ) ] u k d + w k z k = [ H ( k ) + g k H ¯ ( k ) ] x k + e k
where z k R q is the measurement, g k is the scalar white noise with zero mean and variance Q g k , w k and e k are Gaussian zero-mean white noises with covariance Q w k and Q e k . H ( k ) and H ¯ ( k ) are deterministic matrices with compatible dimensions. In this case, the initial value x 0 is known, ν k , w k , g k and e k are independent of each other.
Obviously, there exist multiplicative noises ν k and g k in system (29). As we can not obtain the exact information of the state by (29), we introduce the state estimation to design the controller instead. We first obtain the linear optimal state estimator for by applying standard filtering results in [3]. Then, we will derive the suboptimal linear state estimate feedback controller through the following linearizations.
The aim of this section is to find the suboptimal linear state estimate feedback controller for system (29) in order to minimize the cost function (2).
First, we introduce the linear optimal state estimator in Lemma 2.
Lemma 2. 
Based on the system (29) with input delays and multiplicative noises, the linear optimal estimator is given by
x ^ k + 1 | k = E [ x k + 1 | z 0 , , z k ] = C ( k ) x ^ k | k 1 + D 0 ( k ) u k + D d ( k ) u k d + K k z ˜ k ,
where
z ˜ k = z k C ( k ) x ^ k | k 1 , K k = C ( k ) Σ k | k 1 H ( k ) ( H ( k ) Σ k | k 1 H ( k ) + Q g k H ¯ ( k ) × ( x ^ k | k 1 x ^ k | k 1 + Σ k | k 1 ) H ¯ ( k ) + Q e k ) 1
Besides, the estimator error covariance matrix is
Σ k + 1 | k = E [ ( x k + 1 x ^ k + 1 | k ) ( x k + 1 x ^ k + 1 | k ) | z 0 , , z k ] = C ( k ) Σ k | k 1 C ( k ) K k ( H ( k ) Σ k | k 1 C ( k ) ) + ϕ 2 × [ C ¯ ( k ) ( Σ k | k 1 + x ^ k | k 1 x ^ k | k 1 ) C ¯ ( k ) + D ¯ 0 ( k ) u k × u k D ¯ 0 ( k ) + D ¯ d ( k ) u k d u k d D ¯ d ( k ) ] + Q w k .
The initial values x ^ 0 | 1 = x ¯ 0 and Σ 0 | 1 = P ¯ 0 e .
Proof. 
The proof of Lemma 2 is put into Appendix C. □
Now, the state estimation is obtained, and we can consider (30) as the state instead of the unavailable exact state information. Observing (3) and (30), we know that the filter gain K k on x ^ k | k 1 should be affine, so that we can apply the results of Theorem 1 in this section. Then, we will linearize the filter gain K k .
Applying first order of Taylor expansion on K k through the fixed point x ^ k | k 1 = x ¯ 0 , the linearization of K k yields
K k = K k 0 + K k 1 ( x ^ k | k 1 x ¯ 0 ) + o ( x ^ k | k 1 ) .
Ignoring the quadratic and higher order terms in (31), and plug (30) into it, the approximation of x ^ k + 1 | k becomes
x ^ k + 1 | k C ( k ) x ^ k | k 1 + D 0 ( k ) u k + D d ( k ) u k d + ( K k 0 + K k 1 ( x ^ k | k 1 x ¯ 0 ) ) z ˜ k = ( C ( k ) + K k 1 z ˜ k ) x ^ k | k 1 + D 0 ( k ) u k + D d ( k ) u k d + ( K k 0 K k 1 x ¯ 0 ) z ˜ k .
With (30)-(32), the cost function (2) can be reorganized as
J N e E [ k = 0 N x ^ k | k 1 Q k x ^ k | k 1 + u k R k u k + T r [ Q k Σ k | k 1 ] + x ^ N + 1 | N P N + 1 e x ^ N + 1 | N + T r [ P N + 1 e Σ N + 1 | N ] ] .
In this case, the coupled Riccati equations can be derived as
P k e = C ( k ) P k + 1 e C ( k ) + η k ( K k 1 ) P k + 1 e ( K k 1 ) ( M k e ) ( Ω k e ) 1 M k e + Q k ,
where
Ω k e = R k + D 0 ( k ) P k + 1 e D 0 ( k ) + D d ( k + d ) P k + d + 1 e D d ( k + d ) + D 0 ( k ) P k + 1 d 1 + ( P k + 1 d 1 ) D 0 ( k ) i = 1 d ( M k + i d i ) Ω k + i 1 M k + i d i , M k e = D 0 ( k ) P k + 1 e C ( k ) + ( P k + 1 d 1 ) C ( k ) ,
with
( M k 0 ) e = D 0 ( k ) P k + 1 e D d ( k ) + ( ( P k + 1 d 1 ) e ) D d ( k ) , ( M k j ) e = D 0 ( k ) ( P k + 1 j 1 ) e + ( ( P k + j + 1 d j 1 ) e ) D d ( k + j ) i = 1 j ( ( M k + i d i ) e ) ( Ω k + i e ) 1 ( M k + i j i ) e , ( P k 0 ) e = C ( k ) P k + 1 e D d ( k ) ( M k e ) ( Ω k e ) 1 ( M k 0 ) e , ( P k j ) e = C ( k ) ( P k + 1 j 1 ) e ( M k e ) ( Ω k e ) 1 ( M k j ) e ,
j = 1 , , d 1 , with the terminal values
P N + 1 e , P N + i + 1 e = 0 , ( P N + i j ) e = 0 , ( M N + i j ) e = 0 , Ω N + i e = I , i 1 , j = 0 , , d 1 ,
and
η k = E [ z ˜ k z ˜ k | z 0 , , z k 1 ] = H ( k ) Σ k | k 1 H ( k ) + Q g k H ¯ ( k ) E [ x k x k | z 0 , , z k 1 ] × H ¯ ( k ) + Q e k ,
where
E [ x k + 1 x k + 1 | z 0 , , z k ] = C ( k ) E [ x k x k | z 0 , , z k 1 ] C ( k ) + ϕ 2 C ¯ ( k ) E [ x k x k | z 0 , , z k 1 ] × C ¯ ( k ) + D 0 ( k ) u k u k D 0 ( k ) + D ¯ 0 ( k ) u k u k D ¯ 0 ( k ) + D d ( k ) u k d × u k d D d ( k ) + D ¯ d ( k ) u k d u k d D ¯ d ( k ) Q w k + C ( k ) x ^ k | k 1 u k D 0 ( k ) + ϕ 2 C ¯ ( k ) x ^ k | k 1 u k D ¯ 0 ( k ) + C ( k ) x ^ k | k 1 u k d D d ( k ) + ϕ 2 C ¯ ( k ) × x ^ k | k 1 u k d D ¯ d ( k ) + D 0 ( k ) u k x ^ k | k 1 C ( k ) + ϕ 2 D ¯ 0 ( k ) u k x ^ k | k 1 × C ¯ ( k ) + D 0 ( k ) u k u k d D d ( k ) + ϕ 2 D ¯ 0 ( k ) u k u k d D ¯ d ( k ) + D d ( k ) × u k d x ^ k | k 1 C ( k ) + ϕ 2 D ¯ d ( k ) u k d x ^ k | k 1 C ¯ ( k ) + D d ( k ) u k d × u k D 0 ( k ) + ϕ 2 D ¯ d ( k ) u k d u k D ¯ 0 ( k ) .
with the initial value E [ x 0 x 0 ] = x ¯ 0 x ¯ 0 + P ¯ 0 e .
Now, we can find the suboptimal controller to minimize the cost function (33) subject to (29), by the results of Theorem 1.
Theorem 3. 
The suboptimal linear state estimate feedback controller for system (29) that minimizes the cost function (33) is given by
u k e = ( Ω k e ) 1 M k e x ^ k | k 1 ( Ω k e ) 1 j = 0 d 1 ( M k j ) e u j + k d e .
The minimized cost function is given by
( J N e ) * = x ¯ 0 P 0 e x ¯ 0 + 2 x ¯ 0 j = 0 d 1 ( P 0 j ) e u j d + j = 0 d 1 u j d D d ( j ) P j + 1 e × D d ( j ) u j d + 2 j = 0 d 1 i = 0 d 1 u j d D d ( j ) ( P j + 1 i j 1 ) e u i d e j = 0 d 1 i = 0 d 1 m = 0 d 1 ( u j d e ) ( ( M m j m ) e ) ( Ω m e ) 1 ( M m i m ) e × u i d e + k = 0 N T r [ P k + 1 e Q w k ] ,

4. Numerical examples

Example 1 Consider the scalar case of time-invariant LQG control system (3) in Remark 3. We consider the case that the additive noise w k is the zero-mean white noise. The associate parameters are as
C = 2 , C ¯ = 1 , D 0 = 3 , D ¯ 0 = 1 , D d = 2 , D ¯ d = 2 , d = 5 , ϕ = 1 , Q w = 1 , ρ = 0.6 , w ¯ = 0.2 ,
with the initial values
x 0 = 1 , u 5 = 0.5 , u 4 = 0.8 , u 3 = 1.2 , u 2 = 1 , u 1 = 0.6 ,
and the cost function (2) with Q = 1 , R = 1 , P N + 1 = 1 . When the delay d = 5 , and N = 30 , by applying Theorem 1 and the equations (7)-(13), direct calculations yield that P k , Ω k , M k , P k j , M k j for k = 0 , , N . It can be obviously known that Ω k is positive definite for k = 0 , , N . Thus, there exists a unique u k from Theorem 1, and the optimal controller can be calculated with (18), which is shown in Figure 1.
Accordingly, the associated optimal value of (17) is J N * =107.5150.
In order to illustrate that the proposed LQG controller can minimize performance index, let us consider the time-invariant standard state feedback controller u k = Ω k 1 M k x k . Based on the above parameters and by substituting into cost function, the controller u k are shown in Figure 2, and the associated value is J N * =255.0603, which confirmed the effectiveness of the algorithm.
Example 2 Consider the discrete time-varying LQG control system with multiple delays and multiplicative noises with x k R 2 , u k R 2 , and the cost function (2). The associate coefficients are:
C ( 1 ) = 0 . 2 0 . 1 0 . 8 1 . 1 , C ( 2 ) = 1 . 9 1 . 1 1 . 4 1 . 9 , C ( 3 ) = 1 . 4 0 . 3 0 0 . 4 , C ( 4 ) = 1 . 6 0 . 6 1 . 9 0 . 8 , C ¯ ( 1 ) = 1 . 3 0 . 8 1 . 9 0 . 4 , C ¯ ( 2 ) = 1 . 4 1 . 6 0 . 7 0 . 4 , C ¯ ( 3 ) = 0 . 9 0 . 3 0 . 3 1 . 5 , C ¯ ( 4 ) = 1 . 4 1 1 . 8 1 . 5 , D 0 ( 1 ) = 0 . 3 0 1 . 6 1 . 6 , D 0 ( 2 ) = 1 . 9 1 . 2 0 . 7 0 . 9 , D 0 ( 3 ) = 1 . 7 1 . 3 1 . 8 0 . 1 , D 0 ( 4 ) = 0 . 5 0 . 2 0 . 8 1 , D ¯ 0 ( 1 ) = 0 0 . 8 1 . 7 0 . 7 , D ¯ 0 ( 2 ) = 1 . 9 1 1 . 3 0 . 8 , D ¯ 0 ( 3 ) = 1 . 3 1 . 2 0 . 8 1 . 7 , D ¯ 0 ( 4 ) = 1 . 9 1 . 6 1 . 2 1 . 3 , D d ( 1 ) = 1 . 9 0 . 3 0 . 1 0 . 5 , D d ( 2 ) = 1 . 3 1 . 5 1 0 . 5 , D d ( 3 ) = 1 . 7 0 . 5 1 1 . 1 , D d ( 4 ) = 1 . 7 1 . 5 1 . 4 0 . 6 , D d ( 5 ) = 1 . 8 1 . 9 0 0 . 3 , D d ( 6 ) = 0 . 3 0 . 2 0 . 4 1 . 9 , D d ( 7 ) = 0 . 9 1 0 . 3 1 . 8 , D ¯ d ( 1 ) = 0 . 2 0 . 2 1 . 1 1 . 4 , D ¯ d ( 2 ) = 1 . 9 2 2 1 . 3 , D ¯ d ( 3 ) = 1 1 . 4 1 1 . 3 , D ¯ d ( 4 ) = 1 . 8 1 . 2 1 . 4 0 . 6 , D ¯ d ( 5 ) = 1 . 5 0 . 9 1 . 8 1 . 5 , D ¯ d ( 6 ) = 1 . 8 1 . 9 0 . 4 0 . 1 , D ¯ d ( 7 ) = 1 . 7 0 . 5 1 . 2 0 . 8 , P N + 1 = 0 0 0 0 , Q 1 = 1 0 0 1 , Q 2 = 0 1 1 0 , Q 3 = 0 . 8 1 1 0 . 8 , Q 4 = 1 0 0 1 , R 1 = 2 0 0 2 , R 2 = 0 1 1 0 , R 3 = 1 0 0 1 , R 4 = 1 0 0 1 ,
When the delay time is d = 3 and the final time is N = 4 , given the initial value
x 0 = 1 1 , u 3 = 1.7 1.9 , u 2 = 1 1 , u 1 = 0 1.6 , C ( 0 ) = 2 0 1 2 , C ¯ ( 0 ) = 1 1 2 1 , D 0 ( 0 ) = 1 2 1 0 , D ¯ 0 ( 0 ) = 0 1 2 1 , D d ( 0 ) = 2 1 1 2 , D ¯ d ( 0 ) = 1 2 1 0 ,
by applying Theorem 1 and (8)-(13), it yields that
P 1 = 25.33 2.03 2.03 2.60 , P 2 = 9.10 1.45 1.45 12.47 , P 3 = 2.83 0.76 0.76 2.90 , P 4 = 1 0 0 1 , Ω 1 = 69.41 48.05 48.05 47.49 , Ω 2 = 28.53 5.16 5.16 8.26 , Ω 3 = 9.46 1.83 1.83 7.03 , Ω 4 = 1 0 0 1 , M 1 = 59.03 31.96 46.01 32.31 , M 2 = 15.30 12.32 3.33 4.43 , M 3 = 0.97 0.36 2.39 1.84 , M 4 = 0 0 0 0 .
For i = 1 , 2 , 3 , 4 , Ω i > 0 , thus, there is an optimal solution to the LQG system with multiple delays and state/control noises from Theorem 1. Based on the above data, the optimal controller can be calculated as
u 0 = 0.59 1.35 , u 1 = 0.96 0.46 , u 2 = 1.38 2.34 , u 3 = 0.10 1.34 , u 4 = 0 0 .
According to (17), the optimal performance index of system (3) is J N * = 51 . 1915 .

5. Conclusions

In this paper, the discrete time-varying optimal linear quadratic Gaussian (LQG) control problem involving multiple delays and state/control- dependent noises has been studied. A necessary and sufficient condition for the existence of unique optimal controller to the problem is given, which is based on the obtained maximum principle and the relationship between the state and costate. Under this context, the optimal controller and the minimized performance index are represented. What’s more, as the state variables observed partially, the suboptimal linear state estimate feedback controllers for the LQG models with input delays and multiplicative noises are derived.

Author Contributions

Conceptualization, Qiyan Zhang, Chunyang Sheng, Xiao Lu and Haixia Wang; methodology, Qiyan Zhang, Chunyang Sheng, Xiao Lu and Haixia Wang; validation, Qiyan Zhang, Chunyang Sheng, Xiao Lu and Haixia Wang; formal analysis,Qiyan Zhang, Chunyang Sheng, Xiao Lu and Haixia Wang; investigation, Qiyan Zhang, Chunyang Sheng, Xiao Lu and Haixia Wang; resources, Qiyan Zhang, Chunyang Sheng, Xiao Lu and Haixia Wang; data curation, Qiyan Zhang, Chunyang Sheng, Xiao Lu and Haixia Wang; writing—original draft preparation, Qiyan Zhang and Chunyang Sheng; writing—review and editing, Qiyan Zhang, Chunyang Sheng, Xiao Lu and Haixia Wang; funding acquisition, Chunyang Sheng, Xiao Lu and Haixia Wang. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by the National Natural Science Foundation of China (62273213, 62073199, 62103245, 62203280), Natural Science Foundation of Shandong Province for Innovation and Development Joint Funds (ZR2022LZH001), Natural Science Foundation of Shandong Province (ZR2020MF095, ZR2022MF341), Taishan Scholarship Construction Engineering.

Data Availability Statement

The data of this study is included within the article.

Conflicts of Interest

The authors declare no conflict of interest with respect to the research, authorship, and/or publication of this article.

Appendix A The proof of Lemma 1

Utilizing the maximum principle (4)-(6) to system (3) with cost function (2). We can obtain for k = N ,
0 = ( D 0 ( N ) P N + 1 C ( N ) + ϕ 2 D ¯ 0 ( N ) P N + 1 C ¯ ( N ) ) x N + ( D 0 ( N ) × P N + 1 D 0 ( N ) + ϕ 2 D ¯ 0 ( N ) P N + 1 D ¯ 0 ( N ) + R N ) u N + ( D 0 ( N ) P N + 1 D d ( N ) + ϕ 2 D ¯ 0 ( N ) P N + 1 D ¯ d ( N ) ) u N d + D 0 ( N ) P N + 1 w ¯ N + D ¯ 0 ( N ) P N + 1 ρ .
With (9)-(11), the optimal controller u N is as
u N = Ω N 1 M N x N Ω N 1 j = 0 d 1 M N j u j + N d Ω N 1 Σ N .
From (4)-(6), we also have
ζ N 1 = ( C ( N ) P N + 1 C ( N ) + ϕ 2 C ¯ ( N ) P N + 1 C ( N ) + Q N ) x N + ( C ( N ) P N + 1 D 0 ( N ) + ϕ 2 C ¯ ( N ) P N + 1 D ¯ 0 ( N ) ) u N + ( C ( N ) P N + 1 D d ( N ) + ϕ 2 C ¯ ( N ) P N + 1 D ¯ d ( N ) ) u N d + C ( N ) P N + 1 w ¯ N + C ¯ ( N ) P N + 1 ρ
Substituting (7), (12) and (13), ζ N 1 yields
ζ N 1 = P N x N + j = 0 d 1 P N j u j + N d + Φ N .
We have verified (15) for k = N . Assuming that ζ k 1 are as (15) for all k n + 1 with n > N d , then we will show that (15) also holds for k = n . Set u k to be optimal for all k n + 1 , with equations (3) and (15), ζ n can be calculated as
ζ n = P n + 1 ( C n ( n ) x n + D n 0 ( n ) u n + D n d ( n ) u n d + w n ) + j = 0 d 1 P n + 1 j u j + n + 1 d + Φ n + 1 .
Insert ζ n to (5), (5) will become
0 = M n x n + Ω n u n + j = 0 d 1 M n j u j + n d + Σ n
Thus, the optimal controller is given by
u n = Ω n 1 M n x n Ω n 1 j = 0 d 1 M n j u j + n d Ω n 1 Σ n ,
for n = N , , N d + 1 . Using the equations (3),(5) and (A1), ζ n 1 yields that
ζ n 1 = C ( n ) P n + 1 C ( n ) + ϕ 2 C ¯ ( n ) P n + 1 C ¯ ( n ) + Q n M n Ω n 1 M n × x n + ( C ( n ) P n + 1 D d ( n ) + ϕ 2 C ¯ ( n ) P n + 1 D ¯ d ( n ) M n × Ω n 1 M n 0 ) u n d + j = 1 d 1 C ( n ) P n j 1 M n Ω n 1 M n j u j + n d M n Ω n 1 Σ n + C ( n ) ( P n + 1 w ¯ n + Φ n + 1 ) + C ¯ ( n ) P n + 1 ρ = P n x n + j = 0 d 1 P n j u j + n d + Φ n ,
which implies that (15) holds for k = n , N d < n N .
Then we obtained
ζ N d = P N d + 1 x N d + 1 + j = 0 d 1 P N d + 1 j u j + N 2 d + 1 + Φ N d + 1 , u N d + 1 = Ω N d + 1 1 M N d + 1 x N d + 1 Ω N d + 1 1 j = 0 d 1 M N d + 1 j × u j + k d Ω N d + 1 1 Σ N d + 1 .
Analogy with the method, assuming that ζ k 1 are as (15) for all k n + 1 , n = 0 , , N d , and we will verify that (15) also holds for k = n . As ζ n is calculated as (A1), then for n = 0 , , N d , (6) will be obtained
0 = Ψ + D d ( n ) P n + d + 1 C ( n ) + ϕ 2 D ¯ d ( n ) P n + d + 1 C ¯ ( n ) x n + d + ( M n + d 0 ) ( Ω n + d 1 x n + d Ω n + d 1 j = 0 d 1 M n + d j u j + n Ω n + d 1 × Σ n + d ) + D 0 ( n ) j = 0 d 2 P n + 1 j u j + n d + 1 + D d ( n + d ) j = 0 d 2 P n + d + 1 j u j + n + 1 = Ψ + ( P n + d 1 1 ) C ( n + d ) ( M n + d 2 2 ) Ω n + d 2 1 M n + d 2 × x n + d 2 ( M n + d 2 2 ) Ω n + d 2 1 j = 0 d 1 M n + d 2 j u j + n 2 + ( P n + d 1 1 ) D d ( n + d 2 ) + D 0 ( n ) P n + 1 d 3 u n 2 + ( ( P n + d 0 ) D d ( n + d 1 ) + D 0 ( n ) P n + 1 d 2 ( M n + d 1 1 ) × Ω n + d 1 1 M n + d 1 0 ) u n 1 ( M n + d 0 ) Ω n + d 1 j = 0 d 3 M n + d j u j + n ( M n + d 1 1 ) Ω n + d 1 1 j = 1 d 2 M n + d 1 j u j + n 1 + D 0 ( n ) j = 0 d 4 P n + 1 j × u j + n d + 1 + D d ( n + d ) j = 0 d 4 P n + d + 1 j u j + n + 1 i = d 1 d ( M n + i d i ) Ω n + i 1 Σ n + i + i = d 2 d 1 P n + 1 + i d 1 i w ¯ n + i
where
Ψ = D 0 ( n ) P n + 1 C ( n ) + ϕ 2 D ¯ 0 ( n ) P n + 1 C ¯ ( n ) x n + ( D 0 ( n ) × P n + 1 D 0 ( n ) + ϕ 2 D ¯ 0 ( n ) P n + 1 D ¯ 0 ( n ) + R n + D 0 ( n ) P n + 1 d 1 + D d ( n + d ) P n + d + 1 D d ( n + d ) + ϕ 2 D ¯ d ( n + d ) P n + d + 1 × D ¯ d ( n + d ) ) u n + ( D 0 ( n ) P n + 1 D d ( n ) + ϕ 2 D ¯ 0 ( n ) P n + 1 × D ¯ d ( n ) ) u n d + D 0 ( n ) ( P n + 1 w ¯ n + Φ n + 1 ) + D ¯ 0 ( n ) P n + 1 ρ + D d ( n + d ) ( P n + d + 1 w ¯ n + d + Φ n + 1 ) + D ¯ d ( n + d ) P n + d + 1 ρ
After inserting (3) and (A2), and combing like terms, we can summarize that
0 = Ψ + ( P n + 1 d 1 ) x n + 1 + ( ( P n + 2 d 2 ) D d ( n + 1 ) + D 0 ( n ) P n + 1 0 ( M n + 1 d 1 ) Ω n + 1 1 M n + 1 0 ) u n d + 1 + ( ( P n + 3 d 3 ) D d ( n + 2 ) + D 0 ( n ) P n + 1 1 i = 1 2 ( M n + i d i ) Ω n + i 1 M n + i 2 i ) u n d + 2 + + ( ( P n + d 1 1 ) D d ( n + d 2 ) + D 0 ( n ) P n + 1 d 3 i = 1 d 2 ( M n + i d i ) × Ω n + i 1 M n + i d 2 i ) u n 2 + ( ( P n + d 0 ) D d ( n + d 1 ) + D 0 P n + 1 d 2 i = 1 d 1 ( M n + i d i ) Ω n + i 1 M n + i d 1 i ) u n 1 + i = 1 d ( M n + i d i ) Ω n + i 1 × M n + i d i u n + i = 1 d 1 P n + 1 + i d 1 i w ¯ n + i i = 1 d ( M n + i d i ) Ω n + i 1 Σ n + i = M n x n + Ω n u n + j = 0 d 1 M n j u j + n d + Σ n .
Now, the optimal controller for n = 0 , , N d is obtained as
u n = Ω n 1 M n x n Ω n 1 j = 0 d 1 M n j u j + n d Ω n 1 Σ n .
In the same way, substituting u n into (5), we can also prove that
ζ n 1 = P n x n + j = 0 d 1 P n j u j + n d + Φ n , n = 0 , , N d .
This completes the proof of the lemma.

Appendix B The proof of Theorem 1

"Necessity": Suppose there exists the unique F k 1 -measurable u k to make the cost function (2) minimized. We will show by induction that Ω k , k = d , , N are positive definite and the optimal controller can be designed as (15). Define
J ( k ) = i = k N E x i Q i x i + u i R i u i + x N + 1 P N + 1 x N + 1 ,
for k = 0 , , N , and when k = N the above equation becomes
J ( N ) = E [ x N Q N x N + u N R N u N + ( C N ( N ) x N + D N 0 ( N ) u N + D N d ( N ) u N d + w N ) P N + 1 ( C N ( N ) x N + D N 0 ( N ) × u N + D N d ( N ) u N d + w N ) ] .
Using (3), we can obviously know that the uniqueness of the optimal controller only depends on whether u N > 0 . Then setting x N = 0 , and u N d = 0 , J ( N ) can be presented as
J ( N ) = u N Ω N u N + 2 u N ( D 0 ( N ) P N + 1 w ¯ N + D ¯ 0 ( N ) P N + 1 ρ ) + T r [ P N + 1 Q w N ] .
We know that J ( N ) is expressed as a quadratic function of u N , and as there is a unique solution for system (3), then J ( N ) > 0 , it follows that Ω N > 0 , i.e. Ω k is positive definite for k = N . In order to accomplish the proof, we assume Ω k > 0 for all k n + 1 . Then we will prove that Ω n > 0 . With (3), (5) and (6), for k n + 1 , we construct that
E x k ζ k 1 x k + 1 ζ k = E x k Q k x k + u k R k u k + E [ u k ( D k + d d ( k + d ) ) ζ k + d u k d ( D k d ( k ) ) ζ k ] E w k ζ k .
Adding from k = n + 1 to k = N on both sides of the above equation in order to get the form of J ( N ) , we have
E [ x n + 1 ζ n x N + 1 ζ N ] = k = n + 1 N E x k Q k x k + u k R k u k + k = n + 1 N E u k ( D k + d d ( k + d ) ) ζ k + d u k d ( D k d ( k ) ) ζ k k = n + 1 N w k ζ k .
Then
E k = n + 1 N x k Q k x k + u k R k u k + x N + 1 P N + 1 x N + 1 = E x n + 1 ζ n + k = n + 1 n + d u k d ( D k d ( k ) ) ζ k + k = n + 1 N w k ζ k .
Using (2), it yields that
J ( n ) = E [ x n + 1 ζ n + x n Q n x n + u n R n u n + k = n + 1 n + d u k d ( D k d ( k ) ) ζ k + k = n + 1 N w k ζ k ] .
Setting x n = 0 , u n i = 0 as same as the condition k = N . And plugging (16) into (A4), we obtain
J ( n ) = u n ( D 0 ( n ) P n + 1 D 0 ( n ) + ϕ 2 D ¯ 0 ( n ) P n + 1 D ¯ 0 ( n ) + R n + D 0 ( n ) P n + 1 d 1 + D d ( n + d ) P n + d + 1 D d ( n + d ) + ϕ 2 × D ¯ d ( n + d ) P n + d + 1 D ¯ d ( n + d ) + ( P n + 1 d 1 ) D 0 ( n ) i = 1 d ( M n + i d i ) Ω n + i 1 M n + i d i ) u n + k = n N w k ζ k + u n ( D 0 ( n ) × P n + 1 w ¯ n + D ¯ 0 ( n ) P n + 1 ρ ) + u n ( D d ( n + d ) P n + d + 1 × w ¯ n + D ¯ d ( n + d ) P n + d + 1 ρ ) + u n i = 0 d 1 ( P n + d i i ) × w ¯ n + d 1 i + D 0 ( n ) Φ n + 1 + D d ( n + d ) Φ n + d + 1 .
Similarly to the case Ω N > 0 above, we obviously get Ω n > 0 for all k = 0 , , N . This ends the proof of necessity.
"Sufficiency": Suppose Ω k > 0 for k 0 is true, we will show the uniqueness of the F k 1 -measurable u k to minimize (2). Denoted by
V k ( x k ) = E [ x k P k x k + 2 x k j = 0 d 1 P k j u j d + k + j = 0 d 1 u j d + k ( ( D k + j d ( k + j ) ) × P k + j + 1 D k + j d ( k + j ) ) u j d + k + 2 j = 0 d 1 i = 0 d 1 u j d + k × ( D k + j d ( k + j ) ) P k + j + 1 i j 1 u i d + k j = 0 d 1 i = 0 d 1 m = 0 d 1 u j d + k × ( M k + m j m ) Ω k + m 1 M k + m i m u i d + k ] + 2 x k Φ k .
First, for V k + 1 ( x k + 1 ) , using the equivalent substitution j = j + 1 , i = i + 1 , and m = m + 1 , we calculate as follows
V k + 1 ( x k + 1 ) = E [ x k + 1 P k + 1 x k + 1 + 2 x k + 1 j = 0 d 1 P k + 1 j 1 u j d + k + j = 0 d 1 u j d + k × ( D k + j d ( k + j ) ) P k + j + 1 D k + j d ( k + j ) u j d + k + 2 j = 0 d 1 i = 0 d 1 u j d + k ( D k + j d ( k + j ) ) P k + j + 1 i j 1 u i d + k j = 0 d 1 i = 0 d 1 m = 0 d 1 u j d + k × ( M k + m j m ) Ω k + m 1 M k + m i m u i d + k ] + E [ 2 x k + 1 P k + 1 d 1 u k + u k × ( D k + d d ( k + d ) ) P k + d + 1 D k + d d ( k + d ) u k u k d ( D k d ( k ) ) × P k + 1 D k d ( k ) u k d + 2 j = 0 d 1 u j d + k ( D k + j d ( k + j ) ) P k + 1 + j d 1 j u k 2 i = 0 d 1 u k d ( D k d ( k ) ) P k + 1 i 1 u i d + k 2 u k d ( D k d ( k ) ) P k + 1 d 1 u k + j = 0 d 1 i = 0 d 1 u j d + k ( M k j ) Ω k 1 M k i u k d + i m = 0 d 1 u k ( M k + m d m ) × Ω k + m 1 M k + m d m u k u k ( M k + d 0 ) Ω k + d 1 M k + d 0 u k + u k ( M k d ) × Ω k 1 M k d u k j = 0 d 1 m = 0 d 1 u j d + k ( M k + m j m ) Ω k + m 1 M k + m d m u k i = 0 d 1 m = 0 d 1 u k ( M k + m d m ) Ω k + m 1 M k + m i m u i d + k ] + 2 x k + 1 Φ k + 1 .
Construct the equation V k ( x k ) V k + 1 ( x k + 1 ) , then we have
V k ( x k ) V k + 1 ( x k + 1 ) = x k Q k x k + u k R k u k + ( u k + Ω k 1 M k x k + Ω k 1 j = 0 d 1 M k j u j + k d + Ω k 1 Σ ) Ω k ( u k + Ω k 1 M k x k + Ω k 1 j = 0 d 1 M k j u j + k d Ω k 1 × Σ k ) + Σ k Ω k 1 Σ k T r [ P k + 1 Q w k ] 2 w ¯ k Φ k + 1 .
Denote
Δ k = u k + Ω k 1 M k x k + Ω k 1 j = 0 d 1 M k j u j + k d + Ω k 1 Σ k ,
and by virtue of (A6) and (7)-(13), and adding from k = 0 to k = N on both sides of (A5), then we get
V 0 ( x 0 ) V N + 1 ( x N + 1 ) = k = 0 N [ x k Q k x k + u k R k u k Δ k Ω k Δ k + Σ k Ω k 1 Σ k 2 w ¯ k Φ k + 1 T r [ P k + 1 Q w k ] ]
Then the cost function (2) becomes
J N = V 0 ( x 0 ) + k = 0 N ( Δ k Ω k Δ k Σ k Ω k 1 Σ k + 2 w ¯ k Φ k + 1 ) + k = 0 N T r [ P k + 1 Q w k ] .
As Ω k > 0 , the unique optimal controller must match the condition Δ k = 0 . In this case, the cost function (2) will be the minimum, i.e., the optimal controller is
u k * = Ω k 1 M k x k Ω k 1 j = 0 d 1 M k j u j + k d Ω k 1 Σ k .
and the optimal cost is as (19).
Above all, the proof of sufficiency is completed.

Appendix C The proof of Lemma 2

By applying standard filtering results in [? ], we can obtain the linear optimal estimator for system (29) as follows.
x ^ k + 1 | k = E [ x k + 1 | z 0 , , z k ] = E [ x k + 1 | z ˜ 0 , , z ˜ k ] = E [ x k + 1 | z ˜ k ] + E [ x k + 1 | z ˜ 0 , , z ˜ k 1 ] E [ x k + 1 ]
In view of the jointly gaussian nature of x k + 1 and z ˜ k , we know
E [ x k + 1 | z ˜ k ] = E [ x k + 1 ] + c o v ( x k + 1 , z ˜ k ) [ c o v ( z ˜ k , z ˜ k ) ] 1 z ˜ k .
Using (29) and the orthogonality of x ^ k | k 1 and x k x ^ k | k 1 , the covariance matrixes yield
c o v ( x k + 1 , z ˜ k ) = C ( k ) Σ k | k 1 H ( k ) , c o v ( z ˜ k , z ˜ k ) = H ( k ) Σ k | k 1 H ( k ) + Q g k H ¯ ( k ) ( x ^ k | k 1 x ^ k | k 1 + Σ k | k 1 ) H ¯ ( k ) + Q e k ,
where x k x ^ k | k 1 is independent of e k with zero mean,and the error covariance matrix
Σ k + 1 | k = C ( k ) Σ k | k 1 C ( k ) K k ( H ( k ) Σ k | k 1 C ( k ) ) + ϕ 2 × [ C ¯ ( k ) ( Σ k | k 1 + x ^ k | k + 1 x ^ k | k + 1 ) C ¯ ( k ) + D ¯ 0 ( k ) × u k u k D ¯ 0 ( k ) + D ¯ d ( k ) u k d u k d D ¯ d ( k ) ] + Q f k .
Substituting above equations into (A8), it becomes
x ^ k + 1 | k = C ( k ) x ^ k | k 1 + D 0 ( k ) u k + D d ( k ) u k d + K k z ˜ k
with
K k = C ( k ) Σ k | k 1 H ( k ) ( H ( k ) Σ k | k 1 H ( k ) + Q g k H ¯ ( k ) × ( x ^ k | k 1 x ^ k | k 1 + Σ k | k 1 ) H ¯ ( k ) + Q e k ) 1 .
The proof of Lemma 2 is completed.

References

  1. H. Kwakernaak, Optimal filtering in linear systems with time delays, IEEE Transactions on Automatic Control, vol. 12, no. 2, pp. 169-173, 1967. [CrossRef]
  2. J. B. Moore, X. Y. Zhou, A. E. B. Lim, Discrete time LQG controls with control dependent noise, Systems and Control Letters, vol. 36, no. 3, pp. 199-206, 1999. [CrossRef]
  3. B. D. O. Anderson, J. B. Moore, Optimal filtering, Courier Corporation, 2012.
  4. W. Sun, K. Wei, Z. Liu, et al, Linear quadratic gaussian control for wireless communication reliability for a mobile monitoring robot in a UHV power substation, IEEE Systems Journal, 2022.
  5. A. Kalbat, Linear quadratic gaussian (lqg) control of wind turbines, 2013 3rd International Conference on Electric Power and Energy Conversion Systems, pp. 1-5, 2013.
  6. X. Yu, X. Liu, X. Wang, et al, Vibration control of improved LQG for wheel drive electric vehicle based on uncertain parameters, Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering, vol. 235, no. 8, pp. 2253-2264, 2021. [CrossRef]
  7. A. Florescu, A. Bratcu, I. Munteanu, et al, LQG optimal control applied to on-board energy management system of all-electric vehicles, IEEE Transactions on Control Systems Technology, vol. 23, no. 4, pp. 1427-1439, 2014. [CrossRef]
  8. M. Basin, J. Rodriguez-Gonzalez, R. Martinez-Zuniga, Optimal control for linear systems with time delay in control input, Journal of The Franklin Institute, vol. 341, no. 3, pp. 267-278, 2004.
  9. E. G. Strangas, G. Clerc and H. Razik,and A. Soualhi, Electrical safety-engineering of renewable energy systems, Wiley Press, 2021.
  10. F. Cacace, F. Conte, A. Germani, Memoryless approach to the LQ and LQG problems with variable input delay, IEEE Transactions on Automatic Control, vol. 61, no. 1, pp. 216-221, 2015. [CrossRef]
  11. N. Matni, J. C. Doyle, Optimal distributed LQG state feedback with varying communication delay, 52nd IEEE Conference on Decision and Control, pp. 5890-5896, 2013.
  12. M. Basin M, J. Rodriguez-Gonzalez, Optimal control for linear systems with multiple time delays in control input, IEEE Transactions on Automatic Control, vol. 51, no. 1, pp. 91-97, 2006.
  13. X. Lu, N. Wang, Q. Qi, et al, Further Study on Networked Control Systems with Unreliable Communication Channels, International Journal of Control Automation and Systems, vol. 19, no. 2, pp. 1-10, 2020.
  14. H. Zhang, G. Duan, L. Xie, Linear quadratic regulation for linear time-varying systems with multiple input delays, Automatica, vol. 42, no. 9, pp. 1465-1476, 2006. [CrossRef]
  15. V. Gupta, B. Hassibi, R. M. Murray, Optimal LQG control across packet-dropping links, Systems and Control Letters, vol. 56, no. 6, pp. 439-446, 2007. [CrossRef]
  16. X. Liang, J. Xu, Control for networked control systems with remote and local controllers over unreliable communication channel, Automatica, vol. 98, pp. 86-94, 2018. [CrossRef]
  17. X. Chen, Y. Zhu, Uncertain random linear quadratic control with multiplicative and additive noises, Asian Journal of Control, vol. 23, no. 6, pp. 2849-2864, 2021. [CrossRef]
  18. H. Zhang, L. Li, J. Xu, et al, Linear quadratic regulation and stabilization of discrete-time systems with delay and multiplicative noise, IEEE Transactions on Automatic Control, vol. 60, no. 10, pp. 2599-2613, 2015.
  19. X. Liang, J. Xu, H. Zhang, Discrete-time LQG control with input delay and multiplicative noise, IEEE Transactions on Aerospace and Electronic Systems, vol. 53, no. 6, pp. 3079-3090, 2017. [CrossRef]
  20. X. Liang, J. Xu, H. Zhang, Optimal control and stabilization for networked control systems with packet dropout and input delay, IEEE Transactions on Circuits and Systems II: Express Briefs, vol. 64, no. 9, pp. 1087-1091, 2016. [CrossRef]
Figure 1. The Optimal Controller u k .
Figure 1. The Optimal Controller u k .
Preprints 73765 g001
Figure 2. The Non-optimal Controller u k .
Figure 2. The Non-optimal Controller u k .
Preprints 73765 g002
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
Prerpints.org logo

Preprints.org is a free preprint server supported by MDPI in Basel, Switzerland.

Subscribe

© 2024 MDPI (Basel, Switzerland) unless otherwise stated