Adaptive Dynamic Programming Infinite‐Horizon Optimal Tracking Control for Stochastic Linear Discrete‐Time Systems

This paper investigates stochastic discrete‐time systems with multiplicative state‐dependent and input‐dependent noise via a novel adaptive dynamic programming(ADP) based control method combined with optimal stationary control techniques. Imposing a notably greater difficulty, the tracking control problem without the knowledge of system dynamics and the reference system has been generalized that the system dynamics do not have to be Hurwitz which is more practically relevant. An augmented system has been constructed while a discount factor has been introduced into the cost function. After the discount factor has been brought in to solve the stochastic algebraic Riccati equation(SARE), the linear quadratic tracking(LQT) problem has been proved to be well‐posed. Hence, we develop a second‐order moment formulation to solve the SARE. Based on stochastic adaptive control, a novel on‐policy ADP algorithm has been proposed to solve the LQT problem by only the state and input data. The convergence and stability of the novel ADP algorithm has been rigorously investigated and discussed. Finally, numerical simulations and practical experiments of two distinguished systems are performed to validate the effectiveness and practicability of the proposed ADP methodology.