Distributed Estimation and Data-Driven Formation Control of Air–Ground Systems via Efficient Reinforcement Learning

This article investigates the optimal distributed formation control for heterogeneous air–ground vehicle systems using a data-efficient, off-policy reinforcement learning algorithm. Initially, a distributed predefined-time observer is proposed under a time-varying formation function to provide estimation of the leader’s states, which serves as a fundamental element for formation tracking control. Subsequently, a model-based formation control policy is developed and learned to achieve optimal formation tracking performance. To alleviate the substantial computational burden of existing learning algorithms, we propose a purely data-driven, off-policy learning algorithm that utilizes measured system data to learn the optimal control policy. This approach eliminates the need for the Kronecker product operation in data matrix construction. Furthermore, the proposed algorithm learns an initial stabilizing policy directly from system data. Unlike conventional algorithms that necessitate a full rank condition, extensive data storage, and a predefined initial stabilizing control policy based on known system dynamics, our method eliminates these prerequisites. Finally, we present a numerical example to validate the effectiveness of our algorithm.