Large-Scale Cooperative Task Offloading and Resource Allocation in Heterogeneous MEC Systems via Multiagent Reinforcement Learning

In multi-access edge computing systems, existing task offloading methods have provided ultra-short latency services for heterogeneous tasks on mobile devices (MDs). Nevertheless, the complexity of MEC systems grows exponentially with the number of MDs or edge servers (ES), so learning a good offloading policy is a huge challenge when the number of MDs or ESs is large. Moreover, MDs are often unable to find optimal ESs for offloading since the restricted ESs infrastructures and the spatiotemporally imbalanced task offloading requirements. To solve these problems, we propose a Curriculum Spatio-Temporal Multi-Agent Actor-Critic (CSTMAAC)-based task offloading method. Each ES is regarded as an agent and the problem is formulated as a multi-objective optimization task. To adapt to the large-scale MEC systems, we first introduce an evolutionary curriculum learning by gradually raising the number of trained ES agents in a phased way. Second, to facilitate the coordination of the offloading policies among geographically distributed ESs, we design an attention-based centralized critic-network. Besides, a delayed access mechanism is introduced that uses information about future task processing competition to capture the impact of potential future task processing contention and help ES agents obtain a better offloading strategy. Finally, critic-network is expanded to multi-critics and a dynamic weight mechanism is designed to adaptively optimize multi-objectives and obtain a good balance for multiple objectives. Real-world datasets used in experiments demonstrate that CSTMAAC raises task completion rates and total utility by 13.01% 15.21% and 16.89% 18.32% compared with the existing algorithms.