Optimal Policy Characterization Enhanced Actor-Critic Approach for Electric Vehicle Charging Scheduling in a Power Distribution Network

We study the scheduling of large-scale electric vehicle (EV) charging in a power distribution network under random renewable generation and electricity prices. The problem is formulated as a stochastic dynamic program with unknown state transition probability. To mitigate the curse of dimensionality, we establish the nodal multi-target (NMT) characterization of the optimal scheduling policy: all EVs with the same deadline at the same bus should be charged to approach a single target of remaining energy demand. We prove that the NMT characterization is optimal under arbitrarily random system dynamics. To adaptively learn the dynamics of system uncertainty, we propose a model-free soft-actor-critic (SAC) based method to determine the target levels for the characterized NMT policy. The proposed SAC + NMT approach significantly outperforms existing deep reinforcement learning methods (in our numerical experiments on the IEEE 37-node test feeder), as the established NMT characterization sharply reduces the dimensionality of neural network outputs without loss of optimality.