Data Driven Solution to Market Equilibrium via Deep Reinforcement Learning

Equilibrium analysis is an effective tool to analyze the operation efficiency of the electricity market, but how to efficiently solve the power market equilibrium model with multiagent participation remains to be further studied. The traditional model transformation method has low computational efficiency and poor ability to deal with strong uncertainty. The optimality and convergence of the solution are not fully guaranteed. Therefore, this paper intends to use a multi-agent deep reinforcement learning algorithm to reach a fast solution to the market equilibrium model. Here we take the interaction between multiple virtual power plants (VPPs) and the upstream power grid as an example. A bi-level mathematical programming with equilibrium constraints (MPEC) is constructed. The upper level aims to minimize the cost of running the VPPs, the lower level aims to minimize the cost of running the power grid. A multi-agent reinforcement learning algorithm is employed to solve the MPEC based on deep deterministic policy gradient (DDPG), which is a data-driven, selflearning and model-free approach without the modeling and computational complexity caused by existing methods. Finally, an example of iteration between an IEEE-30 node power grid system and a VPP is given to verify the rationality and effectiveness of the proposed algorithm.