Comparing policy gradient and value function based reinforcement learning methods in simulated electrical power trade

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

In electrical power engineering, reinforcement learning algorithms can be used to model the strategies of electricity market participants. However, traditional value function based reinforcement learning algorithms suffer from convergence issues when used with value function approximators. Function approximation is required in this domain to capture the characteristics of the complex and continuous multivariate problem space. The contribution of this paper is the comparison of policy gradient reinforcement learning methods, using artificial neural networks for policy function approximation, with traditional value function based methods in simulations of electricity trade. The methods are compared using an AC optimal power flow based power exchange auction market model and a reference electric power system model.
LanguageEnglish
Pages373-380
Number of pages8
JournalIEEE Transactions on Power Systems
Volume27
Issue number1
Early online date6 Oct 2011
DOIs
Publication statusPublished - Feb 2012

Fingerprint

Reinforcement learning
Learning algorithms
Electric power systems
Electricity
Neural networks

Keywords

  • artificial Intelligence
  • game theory
  • gradient methods
  • learning control systems
  • neural network applications
  • power system economics

Cite this

@article{e2c8cb02050c4907a68728ee70fd9524,
title = "Comparing policy gradient and value function based reinforcement learning methods in simulated electrical power trade",
abstract = "In electrical power engineering, reinforcement learning algorithms can be used to model the strategies of electricity market participants. However, traditional value function based reinforcement learning algorithms suffer from convergence issues when used with value function approximators. Function approximation is required in this domain to capture the characteristics of the complex and continuous multivariate problem space. The contribution of this paper is the comparison of policy gradient reinforcement learning methods, using artificial neural networks for policy function approximation, with traditional value function based methods in simulations of electricity trade. The methods are compared using an AC optimal power flow based power exchange auction market model and a reference electric power system model.",
keywords = "artificial Intelligence, game theory, gradient methods, learning control systems, neural network applications, power system economics",
author = "Richard Lincoln and Stuart Galloway and Bruce Stephen and Graeme Burt",
year = "2012",
month = "2",
doi = "10.1109/TPWRS.2011.2166091",
language = "English",
volume = "27",
pages = "373--380",
journal = "IEEE Transactions on Power Systems",
issn = "0885-8950",
number = "1",

}

TY - JOUR

T1 - Comparing policy gradient and value function based reinforcement learning methods in simulated electrical power trade

AU - Lincoln, Richard

AU - Galloway, Stuart

AU - Stephen, Bruce

AU - Burt, Graeme

PY - 2012/2

Y1 - 2012/2

N2 - In electrical power engineering, reinforcement learning algorithms can be used to model the strategies of electricity market participants. However, traditional value function based reinforcement learning algorithms suffer from convergence issues when used with value function approximators. Function approximation is required in this domain to capture the characteristics of the complex and continuous multivariate problem space. The contribution of this paper is the comparison of policy gradient reinforcement learning methods, using artificial neural networks for policy function approximation, with traditional value function based methods in simulations of electricity trade. The methods are compared using an AC optimal power flow based power exchange auction market model and a reference electric power system model.

AB - In electrical power engineering, reinforcement learning algorithms can be used to model the strategies of electricity market participants. However, traditional value function based reinforcement learning algorithms suffer from convergence issues when used with value function approximators. Function approximation is required in this domain to capture the characteristics of the complex and continuous multivariate problem space. The contribution of this paper is the comparison of policy gradient reinforcement learning methods, using artificial neural networks for policy function approximation, with traditional value function based methods in simulations of electricity trade. The methods are compared using an AC optimal power flow based power exchange auction market model and a reference electric power system model.

KW - artificial Intelligence

KW - game theory

KW - gradient methods

KW - learning control systems

KW - neural network applications

KW - power system economics

U2 - 10.1109/TPWRS.2011.2166091

DO - 10.1109/TPWRS.2011.2166091

M3 - Article

VL - 27

SP - 373

EP - 380

JO - IEEE Transactions on Power Systems

T2 - IEEE Transactions on Power Systems

JF - IEEE Transactions on Power Systems

SN - 0885-8950

IS - 1

ER -