A novel update mechanism for Q-Networks based on extreme learning machines

Callum Wilson; Annalisa Riccardi; Edmondo Minisci

doi:10.1109/IJCNN48605.2020.9207098

A novel update mechanism for Q-Networks based on extreme learning machines

Callum Wilson, Annalisa Riccardi, Edmondo Minisci

Mechanical And Aerospace Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution book

3 Citations (Scopus)

27 Downloads (Pure)

Abstract

Reinforcement learning is a popular machine learning paradigm which can find near optimal solutions to complex problems. Most often, these procedures involve function approximation using neural networks with gradient based updates to optimise weights for the problem being considered. While this common approach generally works well, there are other update mechanisms which are largely unexplored in reinforcement learning. One such mechanism is Extreme Learning Machines. These were initially proposed to drastically improve the training speed of neural networks and have since seen many applications. Here we attempt to apply extreme learning machines to a reinforcement learning problem in the same manner as gradient based updates. This new algorithm is called Extreme Q-Learning Machine (EQLM). We compare its performance to a typical Q-Network on the cart-pole task - a benchmark reinforcement learning problem - and show EQLM has similar long-term learning performance to a Q-Network.

Original language	English
Title of host publication	2020 International Joint Conference on Neural Networks (IJCNN)
Place of Publication	Piscataway, NJ.
Publisher	IEEE
Number of pages	7
ISBN (Electronic)	9781728169262
ISBN (Print)	9781728169279
DOIs	https://doi.org/10.1109/IJCNN48605.2020.9207098
Publication status	Published - 28 Sept 2020
Event	IEEE World Congress on Computational Intelligence 2020 - Glasgow, United Kingdom Duration: 19 Jul 2020 → 24 Jul 2020 https://wcci2020.org/

Conference

Conference	IEEE World Congress on Computational Intelligence 2020
Abbreviated title	WCCI
Country/Territory	United Kingdom
City	Glasgow
Period	19/07/20 → 24/07/20
Internet address	https://wcci2020.org/

Keywords

reinforcement learning
extreme learning machine (ELM)
neural network

Access to Document

10.1109/IJCNN48605.2020.9207098

Wilson-etal-WCCI2020-A-novel-update-mechanism-for-Q-NetworksAccepted author manuscript, 312 KB

Cite this

@inproceedings{481a023bc55745288d79ac7ceadab638,

title = "A novel update mechanism for Q-Networks based on extreme learning machines",

abstract = "Reinforcement learning is a popular machine learning paradigm which can find near optimal solutions to complex problems. Most often, these procedures involve function approximation using neural networks with gradient based updates to optimise weights for the problem being considered. While this common approach generally works well, there are other update mechanisms which are largely unexplored in reinforcement learning. One such mechanism is Extreme Learning Machines. These were initially proposed to drastically improve the training speed of neural networks and have since seen many applications. Here we attempt to apply extreme learning machines to a reinforcement learning problem in the same manner as gradient based updates. This new algorithm is called Extreme Q-Learning Machine (EQLM). We compare its performance to a typical Q-Network on the cart-pole task - a benchmark reinforcement learning problem - and show EQLM has similar long-term learning performance to a Q-Network.",

keywords = "reinforcement learning, extreme learning machine (ELM), neural network",

author = "Callum Wilson and Annalisa Riccardi and Edmondo Minisci",

note = "{\textcopyright} 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.; IEEE World Congress on Computational Intelligence 2020, WCCI ; Conference date: 19-07-2020 Through 24-07-2020",

year = "2020",

month = sep,

day = "28",

doi = "10.1109/IJCNN48605.2020.9207098",

language = "English",

isbn = "9781728169279",

booktitle = "2020 International Joint Conference on Neural Networks (IJCNN)",

publisher = "IEEE",

url = "https://wcci2020.org/",

}

TY - GEN

T1 - A novel update mechanism for Q-Networks based on extreme learning machines

AU - Wilson, Callum

AU - Riccardi, Annalisa

AU - Minisci, Edmondo

N1 - © 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

PY - 2020/9/28

Y1 - 2020/9/28

N2 - Reinforcement learning is a popular machine learning paradigm which can find near optimal solutions to complex problems. Most often, these procedures involve function approximation using neural networks with gradient based updates to optimise weights for the problem being considered. While this common approach generally works well, there are other update mechanisms which are largely unexplored in reinforcement learning. One such mechanism is Extreme Learning Machines. These were initially proposed to drastically improve the training speed of neural networks and have since seen many applications. Here we attempt to apply extreme learning machines to a reinforcement learning problem in the same manner as gradient based updates. This new algorithm is called Extreme Q-Learning Machine (EQLM). We compare its performance to a typical Q-Network on the cart-pole task - a benchmark reinforcement learning problem - and show EQLM has similar long-term learning performance to a Q-Network.

AB - Reinforcement learning is a popular machine learning paradigm which can find near optimal solutions to complex problems. Most often, these procedures involve function approximation using neural networks with gradient based updates to optimise weights for the problem being considered. While this common approach generally works well, there are other update mechanisms which are largely unexplored in reinforcement learning. One such mechanism is Extreme Learning Machines. These were initially proposed to drastically improve the training speed of neural networks and have since seen many applications. Here we attempt to apply extreme learning machines to a reinforcement learning problem in the same manner as gradient based updates. This new algorithm is called Extreme Q-Learning Machine (EQLM). We compare its performance to a typical Q-Network on the cart-pole task - a benchmark reinforcement learning problem - and show EQLM has similar long-term learning performance to a Q-Network.

KW - reinforcement learning

KW - extreme learning machine (ELM)

KW - neural network

UR - https://arxiv.org/abs/2006.02986

U2 - 10.1109/IJCNN48605.2020.9207098

DO - 10.1109/IJCNN48605.2020.9207098

M3 - Conference contribution book

SN - 9781728169279

BT - 2020 International Joint Conference on Neural Networks (IJCNN)

PB - IEEE

CY - Piscataway, NJ.

T2 - IEEE World Congress on Computational Intelligence 2020

Y2 - 19 July 2020 through 24 July 2020

ER -

A novel update mechanism for Q-Networks based on extreme learning machines

Abstract

Conference

Keywords

Access to Document

Other files and links

Fingerprint