"Superstition" in the network: Deep reinforcement learning plays deceptive games

Philip Bontrager, Ahmed Khalifa, Damien Anderson, Matthew Stephenson, Christoph Salge, Julian Togelius

Research output: Contribution to journalConference Contributionpeer-review

8 Citations (Scopus)
7 Downloads (Pure)

Abstract

Deep reinforcement learning has learned to play many games well, but failed on others. To better characterize the modes and reasons of failure of deep reinforcement learners, we test the widely used Asynchronous Actor-Critic (A2C) algorithm on four deceptive games, which are specially designed to provide challenges to game-playing agents. These games are implemented in the General Video Game AI framework, which allows us to compare the behavior of reinforcement learningbased agents with planning agents based on tree search. We find that several of these games reliably deceive deep reinforcement learners, and that the resulting behavior highlights the shortcomings of the learning algorithm. The particular ways in which agents fail differ from how planning-based agents fail, further illuminating the character of these algorithms. We propose an initial typology of deceptions which could help us better understand pitfalls and failure modes of (deep) reinforcement learning.

Original languageEnglish
Pages (from-to)10-16
Number of pages7
JournalProceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment
Volume15
Issue number1
Publication statusPublished - 8 Oct 2019
Event15th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, AIIDE 2019 - Atlanta, United States
Duration: 8 Oct 201912 Oct 2019

Keywords

  • deep reinforcement learning
  • artificial intelligence
  • AI
  • Asynchronous Actor-Critic (A2C) algorithm

Fingerprint

Dive into the research topics of '"Superstition" in the network: Deep reinforcement learning plays deceptive games'. Together they form a unique fingerprint.

Cite this