Fast-tracking stationary MOMDPs for adaptive management problems

Martin Peron, Peter Bartlett, Kai Helge Becker, Iadine Chades

Research output: Chapter in Book/Report/Conference proceedingConference contribution book

15 Downloads (Pure)

Abstract

Adaptive management is applied in conservation and natural resource management, and consists of making sequential decisions when the transition matrix is uncertain. Informally described as ’learning by doing’, this approach aims to trade off between decisions that help achieve the objective and decisions that will yield a better knowledge of the true transition matrix. When the true transition matrix is assumed to be an element of a finite set of possible matrices, solving a mixed observability Markov decision process (MOMDP) leads to an optimal trade-off but is very computationally demanding. Under the assumption (common in adaptive management) that the true transition matrix is stationary, we propose a polynomial-time algorithm to find a lower bound of the value function. In the corners of the domain of the value function (belief space), this lower bound is provably equal to the optimal value function. We also show that under further assumptions, it is a linear approximation of the optimal value function in a neighborhood around the corners. We evaluate the benefits of our approach by using it to initialize the solvers MO-SARSOP and Perseus on a novel computational sustainability problem and a recent adaptive management data challenge. Our approach leads to an improved initial value function and translates into significant computational gains for both solvers.
Original languageEnglish
Title of host publicationProceedings of the Thirty-First AAAI Conference on Artificial Intelligence
Place of PublicationPalo Alto
Number of pages8
Publication statusAccepted/In press - 12 Nov 2016
EventThirty-First AAAI Conference on Artificial Intelligence - Hilton Hotel, San Francisco, United States
Duration: 4 Feb 20179 Feb 2017
http://www.aaai.org/Conferences/AAAI/aaai17.php

Publication series

NameProceedings of the AAAI Conference on Artificial Intelligence
PublisherAssociation for the Advancement of Artificial Intelligence
ISSN (Print)2159-5399

Conference

ConferenceThirty-First AAAI Conference on Artificial Intelligence
Abbreviated titleAAAI-17
CountryUnited States
CitySan Francisco
Period4/02/179/02/17
Internet address

Keywords

  • adaptive management
  • Markov decision processes
  • optimal trade-offs
  • polynomial-time algorithm
  • value function
  • mixed observability

Fingerprint Dive into the research topics of 'Fast-tracking stationary MOMDPs for adaptive management problems'. Together they form a unique fingerprint.

Cite this