### Abstract

Language | English |
---|---|

Title of host publication | Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence |

Place of Publication | Palo Alto |

Number of pages | 8 |

Publication status | Accepted/In press - 12 Nov 2016 |

Event | Thirty-First AAAI Conference on Artificial Intelligence - Hilton Hotel, San Francisco, United States Duration: 4 Feb 2017 → 9 Feb 2017 http://www.aaai.org/Conferences/AAAI/aaai17.php |

### Publication series

Name | Proceedings of the AAAI Conference on Artificial Intelligence |
---|---|

Publisher | Association for the Advancement of Artificial Intelligence |

ISSN (Print) | 2159-5399 |

### Conference

Conference | Thirty-First AAAI Conference on Artificial Intelligence |
---|---|

Abbreviated title | AAAI-17 |

Country | United States |

City | San Francisco |

Period | 4/02/17 → 9/02/17 |

Internet address |

### Fingerprint

### Keywords

- adaptive management
- Markov decision processes
- optimal trade-offs
- polynomial-time algorithm
- value function
- mixed observability

### Cite this

*Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence*(Proceedings of the AAAI Conference on Artificial Intelligence). Palo Alto.

}

*Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence.*Proceedings of the AAAI Conference on Artificial Intelligence, Palo Alto, Thirty-First AAAI Conference on Artificial Intelligence, San Francisco, United States, 4/02/17.

**Fast-tracking stationary MOMDPs for adaptive management problems.** / Peron, Martin; Bartlett, Peter; Becker, Kai Helge; Chades, Iadine.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution book

TY - GEN

T1 - Fast-tracking stationary MOMDPs for adaptive management problems

AU - Peron, Martin

AU - Bartlett, Peter

AU - Becker, Kai Helge

AU - Chades, Iadine

PY - 2016/11/12

Y1 - 2016/11/12

N2 - Adaptive management is applied in conservation and natural resource management, and consists of making sequential decisions when the transition matrix is uncertain. Informally described as ’learning by doing’, this approach aims to trade off between decisions that help achieve the objective and decisions that will yield a better knowledge of the true transition matrix. When the true transition matrix is assumed to be an element of a finite set of possible matrices, solving a mixed observability Markov decision process (MOMDP) leads to an optimal trade-off but is very computationally demanding. Under the assumption (common in adaptive management) that the true transition matrix is stationary, we propose a polynomial-time algorithm to find a lower bound of the value function. In the corners of the domain of the value function (belief space), this lower bound is provably equal to the optimal value function. We also show that under further assumptions, it is a linear approximation of the optimal value function in a neighborhood around the corners. We evaluate the benefits of our approach by using it to initialize the solvers MO-SARSOP and Perseus on a novel computational sustainability problem and a recent adaptive management data challenge. Our approach leads to an improved initial value function and translates into significant computational gains for both solvers.

AB - Adaptive management is applied in conservation and natural resource management, and consists of making sequential decisions when the transition matrix is uncertain. Informally described as ’learning by doing’, this approach aims to trade off between decisions that help achieve the objective and decisions that will yield a better knowledge of the true transition matrix. When the true transition matrix is assumed to be an element of a finite set of possible matrices, solving a mixed observability Markov decision process (MOMDP) leads to an optimal trade-off but is very computationally demanding. Under the assumption (common in adaptive management) that the true transition matrix is stationary, we propose a polynomial-time algorithm to find a lower bound of the value function. In the corners of the domain of the value function (belief space), this lower bound is provably equal to the optimal value function. We also show that under further assumptions, it is a linear approximation of the optimal value function in a neighborhood around the corners. We evaluate the benefits of our approach by using it to initialize the solvers MO-SARSOP and Perseus on a novel computational sustainability problem and a recent adaptive management data challenge. Our approach leads to an improved initial value function and translates into significant computational gains for both solvers.

KW - adaptive management

KW - Markov decision processes

KW - optimal trade-offs

KW - polynomial-time algorithm

KW - value function

KW - mixed observability

UR - http://www.aaai.org/Conferences/AAAI/aaai17.php

UR - http://www.aaai.org/Press/Proceedings/proceedings.php

M3 - Conference contribution book

T3 - Proceedings of the AAAI Conference on Artificial Intelligence

BT - Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence

CY - Palo Alto

ER -