Continuous-observation partially observable semi-Markov decision processes for machine maintenance

Mimi Zhang, Matthew Revie

Research output: Contribution to journalArticlepeer-review

12 Citations (Scopus)
191 Downloads (Pure)


Partially observable semi-Markov decision processes (POSMDPs) provide a rich framework for planning under both state transition uncertainty and observation uncertainty. In this paper, we widen the literature on POSMDP by studying discrete-state, discrete-action yet continuous-observation POSMDPs. We prove that the resultant α-vector set is continuous and therefore propose a point-based value iteration algorithm. This paper also bridges the gap between POSMDP and machine maintenance by incorporating various types of maintenance actions, such as actions changing machine state, actions changing degradation rate, and the temporally extended action "do nothing''. Both finite and infinite planning horizons are reviewed, and the solution methodology for each type of planning horizon is given. We illustrate the maintenance decision process via a real industrial problem and demonstrate that the developed framework can be readily applied to solve relevant maintenance problems.
Original languageEnglish
Pages (from-to)202-218
Number of pages17
JournalIEEE Transactions on Reliability
Issue number1
Early online date6 Dec 2016
Publication statusPublished - 31 Mar 2017


  • condition-based maintenance
  • degenerate distribution
  • imperfect maintenance
  • multi-state systems
  • point-based value iteration

Fingerprint Dive into the research topics of 'Continuous-observation partially observable semi-Markov decision processes for machine maintenance'. Together they form a unique fingerprint.

Cite this