Quantifying uncertainty in intervention effectiveness with structured expert judgement: an application to obstetric fistula

Abigail R. Colson, Sweta Adhikari, Ambereen Sleemi, Ramanan Laxminarayan

Research output: Contribution to journalArticle

Abstract

To demonstrate a new application of structured expert judgement to assess the effectiveness of surgery to correct obstetric fistula in a low-income setting. Intervention effectiveness is a major input of evidence-informed priority setting in healthcare, but information on intervention effectiveness is generally lacking. This is particularly problematic in the context of poorly resourced healthcare settings where even efficacious interventions fail to translate into improvements in health. The few intervention effectiveness studies related to obstetric fistula treatment focus on the experience of single facilities and do not consider the impact of multiple factors that may affect health outcomes. We use the classical model of structured expert judgement, a method that has been used to quantify uncertainty in the areas of engineering and environmental risk assessment when data are unavailable. Under this method, experts quantify their uncertainty about rates of long-term disability in patients with fistula following treatment in different contexts, but the information content drawn from their responses is statistically conditioned on the accuracy and informativeness of their responses to a set of calibration questions. Through this method, we develop best estimates and uncertainty bounds for the rate of disability associated with each treatment scenario and setting. Eight experts in obstetric fistula repair in low and middle income countries. Estimates developed using performance weights were statistically superior to those involving a simple averaging of expert responses. The performance-weight decision maker's assessments are narrower for 9 of the 10 calibration questions and 21 of 23 variables of interest. We find that structured expert judgement is a viable approach to investigating the effectiveness of medical interventions where randomised controlled trials are not possible. Understanding the effectiveness of surgery performed at different types of facilities can guide programme planning to increase access to fistula treatment.
LanguageEnglish
Article numbere007233
Number of pages8
JournalBMJ Open
Volume5
Issue number6
DOIs
Publication statusPublished - 3 Jun 2015

Fingerprint

Obstetrics
Uncertainty
Fistula
Calibration
Delivery of Health Care
Weights and Measures
Health
Therapeutics
Randomized Controlled Trials
Expert judgment
Healthcare
Surgery

Keywords

  • obstetric fistula
  • corrective surgery
  • expert judgement

Cite this

@article{0d40d963df9a402fb81892a23df7ceee,
title = "Quantifying uncertainty in intervention effectiveness with structured expert judgement: an application to obstetric fistula",
abstract = "To demonstrate a new application of structured expert judgement to assess the effectiveness of surgery to correct obstetric fistula in a low-income setting. Intervention effectiveness is a major input of evidence-informed priority setting in healthcare, but information on intervention effectiveness is generally lacking. This is particularly problematic in the context of poorly resourced healthcare settings where even efficacious interventions fail to translate into improvements in health. The few intervention effectiveness studies related to obstetric fistula treatment focus on the experience of single facilities and do not consider the impact of multiple factors that may affect health outcomes. We use the classical model of structured expert judgement, a method that has been used to quantify uncertainty in the areas of engineering and environmental risk assessment when data are unavailable. Under this method, experts quantify their uncertainty about rates of long-term disability in patients with fistula following treatment in different contexts, but the information content drawn from their responses is statistically conditioned on the accuracy and informativeness of their responses to a set of calibration questions. Through this method, we develop best estimates and uncertainty bounds for the rate of disability associated with each treatment scenario and setting. Eight experts in obstetric fistula repair in low and middle income countries. Estimates developed using performance weights were statistically superior to those involving a simple averaging of expert responses. The performance-weight decision maker's assessments are narrower for 9 of the 10 calibration questions and 21 of 23 variables of interest. We find that structured expert judgement is a viable approach to investigating the effectiveness of medical interventions where randomised controlled trials are not possible. Understanding the effectiveness of surgery performed at different types of facilities can guide programme planning to increase access to fistula treatment.",
keywords = "obstetric fistula, corrective surgery, expert judgement",
author = "Colson, {Abigail R.} and Sweta Adhikari and Ambereen Sleemi and Ramanan Laxminarayan",
year = "2015",
month = "6",
day = "3",
doi = "10.1136/bmjopen-2014-007233",
language = "English",
volume = "5",
journal = "BMJ Open",
issn = "2044-6055",
number = "6",

}

Quantifying uncertainty in intervention effectiveness with structured expert judgement : an application to obstetric fistula. / Colson, Abigail R.; Adhikari, Sweta; Sleemi, Ambereen; Laxminarayan, Ramanan.

In: BMJ Open, Vol. 5, No. 6, e007233, 03.06.2015.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Quantifying uncertainty in intervention effectiveness with structured expert judgement

T2 - BMJ Open

AU - Colson, Abigail R.

AU - Adhikari, Sweta

AU - Sleemi, Ambereen

AU - Laxminarayan, Ramanan

PY - 2015/6/3

Y1 - 2015/6/3

N2 - To demonstrate a new application of structured expert judgement to assess the effectiveness of surgery to correct obstetric fistula in a low-income setting. Intervention effectiveness is a major input of evidence-informed priority setting in healthcare, but information on intervention effectiveness is generally lacking. This is particularly problematic in the context of poorly resourced healthcare settings where even efficacious interventions fail to translate into improvements in health. The few intervention effectiveness studies related to obstetric fistula treatment focus on the experience of single facilities and do not consider the impact of multiple factors that may affect health outcomes. We use the classical model of structured expert judgement, a method that has been used to quantify uncertainty in the areas of engineering and environmental risk assessment when data are unavailable. Under this method, experts quantify their uncertainty about rates of long-term disability in patients with fistula following treatment in different contexts, but the information content drawn from their responses is statistically conditioned on the accuracy and informativeness of their responses to a set of calibration questions. Through this method, we develop best estimates and uncertainty bounds for the rate of disability associated with each treatment scenario and setting. Eight experts in obstetric fistula repair in low and middle income countries. Estimates developed using performance weights were statistically superior to those involving a simple averaging of expert responses. The performance-weight decision maker's assessments are narrower for 9 of the 10 calibration questions and 21 of 23 variables of interest. We find that structured expert judgement is a viable approach to investigating the effectiveness of medical interventions where randomised controlled trials are not possible. Understanding the effectiveness of surgery performed at different types of facilities can guide programme planning to increase access to fistula treatment.

AB - To demonstrate a new application of structured expert judgement to assess the effectiveness of surgery to correct obstetric fistula in a low-income setting. Intervention effectiveness is a major input of evidence-informed priority setting in healthcare, but information on intervention effectiveness is generally lacking. This is particularly problematic in the context of poorly resourced healthcare settings where even efficacious interventions fail to translate into improvements in health. The few intervention effectiveness studies related to obstetric fistula treatment focus on the experience of single facilities and do not consider the impact of multiple factors that may affect health outcomes. We use the classical model of structured expert judgement, a method that has been used to quantify uncertainty in the areas of engineering and environmental risk assessment when data are unavailable. Under this method, experts quantify their uncertainty about rates of long-term disability in patients with fistula following treatment in different contexts, but the information content drawn from their responses is statistically conditioned on the accuracy and informativeness of their responses to a set of calibration questions. Through this method, we develop best estimates and uncertainty bounds for the rate of disability associated with each treatment scenario and setting. Eight experts in obstetric fistula repair in low and middle income countries. Estimates developed using performance weights were statistically superior to those involving a simple averaging of expert responses. The performance-weight decision maker's assessments are narrower for 9 of the 10 calibration questions and 21 of 23 variables of interest. We find that structured expert judgement is a viable approach to investigating the effectiveness of medical interventions where randomised controlled trials are not possible. Understanding the effectiveness of surgery performed at different types of facilities can guide programme planning to increase access to fistula treatment.

KW - obstetric fistula

KW - corrective surgery

KW - expert judgement

UR - http://bmjopen.bmj.com/content/5/6/e007233

U2 - 10.1136/bmjopen-2014-007233

DO - 10.1136/bmjopen-2014-007233

M3 - Article

VL - 5

JO - BMJ Open

JF - BMJ Open

SN - 2044-6055

IS - 6

M1 - e007233

ER -