Power analysis for generalized linear mixed models in ecology and evolution

Paul C. D. Johnson, Sarah J. E. Barry, Heather M. Ferguson, Pie Müller

Research output: Contribution to journalArticle

59 Citations (Scopus)

Abstract

'Will my study answer my research question?' is the most fundamental question a researcher can ask when designing a study, yet when phrased in statistical terms - 'What is the power of my study?' or 'How precise will my parameter estimate be?' - few researchers in ecology and evolution (EE) try to answer it, despite the detrimental consequences of performing under- or over-powered research. We suggest that this reluctance is due in large part to the unsuitability of simple methods of power analysis (broadly defined as any attempt to quantify prospectively the 'informativeness' of a study) for the complex models commonly used in EE research. With the aim of encouraging the use of power analysis, we present simulation from generalized linear mixed models (GLMMs) as a flexible and accessible approach to power analysis that can account for random effects, overdispersion and diverse response distributions.We illustrate the benefits of simulation-based power analysis in two research scenarios: estimating the precision of a survey to estimate tick burdens on grouse chicks and estimating the power of a trial to compare the efficacy of insecticide-treated nets in malaria mosquito control. We provide a freely available R function, sim.glmm, for simulating from GLMMs.Analysis of simulated data revealed that the effects of accounting for realistic levels of random effects and overdispersion on power and precision estimates were substantial, with correspondingly severe implications for study design in the form of up to fivefold increases in sampling effort. We also show the utility of simulations for identifying scenarios where GLMM-fitting methods can perform poorly.These results illustrate the inadequacy of standard analytical power analysis methods and the flexibility of simulation-based power analysis for GLMMs. The wider use of these methods should contribute to improving the quality of study design in EE.

LanguageEnglish
Pages133-142
Number of pages10
JournalMethods in Ecology and Evolution
Volume6
Issue number2
Early online date6 Dec 2014
DOIs
Publication statusPublished - 1 Feb 2015

Fingerprint

ecology
researchers
experimental design
simulation
grouse
mosquito control
methodology
malaria
ticks
tick
insecticides
chicks
analysis
mosquito
insecticide
method
sampling
effect

Keywords

  • experimental design
  • sample size
  • precision
  • generalized linear model
  • random effects
  • simulation
  • overdispersion

Cite this

Johnson, Paul C. D. ; Barry, Sarah J. E. ; Ferguson, Heather M. ; Müller, Pie. / Power analysis for generalized linear mixed models in ecology and evolution. In: Methods in Ecology and Evolution. 2015 ; Vol. 6, No. 2. pp. 133-142.
@article{19c4afffc7204960b3c4b9e3f99443c8,
title = "Power analysis for generalized linear mixed models in ecology and evolution",
abstract = "'Will my study answer my research question?' is the most fundamental question a researcher can ask when designing a study, yet when phrased in statistical terms - 'What is the power of my study?' or 'How precise will my parameter estimate be?' - few researchers in ecology and evolution (EE) try to answer it, despite the detrimental consequences of performing under- or over-powered research. We suggest that this reluctance is due in large part to the unsuitability of simple methods of power analysis (broadly defined as any attempt to quantify prospectively the 'informativeness' of a study) for the complex models commonly used in EE research. With the aim of encouraging the use of power analysis, we present simulation from generalized linear mixed models (GLMMs) as a flexible and accessible approach to power analysis that can account for random effects, overdispersion and diverse response distributions.We illustrate the benefits of simulation-based power analysis in two research scenarios: estimating the precision of a survey to estimate tick burdens on grouse chicks and estimating the power of a trial to compare the efficacy of insecticide-treated nets in malaria mosquito control. We provide a freely available R function, sim.glmm, for simulating from GLMMs.Analysis of simulated data revealed that the effects of accounting for realistic levels of random effects and overdispersion on power and precision estimates were substantial, with correspondingly severe implications for study design in the form of up to fivefold increases in sampling effort. We also show the utility of simulations for identifying scenarios where GLMM-fitting methods can perform poorly.These results illustrate the inadequacy of standard analytical power analysis methods and the flexibility of simulation-based power analysis for GLMMs. The wider use of these methods should contribute to improving the quality of study design in EE.",
keywords = "experimental design, sample size, precision, generalized linear model, random effects, simulation, overdispersion",
author = "Johnson, {Paul C. D.} and Barry, {Sarah J. E.} and Ferguson, {Heather M.} and Pie M{\"u}ller",
year = "2015",
month = "2",
day = "1",
doi = "10.1111/2041-210X.12306",
language = "English",
volume = "6",
pages = "133--142",
journal = "Methods in Ecology and Evolution",
issn = "2041-210X",
number = "2",

}

Power analysis for generalized linear mixed models in ecology and evolution. / Johnson, Paul C. D.; Barry, Sarah J. E.; Ferguson, Heather M.; Müller, Pie.

In: Methods in Ecology and Evolution, Vol. 6, No. 2, 01.02.2015, p. 133-142.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Power analysis for generalized linear mixed models in ecology and evolution

AU - Johnson, Paul C. D.

AU - Barry, Sarah J. E.

AU - Ferguson, Heather M.

AU - Müller, Pie

PY - 2015/2/1

Y1 - 2015/2/1

N2 - 'Will my study answer my research question?' is the most fundamental question a researcher can ask when designing a study, yet when phrased in statistical terms - 'What is the power of my study?' or 'How precise will my parameter estimate be?' - few researchers in ecology and evolution (EE) try to answer it, despite the detrimental consequences of performing under- or over-powered research. We suggest that this reluctance is due in large part to the unsuitability of simple methods of power analysis (broadly defined as any attempt to quantify prospectively the 'informativeness' of a study) for the complex models commonly used in EE research. With the aim of encouraging the use of power analysis, we present simulation from generalized linear mixed models (GLMMs) as a flexible and accessible approach to power analysis that can account for random effects, overdispersion and diverse response distributions.We illustrate the benefits of simulation-based power analysis in two research scenarios: estimating the precision of a survey to estimate tick burdens on grouse chicks and estimating the power of a trial to compare the efficacy of insecticide-treated nets in malaria mosquito control. We provide a freely available R function, sim.glmm, for simulating from GLMMs.Analysis of simulated data revealed that the effects of accounting for realistic levels of random effects and overdispersion on power and precision estimates were substantial, with correspondingly severe implications for study design in the form of up to fivefold increases in sampling effort. We also show the utility of simulations for identifying scenarios where GLMM-fitting methods can perform poorly.These results illustrate the inadequacy of standard analytical power analysis methods and the flexibility of simulation-based power analysis for GLMMs. The wider use of these methods should contribute to improving the quality of study design in EE.

AB - 'Will my study answer my research question?' is the most fundamental question a researcher can ask when designing a study, yet when phrased in statistical terms - 'What is the power of my study?' or 'How precise will my parameter estimate be?' - few researchers in ecology and evolution (EE) try to answer it, despite the detrimental consequences of performing under- or over-powered research. We suggest that this reluctance is due in large part to the unsuitability of simple methods of power analysis (broadly defined as any attempt to quantify prospectively the 'informativeness' of a study) for the complex models commonly used in EE research. With the aim of encouraging the use of power analysis, we present simulation from generalized linear mixed models (GLMMs) as a flexible and accessible approach to power analysis that can account for random effects, overdispersion and diverse response distributions.We illustrate the benefits of simulation-based power analysis in two research scenarios: estimating the precision of a survey to estimate tick burdens on grouse chicks and estimating the power of a trial to compare the efficacy of insecticide-treated nets in malaria mosquito control. We provide a freely available R function, sim.glmm, for simulating from GLMMs.Analysis of simulated data revealed that the effects of accounting for realistic levels of random effects and overdispersion on power and precision estimates were substantial, with correspondingly severe implications for study design in the form of up to fivefold increases in sampling effort. We also show the utility of simulations for identifying scenarios where GLMM-fitting methods can perform poorly.These results illustrate the inadequacy of standard analytical power analysis methods and the flexibility of simulation-based power analysis for GLMMs. The wider use of these methods should contribute to improving the quality of study design in EE.

KW - experimental design

KW - sample size

KW - precision

KW - generalized linear model

KW - random effects

KW - simulation

KW - overdispersion

U2 - 10.1111/2041-210X.12306

DO - 10.1111/2041-210X.12306

M3 - Article

VL - 6

SP - 133

EP - 142

JO - Methods in Ecology and Evolution

T2 - Methods in Ecology and Evolution

JF - Methods in Ecology and Evolution

SN - 2041-210X

IS - 2

ER -