Retrospective time series analysis of veterinary laboratory data: Preparing a historical baseline for cluster detection in syndromic surveillance

Fernanda C. Dórea, Crawford W. Revie, Beverly J. McEwen, W. Bruce McNab, David Kelton, Javier Sanchez

Research output: Contribution to journalArticle

12 Citations (Scopus)

Abstract

The practice of disease surveillance has shifted in the last two decades towards the introduction of systems capable of early detection of disease. Modern biosurveillance systems explore different sources of pre-diagnostic data, such as patient's chief complaint upon emergency visit or laboratory test orders. These sources of data can provide more rapid detection than traditional surveillance based on case confirmation, but are less specific, and therefore their use poses challenges related to the presence of background noise and unlabelled temporal aberrations in historical data. The overall goal of this study was to carry out retrospective analysis using three years of laboratory test submissions to the Animal Health Laboratory in the province of Ontario, Canada, in order to prepare the data for use in syndromic surveillance. Daily cases were grouped into syndromes and counts for each syndrome were monitored on a daily basis when medians were higher than one case per day, and weekly otherwise. Poisson regression accounting for day-of-week and month was able to capture the day-of-week effect with minimal influence from temporal aberrations. Applying Poisson regression in an iterative manner, that removed data points above the predicted 95th percentile of daily counts, allowed for the removal of these aberrations in the absence of labelled outbreaks, while maintaining the day-of-week effect that was present in the original data. This resulted in the construction of time series that represent the baseline patterns over the past three years, free of temporal aberrations. The final method was thus able to remove temporal aberrations while keeping the original explainable effects in the data, did not need a training period free of aberrations, had minimal adjustment to the aberrations present in the raw data, and did not require labelled outbreaks. Moreover, it was readily applicable to the weekly data by substituting Poisson regression with moving 95th percentiles.

LanguageEnglish
Pages219-227
Number of pages9
JournalPreventive Veterinary Medicine
Volume109
Issue number3-4
Early online date12 Nov 2012
DOIs
Publication statusPublished - 1 May 2013

Fingerprint

time series analysis
disease detection
disease surveillance
monitoring
animal health
Ontario
Canada
Disease Outbreaks
Biosurveillance
Information Storage and Retrieval
Laboratory Animals
Noise
Early Diagnosis
Emergencies
laboratory experimentation
methodology
Health

Keywords

  • animal health surveillance
  • disease trends
  • laboratory
  • retrospective analysis
  • syndromic surveillance
  • time series analysis

Cite this

Dórea, Fernanda C. ; Revie, Crawford W. ; McEwen, Beverly J. ; McNab, W. Bruce ; Kelton, David ; Sanchez, Javier. / Retrospective time series analysis of veterinary laboratory data : Preparing a historical baseline for cluster detection in syndromic surveillance. In: Preventive Veterinary Medicine. 2013 ; Vol. 109, No. 3-4. pp. 219-227.
@article{cb3ecf77fa41489fa212b9c441c866a2,
title = "Retrospective time series analysis of veterinary laboratory data: Preparing a historical baseline for cluster detection in syndromic surveillance",
abstract = "The practice of disease surveillance has shifted in the last two decades towards the introduction of systems capable of early detection of disease. Modern biosurveillance systems explore different sources of pre-diagnostic data, such as patient's chief complaint upon emergency visit or laboratory test orders. These sources of data can provide more rapid detection than traditional surveillance based on case confirmation, but are less specific, and therefore their use poses challenges related to the presence of background noise and unlabelled temporal aberrations in historical data. The overall goal of this study was to carry out retrospective analysis using three years of laboratory test submissions to the Animal Health Laboratory in the province of Ontario, Canada, in order to prepare the data for use in syndromic surveillance. Daily cases were grouped into syndromes and counts for each syndrome were monitored on a daily basis when medians were higher than one case per day, and weekly otherwise. Poisson regression accounting for day-of-week and month was able to capture the day-of-week effect with minimal influence from temporal aberrations. Applying Poisson regression in an iterative manner, that removed data points above the predicted 95th percentile of daily counts, allowed for the removal of these aberrations in the absence of labelled outbreaks, while maintaining the day-of-week effect that was present in the original data. This resulted in the construction of time series that represent the baseline patterns over the past three years, free of temporal aberrations. The final method was thus able to remove temporal aberrations while keeping the original explainable effects in the data, did not need a training period free of aberrations, had minimal adjustment to the aberrations present in the raw data, and did not require labelled outbreaks. Moreover, it was readily applicable to the weekly data by substituting Poisson regression with moving 95th percentiles.",
keywords = "animal health surveillance, disease trends, laboratory, retrospective analysis, syndromic surveillance, time series analysis",
author = "D{\'o}rea, {Fernanda C.} and Revie, {Crawford W.} and McEwen, {Beverly J.} and McNab, {W. Bruce} and David Kelton and Javier Sanchez",
year = "2013",
month = "5",
day = "1",
doi = "10.1016/j.prevetmed.2012.10.010",
language = "English",
volume = "109",
pages = "219--227",
journal = "Preventive Veterinary Medicine",
issn = "0167-5877",
number = "3-4",

}

Retrospective time series analysis of veterinary laboratory data : Preparing a historical baseline for cluster detection in syndromic surveillance. / Dórea, Fernanda C.; Revie, Crawford W.; McEwen, Beverly J.; McNab, W. Bruce; Kelton, David; Sanchez, Javier.

In: Preventive Veterinary Medicine, Vol. 109, No. 3-4, 01.05.2013, p. 219-227.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Retrospective time series analysis of veterinary laboratory data

T2 - Preventive Veterinary Medicine

AU - Dórea, Fernanda C.

AU - Revie, Crawford W.

AU - McEwen, Beverly J.

AU - McNab, W. Bruce

AU - Kelton, David

AU - Sanchez, Javier

PY - 2013/5/1

Y1 - 2013/5/1

N2 - The practice of disease surveillance has shifted in the last two decades towards the introduction of systems capable of early detection of disease. Modern biosurveillance systems explore different sources of pre-diagnostic data, such as patient's chief complaint upon emergency visit or laboratory test orders. These sources of data can provide more rapid detection than traditional surveillance based on case confirmation, but are less specific, and therefore their use poses challenges related to the presence of background noise and unlabelled temporal aberrations in historical data. The overall goal of this study was to carry out retrospective analysis using three years of laboratory test submissions to the Animal Health Laboratory in the province of Ontario, Canada, in order to prepare the data for use in syndromic surveillance. Daily cases were grouped into syndromes and counts for each syndrome were monitored on a daily basis when medians were higher than one case per day, and weekly otherwise. Poisson regression accounting for day-of-week and month was able to capture the day-of-week effect with minimal influence from temporal aberrations. Applying Poisson regression in an iterative manner, that removed data points above the predicted 95th percentile of daily counts, allowed for the removal of these aberrations in the absence of labelled outbreaks, while maintaining the day-of-week effect that was present in the original data. This resulted in the construction of time series that represent the baseline patterns over the past three years, free of temporal aberrations. The final method was thus able to remove temporal aberrations while keeping the original explainable effects in the data, did not need a training period free of aberrations, had minimal adjustment to the aberrations present in the raw data, and did not require labelled outbreaks. Moreover, it was readily applicable to the weekly data by substituting Poisson regression with moving 95th percentiles.

AB - The practice of disease surveillance has shifted in the last two decades towards the introduction of systems capable of early detection of disease. Modern biosurveillance systems explore different sources of pre-diagnostic data, such as patient's chief complaint upon emergency visit or laboratory test orders. These sources of data can provide more rapid detection than traditional surveillance based on case confirmation, but are less specific, and therefore their use poses challenges related to the presence of background noise and unlabelled temporal aberrations in historical data. The overall goal of this study was to carry out retrospective analysis using three years of laboratory test submissions to the Animal Health Laboratory in the province of Ontario, Canada, in order to prepare the data for use in syndromic surveillance. Daily cases were grouped into syndromes and counts for each syndrome were monitored on a daily basis when medians were higher than one case per day, and weekly otherwise. Poisson regression accounting for day-of-week and month was able to capture the day-of-week effect with minimal influence from temporal aberrations. Applying Poisson regression in an iterative manner, that removed data points above the predicted 95th percentile of daily counts, allowed for the removal of these aberrations in the absence of labelled outbreaks, while maintaining the day-of-week effect that was present in the original data. This resulted in the construction of time series that represent the baseline patterns over the past three years, free of temporal aberrations. The final method was thus able to remove temporal aberrations while keeping the original explainable effects in the data, did not need a training period free of aberrations, had minimal adjustment to the aberrations present in the raw data, and did not require labelled outbreaks. Moreover, it was readily applicable to the weekly data by substituting Poisson regression with moving 95th percentiles.

KW - animal health surveillance

KW - disease trends

KW - laboratory

KW - retrospective analysis

KW - syndromic surveillance

KW - time series analysis

UR - http://www.scopus.com/inward/record.url?scp=84875407575&partnerID=8YFLogxK

U2 - 10.1016/j.prevetmed.2012.10.010

DO - 10.1016/j.prevetmed.2012.10.010

M3 - Article

VL - 109

SP - 219

EP - 227

JO - Preventive Veterinary Medicine

JF - Preventive Veterinary Medicine

SN - 0167-5877

IS - 3-4

ER -