Predictive validity and interrater reliability of the FACE-CARAS toolkit in a CAMHS setting

Stephen A. Evans, David Young, Paul A. Tiffin

Research output: Contribution to journalArticle

Abstract

Background: The FACE-CARAS (Functional Analysis in Care Environments-Child and Adolescent Risk-Assessment Suite) toolkit has been developed to support practitioners in Child and Adolescent Mental Health Services (CAMHS) in performing a structured risk assessment. It covers a number of risk domains including violence, suicide, self-harm, experienced abuse, and exploitation. Interrater and internal reliability has already been established but not predictive validity.
Aims/Hypothesis: Our aim was to establish the predictive validity of the FACE-CARAS in a CAMHS population.
Methods: Records from 123 young people with FACE-CARAS ratings completed by clinicians were examined in a retrospective file review to extract data on a relevant list of adverse outcomes at three and at 6 months following the assessment. Although this was not a prospective longitudinal study, researchers were blind to the clinicians' ratings, allowing valid testing of predictive power. Cases were drawn from across generic and specialist CAMHS teams in approximately equal proportions. Data were analysed using receiver operator characteristic statistics.
Results: Areas under the curve values in five of the seven risk areas approached or were greater than 0.8 indicated that the FACE-CARAS profile score was a good potential predictor of risks of self-harm, suicidal behaviours, serious self-neglect, abuse or exploitation by others, and violence to others at both 3 and 6 months. It was weakly "predictive" of accidental self-harm and no better than chance at signalling physical ill health.
Conclusions: Findings support the use of the "profile summary" section of the tool as likely to generate clinically useful risk predictions. We were concerned that clinical use of the scale did not conform to research standards and often left subscales incompletely rated; however, the fact that the tool nonetheless proved a good predictor of most key adversities under scrutiny may add weight to its value in clinical practice. Further work with the FACE-CARAS subscales is recommended.
LanguageEnglish
Pages47-56
Number of pages10
JournalCriminal Behaviour and Mental Health
Volume29
Issue number1
Early online date3 Jan 2019
DOIs
Publication statusPublished - 1 Feb 2019

Fingerprint

Adolescent Health Services
Mental Health Services
Child Care
Reproducibility of Results
Violence
Suicide
Area Under Curve
Longitudinal Studies
Research Personnel
Prospective Studies
Weights and Measures
Health
Research
Population

Keywords

  • CAMHS risk assessment
  • FACE-CARAS
  • risk assessment

Cite this

@article{f04db6600f6643da934c5aee9c7d4468,
title = "Predictive validity and interrater reliability of the FACE-CARAS toolkit in a CAMHS setting",
abstract = "Background: The FACE-CARAS (Functional Analysis in Care Environments-Child and Adolescent Risk-Assessment Suite) toolkit has been developed to support practitioners in Child and Adolescent Mental Health Services (CAMHS) in performing a structured risk assessment. It covers a number of risk domains including violence, suicide, self-harm, experienced abuse, and exploitation. Interrater and internal reliability has already been established but not predictive validity.Aims/Hypothesis: Our aim was to establish the predictive validity of the FACE-CARAS in a CAMHS population.Methods: Records from 123 young people with FACE-CARAS ratings completed by clinicians were examined in a retrospective file review to extract data on a relevant list of adverse outcomes at three and at 6 months following the assessment. Although this was not a prospective longitudinal study, researchers were blind to the clinicians' ratings, allowing valid testing of predictive power. Cases were drawn from across generic and specialist CAMHS teams in approximately equal proportions. Data were analysed using receiver operator characteristic statistics.Results: Areas under the curve values in five of the seven risk areas approached or were greater than 0.8 indicated that the FACE-CARAS profile score was a good potential predictor of risks of self-harm, suicidal behaviours, serious self-neglect, abuse or exploitation by others, and violence to others at both 3 and 6 months. It was weakly {"}predictive{"} of accidental self-harm and no better than chance at signalling physical ill health.Conclusions: Findings support the use of the {"}profile summary{"} section of the tool as likely to generate clinically useful risk predictions. We were concerned that clinical use of the scale did not conform to research standards and often left subscales incompletely rated; however, the fact that the tool nonetheless proved a good predictor of most key adversities under scrutiny may add weight to its value in clinical practice. Further work with the FACE-CARAS subscales is recommended.",
keywords = "CAMHS risk assessment, FACE-CARAS, risk assessment",
author = "Evans, {Stephen A.} and David Young and Tiffin, {Paul A.}",
note = "This is the peer reviewed version of the following article: Evans, S. A., Young, D., & Tiffin, P. A. (2019). Predictive validity and interrater reliability of the FACE-CARAS toolkit in a CAMHS setting. Criminal Behaviour and Mental Health, 29(1), 47-56., which has been published in final form at https://doi.org/10.1002/cbm.2104. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Use of Self-Archived Versions.",
year = "2019",
month = "2",
day = "1",
doi = "10.1002/cbm.2104",
language = "English",
volume = "29",
pages = "47--56",
journal = "Criminal Behaviour and Mental Health",
issn = "0957-9664",
number = "1",

}

Predictive validity and interrater reliability of the FACE-CARAS toolkit in a CAMHS setting. / Evans, Stephen A.; Young, David; Tiffin, Paul A.

In: Criminal Behaviour and Mental Health, Vol. 29, No. 1, 01.02.2019, p. 47-56.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Predictive validity and interrater reliability of the FACE-CARAS toolkit in a CAMHS setting

AU - Evans, Stephen A.

AU - Young, David

AU - Tiffin, Paul A.

N1 - This is the peer reviewed version of the following article: Evans, S. A., Young, D., & Tiffin, P. A. (2019). Predictive validity and interrater reliability of the FACE-CARAS toolkit in a CAMHS setting. Criminal Behaviour and Mental Health, 29(1), 47-56., which has been published in final form at https://doi.org/10.1002/cbm.2104. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Use of Self-Archived Versions.

PY - 2019/2/1

Y1 - 2019/2/1

N2 - Background: The FACE-CARAS (Functional Analysis in Care Environments-Child and Adolescent Risk-Assessment Suite) toolkit has been developed to support practitioners in Child and Adolescent Mental Health Services (CAMHS) in performing a structured risk assessment. It covers a number of risk domains including violence, suicide, self-harm, experienced abuse, and exploitation. Interrater and internal reliability has already been established but not predictive validity.Aims/Hypothesis: Our aim was to establish the predictive validity of the FACE-CARAS in a CAMHS population.Methods: Records from 123 young people with FACE-CARAS ratings completed by clinicians were examined in a retrospective file review to extract data on a relevant list of adverse outcomes at three and at 6 months following the assessment. Although this was not a prospective longitudinal study, researchers were blind to the clinicians' ratings, allowing valid testing of predictive power. Cases were drawn from across generic and specialist CAMHS teams in approximately equal proportions. Data were analysed using receiver operator characteristic statistics.Results: Areas under the curve values in five of the seven risk areas approached or were greater than 0.8 indicated that the FACE-CARAS profile score was a good potential predictor of risks of self-harm, suicidal behaviours, serious self-neglect, abuse or exploitation by others, and violence to others at both 3 and 6 months. It was weakly "predictive" of accidental self-harm and no better than chance at signalling physical ill health.Conclusions: Findings support the use of the "profile summary" section of the tool as likely to generate clinically useful risk predictions. We were concerned that clinical use of the scale did not conform to research standards and often left subscales incompletely rated; however, the fact that the tool nonetheless proved a good predictor of most key adversities under scrutiny may add weight to its value in clinical practice. Further work with the FACE-CARAS subscales is recommended.

AB - Background: The FACE-CARAS (Functional Analysis in Care Environments-Child and Adolescent Risk-Assessment Suite) toolkit has been developed to support practitioners in Child and Adolescent Mental Health Services (CAMHS) in performing a structured risk assessment. It covers a number of risk domains including violence, suicide, self-harm, experienced abuse, and exploitation. Interrater and internal reliability has already been established but not predictive validity.Aims/Hypothesis: Our aim was to establish the predictive validity of the FACE-CARAS in a CAMHS population.Methods: Records from 123 young people with FACE-CARAS ratings completed by clinicians were examined in a retrospective file review to extract data on a relevant list of adverse outcomes at three and at 6 months following the assessment. Although this was not a prospective longitudinal study, researchers were blind to the clinicians' ratings, allowing valid testing of predictive power. Cases were drawn from across generic and specialist CAMHS teams in approximately equal proportions. Data were analysed using receiver operator characteristic statistics.Results: Areas under the curve values in five of the seven risk areas approached or were greater than 0.8 indicated that the FACE-CARAS profile score was a good potential predictor of risks of self-harm, suicidal behaviours, serious self-neglect, abuse or exploitation by others, and violence to others at both 3 and 6 months. It was weakly "predictive" of accidental self-harm and no better than chance at signalling physical ill health.Conclusions: Findings support the use of the "profile summary" section of the tool as likely to generate clinically useful risk predictions. We were concerned that clinical use of the scale did not conform to research standards and often left subscales incompletely rated; however, the fact that the tool nonetheless proved a good predictor of most key adversities under scrutiny may add weight to its value in clinical practice. Further work with the FACE-CARAS subscales is recommended.

KW - CAMHS risk assessment

KW - FACE-CARAS

KW - risk assessment

U2 - 10.1002/cbm.2104

DO - 10.1002/cbm.2104

M3 - Article

VL - 29

SP - 47

EP - 56

JO - Criminal Behaviour and Mental Health

T2 - Criminal Behaviour and Mental Health

JF - Criminal Behaviour and Mental Health

SN - 0957-9664

IS - 1

ER -