Discriminant content validity: a quantitative methodology for assessing content of theory-based measures, with illustrative applications

Marie A. Johnston, Diane L. Dixon, Jo Hart, Liz Glidewell, Carin D. Schröder, Beth S. Pollard

Research output: Contribution to journalArticle

30 Citations (Scopus)

Abstract

In studies involving theoretical constructs, it is important that measures have good content validity and that there is not contamination of measures by content from other constructs. While reliability and construct validity are routinely reported, to date, there has not been a satisfactory, transparent, and systematic method of assessing and reporting content validity. In this paper, we describe a methodology of discriminant content validity (DCV) and illustrate its application in three studies. Discriminant content validity involves six steps: construct definition, item selection, judge identification, judgement format, single-sample test of content validity, and assessment of discriminant items. In three studies, these steps were applied to a measure of illness perceptions (IPQ-R) and control cognitions. The IPQ-R performed well with most items being purely related to their target construct, although timeline and consequences had small problems. By contrast, the study of control cognitions identified problems in measuring constructs independently. In the final study, direct estimation response formats for theory of planned behaviour constructs were found to have as good DCV as Likert format. The DCV method allowed quantitative assessment of each item and can therefore inform the content validity of the measures assessed. The methods can be applied to assess content validity before or after collecting data to select the appropriate items to measure theoretical constructs. Further, the data reported for each item in Appendix S1 can be used in item or measure selection. Statement of contribution: What is already known on this subject? There are agreed methods of assessing and reporting construct validity of measures of theoretical constructs, but not their content validity. Content validity is rarely reported in a systematic and transparent manner. What does this study add? The paper proposes discriminant content validity (DCV), a systematic and transparent method of assessing and reporting whether items assess the intended theoretical construct and only that construct. In three studies, DCV was applied to measures of illness perceptions, control cognitions, and theory of planned behaviour response formats. Appendix S1 gives content validity indices for each item of each questionnaire investigated. Discriminant content validity is ideally applied while the measure is being developed, before using to measure the construct(s), but can also be applied after using a measure.
LanguageEnglish
Pages240-257
Number of pages18
JournalBritish Journal of Health Psychology
Volume19
Issue number2
Early online date15 Mar 2014
DOIs
Publication statusPublished - May 2014

Fingerprint

Cognition
Reproducibility of Results
Theoretical Models
TimeLine
Surveys and Questionnaires

Keywords

  • content validity
  • measurement
  • psychometrics
  • questionnaires
  • theory
  • validity

Cite this

Johnston, Marie A. ; Dixon, Diane L. ; Hart, Jo ; Glidewell, Liz ; Schröder, Carin D. ; Pollard, Beth S. / Discriminant content validity : a quantitative methodology for assessing content of theory-based measures, with illustrative applications. In: British Journal of Health Psychology. 2014 ; Vol. 19, No. 2. pp. 240-257.
@article{5d4a41d4bb644c61be49972047934942,
title = "Discriminant content validity: a quantitative methodology for assessing content of theory-based measures, with illustrative applications",
abstract = "In studies involving theoretical constructs, it is important that measures have good content validity and that there is not contamination of measures by content from other constructs. While reliability and construct validity are routinely reported, to date, there has not been a satisfactory, transparent, and systematic method of assessing and reporting content validity. In this paper, we describe a methodology of discriminant content validity (DCV) and illustrate its application in three studies. Discriminant content validity involves six steps: construct definition, item selection, judge identification, judgement format, single-sample test of content validity, and assessment of discriminant items. In three studies, these steps were applied to a measure of illness perceptions (IPQ-R) and control cognitions. The IPQ-R performed well with most items being purely related to their target construct, although timeline and consequences had small problems. By contrast, the study of control cognitions identified problems in measuring constructs independently. In the final study, direct estimation response formats for theory of planned behaviour constructs were found to have as good DCV as Likert format. The DCV method allowed quantitative assessment of each item and can therefore inform the content validity of the measures assessed. The methods can be applied to assess content validity before or after collecting data to select the appropriate items to measure theoretical constructs. Further, the data reported for each item in Appendix S1 can be used in item or measure selection. Statement of contribution: What is already known on this subject? There are agreed methods of assessing and reporting construct validity of measures of theoretical constructs, but not their content validity. Content validity is rarely reported in a systematic and transparent manner. What does this study add? The paper proposes discriminant content validity (DCV), a systematic and transparent method of assessing and reporting whether items assess the intended theoretical construct and only that construct. In three studies, DCV was applied to measures of illness perceptions, control cognitions, and theory of planned behaviour response formats. Appendix S1 gives content validity indices for each item of each questionnaire investigated. Discriminant content validity is ideally applied while the measure is being developed, before using to measure the construct(s), but can also be applied after using a measure.",
keywords = "content validity, measurement, psychometrics, questionnaires, theory, validity",
author = "Johnston, {Marie A.} and Dixon, {Diane L.} and Jo Hart and Liz Glidewell and Schr{\"o}der, {Carin D.} and Pollard, {Beth S.}",
year = "2014",
month = "5",
doi = "10.1111/bjhp.12095",
language = "English",
volume = "19",
pages = "240--257",
journal = "British Journal of Health Psychology",
issn = "1359-107X",
number = "2",

}

Discriminant content validity : a quantitative methodology for assessing content of theory-based measures, with illustrative applications. / Johnston, Marie A.; Dixon, Diane L.; Hart, Jo; Glidewell, Liz; Schröder, Carin D.; Pollard, Beth S.

In: British Journal of Health Psychology, Vol. 19, No. 2, 05.2014, p. 240-257.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Discriminant content validity

T2 - British Journal of Health Psychology

AU - Johnston, Marie A.

AU - Dixon, Diane L.

AU - Hart, Jo

AU - Glidewell, Liz

AU - Schröder, Carin D.

AU - Pollard, Beth S.

PY - 2014/5

Y1 - 2014/5

N2 - In studies involving theoretical constructs, it is important that measures have good content validity and that there is not contamination of measures by content from other constructs. While reliability and construct validity are routinely reported, to date, there has not been a satisfactory, transparent, and systematic method of assessing and reporting content validity. In this paper, we describe a methodology of discriminant content validity (DCV) and illustrate its application in three studies. Discriminant content validity involves six steps: construct definition, item selection, judge identification, judgement format, single-sample test of content validity, and assessment of discriminant items. In three studies, these steps were applied to a measure of illness perceptions (IPQ-R) and control cognitions. The IPQ-R performed well with most items being purely related to their target construct, although timeline and consequences had small problems. By contrast, the study of control cognitions identified problems in measuring constructs independently. In the final study, direct estimation response formats for theory of planned behaviour constructs were found to have as good DCV as Likert format. The DCV method allowed quantitative assessment of each item and can therefore inform the content validity of the measures assessed. The methods can be applied to assess content validity before or after collecting data to select the appropriate items to measure theoretical constructs. Further, the data reported for each item in Appendix S1 can be used in item or measure selection. Statement of contribution: What is already known on this subject? There are agreed methods of assessing and reporting construct validity of measures of theoretical constructs, but not their content validity. Content validity is rarely reported in a systematic and transparent manner. What does this study add? The paper proposes discriminant content validity (DCV), a systematic and transparent method of assessing and reporting whether items assess the intended theoretical construct and only that construct. In three studies, DCV was applied to measures of illness perceptions, control cognitions, and theory of planned behaviour response formats. Appendix S1 gives content validity indices for each item of each questionnaire investigated. Discriminant content validity is ideally applied while the measure is being developed, before using to measure the construct(s), but can also be applied after using a measure.

AB - In studies involving theoretical constructs, it is important that measures have good content validity and that there is not contamination of measures by content from other constructs. While reliability and construct validity are routinely reported, to date, there has not been a satisfactory, transparent, and systematic method of assessing and reporting content validity. In this paper, we describe a methodology of discriminant content validity (DCV) and illustrate its application in three studies. Discriminant content validity involves six steps: construct definition, item selection, judge identification, judgement format, single-sample test of content validity, and assessment of discriminant items. In three studies, these steps were applied to a measure of illness perceptions (IPQ-R) and control cognitions. The IPQ-R performed well with most items being purely related to their target construct, although timeline and consequences had small problems. By contrast, the study of control cognitions identified problems in measuring constructs independently. In the final study, direct estimation response formats for theory of planned behaviour constructs were found to have as good DCV as Likert format. The DCV method allowed quantitative assessment of each item and can therefore inform the content validity of the measures assessed. The methods can be applied to assess content validity before or after collecting data to select the appropriate items to measure theoretical constructs. Further, the data reported for each item in Appendix S1 can be used in item or measure selection. Statement of contribution: What is already known on this subject? There are agreed methods of assessing and reporting construct validity of measures of theoretical constructs, but not their content validity. Content validity is rarely reported in a systematic and transparent manner. What does this study add? The paper proposes discriminant content validity (DCV), a systematic and transparent method of assessing and reporting whether items assess the intended theoretical construct and only that construct. In three studies, DCV was applied to measures of illness perceptions, control cognitions, and theory of planned behaviour response formats. Appendix S1 gives content validity indices for each item of each questionnaire investigated. Discriminant content validity is ideally applied while the measure is being developed, before using to measure the construct(s), but can also be applied after using a measure.

KW - content validity

KW - measurement

KW - psychometrics

KW - questionnaires

KW - theory

KW - validity

UR - http://www.scopus.com/inward/record.url?scp=84895913874&partnerID=8YFLogxK

U2 - 10.1111/bjhp.12095

DO - 10.1111/bjhp.12095

M3 - Article

VL - 19

SP - 240

EP - 257

JO - British Journal of Health Psychology

JF - British Journal of Health Psychology

SN - 1359-107X

IS - 2

ER -