Table 4.
Elements of a comprehensive research question of a study on reliability or measurement error
| Element of the research question | Consensusa (%) | |
|---|---|---|
| 1 | the name of the outcome measurement instrument | 42/45 (93%) (R1b) |
| 2 | the version of the outcome measurement instrument or way of operationalization of the measurement protocolc |
42/45 (93%) (R1) (version) 33/45 (73%) (R1) (operationalization) |
| 3 | the construct measured by the measurement instrument | 40/45 (89%) (R1) |
| 4 | a specification whether one is interested in a reliability parameter (i.e. a relative parameter such as an ICC, Generalizability coefficient φ, or Kappa κ) or a parameter of measurement error (i.e. an absolute parameter expressed in the unit of measurement e.g. SEM, LoA or SDC; or expressed as agreement or misclassification, e.g. the percentage specific agreement). | 36/42 (86%) (R2d) |
| 5 | a specification of the components of the measurement instrument that will be repeated (especially when only part of the measurement instrument is repeated, e.g. only assignment of the score based on the same images) | 38/45 (84%) (R1) |
| 6 | a specification of the source(s) of variation that will be variede | 41/45 (91%) (R1) |
| 7 | a specification of the patientf populationg studied | 42/45 (93%) (R1) |
a Consensus was set at 67% of the panelists (strongly) agreed to a proposal, the denominator can be decreased because panelists considered themselves to have ‘no expertise’ on a specific proposal or dropped-out; b R1: consensus reached in Round 1; c In Generalizability theory these are the facets of stratification (FoS), when patients are nested in a facet [16]; d R2: consensus reached in Round 2; e In Generalizability theory these are the random or fixed facets of generalizability (FoG), e.g. time or occasion, the (level of expertise of) professionals, the machines, or other components of the measurement [16]; f In the COSMIN methodology we use the word patient. However, sometimes the target population doesn’t consist of patients, but e.g. healthy individuals, caregivers, clinicians, or the body structures (e.g. joints, or lesions). In these cases, the word patient should be read as e.g. healthy volunteer, or clinician; g In Generalizability theory these are the Object of Measurement (OoM) or the facet of differentiation [16]