Abstract
Background
The International Consortium for Health Outcomes Measurement (ICHOM) develops condition-specific Standard Sets of outcomes to be measured in clinical practice for value-based healthcare evaluation. Standard Sets are developed by different working groups, which is inefficient and may lead to inconsistencies in selected PROs and PROMs. We aimed to identify common PROs across ICHOM Standard Sets and examined to what extend these PROs can be measured with a generic set of PROMs: the Patient-Reported Outcomes Measurement Information System (PROMIS®).
Methods
We extracted all PROs and recommended PROMs from 39 ICHOM Standard Sets. Similar PROs were categorized into unique PRO concepts. We examined which of these PRO concepts can be measured with PROMIS.
Results
A total of 307 PROs were identified in 39 ICHOM Standard Sets and 114 unique PROMs are recommended for measuring these PROs. The 307 PROs could be categorized into 22 unique PRO concepts. More than half (17/22) of these PRO concepts (covering about 75% of the PROs and 75% of the PROMs) can be measured with a PROMIS measure.
Conclusion
Considerable overlap was found in PROs across ICHOM Standard Sets, and large differences in terminology used and PROMs recommended, even for the same PROs. We recommend a more universal and standardized approach to the selection of PROs and PROMs. Such an approach, focusing on a set of core PROs for all patients, measured with a system like PROMIS, may provide more opportunities for patient-centered care and facilitate the uptake of Standard Sets in clinical practice.
Keywords: Patient-reported outcomes, Outcome measurement, Healthcare evaluation
Background
Measuring outcomes that matter to patients—including Patient-Reported Outcomes (PROs, Table 1)—can help healthcare providers to benchmark treatment effects against their peers and, by identifying and learning from best practices, provide opportunities to improve quality of care [1–3].
Table 1.
Glossary of terms
| Abbreviation | Full text | Definition | Examples |
|---|---|---|---|
| PRO | Patient-reported outcome | Any report of the status of a patient’s health condition that comes directly from the patient, without interpretation of the patient’s response by a clinician or anyone else [61] |
Depression Pain |
| PROM | Patient-reported outcome measure | A questionnaire used to measure the PRO |
Patient Health Questionnaire (PHQ-9) Visual Analogue Scale (VAS) for pain |
| PROMIS | Patient-reported outcomes measurement information system | A set of generic PROMs that can be used as short forms or CAT to measure aspects of physical, mental, and social health in adults and children across medical conditions | www.healthmeasures.net/promis |
| PROMIS measure | Patient-reported outcomes measurement information system measure | A PROM (short form or CAT) from the PROMIS system |
PROMIS® Item Bank V1.0 Depression-Short Form 8a PROMIS® Item Bank V1.1 Pain Interference CAT |
CAT Computerized Adaptive Test, where a computer algorithm consecutively selects questions from a large set of questions (item bank), based on responses to previous questions
In 2012, the International Consortium for Health Outcomes Measurement (ICHOM) was founded with the aim to define Standard Sets of outcomes for every major medical condition [4]. Standard Sets are minimum sets of outcomes that matter most to patients. These sets should be measured and reported in all patients with a specific condition (or disease). Standard Sets have been developed using a standardized consensus methodology among a team of experts and patient representatives in the field [5]. Up to May 2021, 39 Standard Sets were published and another five were in progress [4, 6–35].
A potential barrier for implementing ICHOM Standard Sets is that they are independently developed by different working groups. Although it is important that Standard Sets are developed by people who have expertise in the particular condition, collaboration and harmonization across Standard Sets is currently limited, which leads to large differences and inconsistencies in selected PROs, terminology used, and recommended Patient-Reported Outcome Measures (PROMs), even for the same PROs [4, 6–17]. This complicates the implementation and use of Standard Sets in clinical practice.
A system of common data elements across conditions could improve the situation and may speed up the development and uptake of Standard Sets considerably. Generally, all people want to feel and function ‘normally’, i.e., live without symptoms, such as pain, fatigue or depression, and be able to carry out daily activities and social roles. These feelings and functions can be affected by different health conditions. For example, climbing stairs can be affected by knee osteoarthritis (e.g. because of pain), lung disease (e.g. because of breathlessness) of heart failure (e.g. because of fatigue). Different conditions can result in the same patient-reported problem; in this case, difficulty climbing stairs. Because human values, experiences, and desires cut across health status, there is considerable overlap in relevant PROs across conditions [36].
Ideally, these non-condition specific (or common) outcomes could be measured with just one set of universal (i.e. generic) PROMs across conditions. This is, however, currently not the case. A large number of different PROMs is being used for measuring common outcomes within and between different patient groups. One reason for this is that it has been considered important to develop disease-specific PROMs, while for common outcomes like pain or fatigue, this may not be necessary. Another reason is that many PROMs were developed because of criticism on the content or insufficient measurement properties of existing PROMs. However, with recent methodological innovations in PROM development, such as application of item response theory (IRT), universal PROMs have been developed with good measurement properties, that can be applied across medical conditions, including patients without a medical diagnosis and patients with multiple (chronic) conditions [37, 38]. One such cross-cutting IRT-based measurement option is the Patient-Reported Outcomes Measurement Information System (PROMIS®, Table 1). PROMIS researchers developed a conceptual framework of commonly relevant PROs across the broad domains of physical, mental, and social functioning. They also developed PROMs within each of these domains, that are universally applicable across patients populations [39, 40]. PROMIS researchers used IRT to create item banks (i.e. large sets of questions), enabling the possibility of applying short forms where needed and computerized adaptive tests (CAT) where possible, that yield highly reliable and comparable scores with a few relevant items only [41–43].
The aim of this study was to identify common PROs across ICHOM Standard Sets and examine the extent to which these PROs can be measured with PROMIS.
Methods
One author (MZ) downloaded the reference guides of all 39 available ICHOM Standard Sets from the ICHOM website on June 2021 and extracted all individuals PROs and recommended PROMs (including single items, unidimensional scales, and multidimensional questionnaires) [4]. The literal PRO terminology used in the reference guides was extracted. Another author (CT) checked the data extraction, and classified all PROs within the domains of global health, physical health, mental health, and social health, in accordance with the PROMIS conceptual framework. A third author (DC) reviewed the classification. Finally, one author (CT) searched the HealthMeasures website [44] to examine which of these PROs can be measured with a PROMIS measure.
Results
A total of 307 PROs were extracted from the 39 Standard Sets (Table 2). The number of PROs included in these Standard Sets varies from 1 (cataract) to 27 (Parkinson’s disease). The 307 PROs refer to 22 unique PRO concepts, three in the domain of global health (general health status, health-related quality of life, quality of life), fourteen in the domain of physical health (six included in the PROMIS profile (i.e. core) domains: fatigue, pain intensity, physical function, mobility, upper extremity function, sleep disturbances; three included in the PROMIS additional Domains: dyspnea, gastrointestinal symptoms, sexual function; and five that are not included in the PROMIS conceptual framework: hearing, speech/communication, urinary symptoms, vision, other), four in the domain of mental health (general mental health, anxiety, depression, cognitive function), and one in the domain of social health (ability to participate in social roles/peer relationships) (Table 2). The most commonly included PROs are ability to participate in social roles/peer relationships (included in 25 out of 39 Standard Sets, of which 11 used the same term, i.e. ‘social function(ing)’), physical function (included in 21 out of 39 Standard Sets, of which 10 used the same term, i.e. ‘physical function(ing)’), health-related quality of life (included in 18 Standard Sets), pain intensity (included in 17 Standard Sets, of which 12 used the same term, i.e. ‘pain’), depression (included in 16 Standard Sets), general mental health (included in 14 Standard Sets), anxiety (included in 12 Standard Sets), and fatigue and overall quality of life (both included in 10 Standard Sets)..
Table 2.
Patient-Reported Outcomes included in ICHOM Standard Sets, classified according to the PROMIS domain framework [62]
| PROMIS Global Health domain | PROMIS profile domains | PROMIS additional domains | Non-PROMIS domains | Total | |||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Physical health | Mental health | Social health | Physical health | Mental health | Physical health | ||||||||||||||||||
| Physical function | |||||||||||||||||||||||
| General health status |
Health-related quality of life | Quality of life | Fatigue | Pain | General | Mobility | Upper extre-mity | Sleep distur-bance | General | Anxiety | Depres-sion | Ability to participate in social roles/peer relation-ships | Dyspnea | Gastro-intestinal Symptoms | Sexual function | Cognitive function | Hearing | Speech communication | Urinary symp-toms | Vision | Other | ||
| Addiction | 1 | 1 | 1 | 1 | 5 | 10 | |||||||||||||||||
| Adult oral health | 1 | 1 | 2 | 1 | 4 | 9 | |||||||||||||||||
| Advanced prostate cancer | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 8 | ||||||||||||||
| Artrial fibrilation | 1 | 1 | 1 | 1 | 1 | 1 | 6 | ||||||||||||||||
| Breast cancer | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 2 | 2 | 1 | 5 | 18 | ||||||||||
| Cataract | 1 | 1 | |||||||||||||||||||||
| Chronic kidney disease | 1 | 1 | 1 | 2 | 1 | 6 | |||||||||||||||||
| Cleft lip and palate | 1 | 1 | 2 | 1 | 1 | 5 | 11 | ||||||||||||||||
| Colorectal cancer | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 6 | 3 | 1 | 18 | |||||||||||
| Congetinal heart disease | 1 | 1 | 1 | 1 | 1 | 5 | |||||||||||||||||
| Congenital upper limb anomalies | 1 | 1 | 1 | 1 | 4 | ||||||||||||||||||
| Coronary artery disease | 1 | 1 | 1 | 1 | 1 | 5 | |||||||||||||||||
| COVID-19 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 10 | ||||||||||||
| Craniofacial microsomia | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 4 | 13 | ||||||||||||
| Dementia | 1 | 1 | 1 | 3 | |||||||||||||||||||
| Depression and anxiety | 1 | 4 | 1 | 2 | 3 | 11 | |||||||||||||||||
| Depression and anxiety for children and young people | 2 | 1 | 2 | 2 | 7 | ||||||||||||||||||
| Diabetes | 1 | 1 | 1 | 3 | |||||||||||||||||||
| Hand and wrist conditions | 1 | 1 | 1 | 1 | 1 | 1 | 6 | ||||||||||||||||
| Heart failure | 1 | 1 | 2 | 1 | 1 | 1 | 1 | 2 | 10 | ||||||||||||||
| Hip and knee osteoarthritis | 1 | 1 | 1 | 1 | 4 | ||||||||||||||||||
| Hypertension in low and middle income countries | 1 | 1 | 2 | ||||||||||||||||||||
| Inflammatory arthritis | 1 | 1 | 1 | 1 | 1 | 5 | |||||||||||||||||
| Inflammatory bowel disease | 1 | 1 | 1 | 2 | 3 | 8 | |||||||||||||||||
| Localized prostate cancer | 1 | 1 | 2 | 1 | 5 | ||||||||||||||||||
| Low back pain | 1 | 2 | 1 | 1 | 5 | ||||||||||||||||||
| Lung cancer | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 2 | 1 | 10 | |||||||||||||
| Macula degeneration | 1 | 1 | 1 | 4 | |||||||||||||||||||
| Older person | 1 | 1 | 1 | 2 | 1 | 6 | |||||||||||||||||
| Overactive bladder | 1 | 1 | 1 | 3 | |||||||||||||||||||
| Overall adult health | 2 | 2 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 3 | 1 | 1 | 16 | ||||||||||
| Overall pediatric health | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 13 | |||||||||
| Parkinson's disease | 1 | 1 | 1 | 1 | 5 | 1 | 2 | 2 | 2 | 1 | 1 | 1 | 1 | 7 | 27 | ||||||||
| Pediatric facial palsy | 1 | 1 | 2 | 4 | |||||||||||||||||||
| Personality disorders | 1 | 2 | 3 | 1 | 2 | 9 | |||||||||||||||||
| Pregnancy and childbirth | 1 | 1 | 1 | 1 | 4 | ||||||||||||||||||
| Preterm and hospitalized newborn health | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 3 | 1 | 11 | |||||||||||||
| Psychotic disorders | 1 | 1 | 2 | 1 | 2 | 2 | 2 | 11 | |||||||||||||||
| Stroke | 2 | 1 | 1 | 1 | 1 | 1 | 1 | 8 | |||||||||||||||
| Total | 11 | 18 | 11 | 11 | 18 | 28 | 6 | 3 | 8 | 16 | 15 | 18 | 37 | 6 | 13 | 12 | 7 | 4 | 7 | 5 | 6 | 58 | |
| Number of sets that include the outcome | 8 | 18 | 10 | 10 | 17 | 21 | 6 | 3 | 7 | 14 | 12 | 16 | 25 | 5 | 6 | 9 | 7 | 4 | 7 | 4 | 6 | 25 | |
The numbers in each cell represent the number of outcomes covering the respective PRO concept included in the ICHOM Standard Set Reference Guide
In total, 176 PROMs (of which 114 are unique) are recommended for measuring the 307 PROs. The number of different PROMs recommended for the 22 unique PRO concepts varies from four (for measuring urinary symptoms) to 28 (for measuring physical function). Not many PROMs are included in more than one Standard Set: the PROMIS Global Health is included in twelve Standard Sets, the Patient Health Questionnaire (PHQ) in six Standard Sets, World Health Organization Disability Assessment Schedule (WHODAS-2) in six Standard Sets, EuroQol 5D (EQ-5D) in five Standard Sets, European Organization for Research and Treatment for Cancer Quality of Life Questionnaire (EORTC QLQ-C30) in four Standard Sets, Kidscreen in four Standards Sets, and another 22 PROMs are included in two or three Standard Sets.
More than half (17/22) of the PRO concepts (covering about 75% of the extracted PROs and also about 75% of the recommended PROMs) can be measured with PROMIS. Table 3 shows which PROMIS item banks are available to measure these PRO concepts. Figure 1 shows the most commonly included PROs in ICHOM Standard Sets that can be measured with PROMIS.
Table 3.
Common PROs that can be measured with PROMIS
| PRO | PROMIS adult item bank | Example item | PROMIS pediatric item bank | Example item |
|---|---|---|---|---|
| General health status (including general physical health and general mental health)/health-related quality of life | PROMIS v1.2 Global Health | In general, would you say your health is | PROMIS v1.0 Global Health | In general, would you say your health is |
| Quality of life | PROMIS Global02 | In general, would you say your quality of life is | ||
| Fatigue | PROMIS v1.0 Fatigue | How often did you feel tired? | PROMIS v2.0 Fatigue | I had trouble finishing things because I was too tired |
| Pain | PROMIS v1.1 Pain Interference | How much did pain interfere with your day to day activities? | PROMIS v2.0 Pain Interference | It was hard for me to pay attention when I had pain |
| PROMIS v2.0 Pain Behavior | When I was in pain I lay down | PROMIS v1.0 Pain Behavior | I protected the part of my body that hurt | |
| PROMIS v1.0 Pain Intensity | How intense was your pain at its worst? | PROMIS v1.0 Pain Intensity | How bad was your pain on average? | |
| PROMIS v2.0 Pain Quality | Did your pain feel like pins and needles? | PROMIS v2.0 Pain Quality | Did your pain feel awful? | |
| Physical function | PROMIS v2.0 Physical Function | Are you able to sit on and get up from the toilet? | ||
| PROMIS v2.0 Mobility | Are you able to go for a walk of at least 15 min? | PROMIS v2.0 Physical Function—Mobility | I could walk up stairs without holding on to anything | |
| PROMIS v2.0 Upper Extremity | Are you able to wash your back? | PROMIS v2.0 Physical Function—Upper Extremity | I could pull a shirt on over my head by myself | |
| Sexual function | PROMIS v2.0 Sexual Function and Satisfaction | How satisfied have you been with your sexual relationship(s)? | ||
| Sleep disturbance | PROMIS v1.0 Sleep Disturbances | I had difficulty falling asleep | PROMIS v1.0 Sleep Disturbances | I had difficulty falling asleep |
| Gastrointestinal symptoms | PROMIS v1.0 Gastrointestinal Symptoms | How often did you feel bloated? | ||
| Anxiety | PROMIS v1.0 Anxiety | I felt anxious | PROMIS v2.0 Anxiety | I felt scared |
| Depression | PROMIS v1.0 Depression | I felt sad | PROMIS v2.0 Depressive symptoms | I felt sad |
| Cognitive function | PROMIS v2.0 Cognitive Function | I have had trouble concentrating | PROMIS v2.0 Cognitive Function | It is hard for me to pay attention to one thing for more than 5–10 min |
| PROMIS v2.0 Cognitive Function Abilities subset | My memory has been as good as usual | |||
| Ability to participate in social roles/peer relationships | PROMIS v2.0 Abilities to Participate in Social Roles and Activities | I have trouble doing all of the family activities that I want to do | PROMIS v2.0 Peer Relationships | I felt accepted by other kids my age |
| PROMIS v2.0 Satisfaction with Social Roles and Activities | I am satisfied with my ability to do things for fun with others |
PROMIS Patient-Reported Outcomes Measurement Information System
Fig. 1.
Most commonly included PROs included in ICHOM Standard Sets that can be measured with PROMIS (the size of each wedge represents the relative number of ICHOM sets (out of 39 sets) that include this PRO concept, the shading does not have a particular meaning)
Discussion
We identified 307 different PROs in 39 ICHOM Standard Sets. The 307 PROs refer to 22 unique PRO concepts. A total of 114 different PROMs are being recommended for measuring these 307 PROs. Seventeen of the 22 unique PRO concepts, covering about 75% of the PROs and also about 75% of the PROMs, can be measured with PROMIS.
There is room for harmonization of PROs across ICHOM Standard Sets. While value-based healthcare is trying to eliminate the silos between medical specialties, it currently seems to have created new silos between conditions. Some PROs are included in only a few Standard Sets, while they seem relevant for many patients. For example, fatigue is included in the Standard Sets for inflammatory arthritis and heart failure, but not in the Standard Set for coronary artery disease. Sleep disturbances is only included in seven of the 39 Standard Sets, although it is a common symptom in many other diseases [45]. These results may partly be explained by a selection of only the most relevant PROs per condition. However, it is questionable to what extend this ranking represents the patient’s perspective. A more universal and standardized approach, focusing on a core set of PROs that are relevant for most patients, may provide more opportunities for patient-centered care. The profile domains from the PROMIS conceptual framework, including Fatigue, Pain Intensity, Pain Interference, Physical Function, Sleep Disturbance, Anxiety, Depression, and the Ability to Participate in Social Roles and Activities (included in the PROMIS-29 Profile measure [46] and also available as CATs), have been found relevant to many disease populations [47–52] and seem to be a good starting point. The core PROs could then be supplemented with disease-specific PROs (e.g. disease-specific symptoms) where needed, covering the additional 25% of PROs included in the ICHOM sets that cannot not be measured with PROMIS.
We also found large differences in PRO terminology used. For example, the PRO concept physical function is included in 21 Standard Sets, but only 10 Standard Sets use a similar term, i.e. ‘physical function(ing)’. Other terms used are ‘activities of daily living’, ‘activity limitations’, ‘disability’, ‘functional status’, ‘mobility’, and ‘motor function’. It is unclear whether these terms really refer to different concepts or whether they actually refer to the same concept. Definitions of the PROs are mostly lacking in the ICHOM reference guides. Eighteen Standard Sets included the PRO ‘health-related quality of life’. This is a very broad concept and can cover different PRO concepts. It is unclear whether this term refers to the same concept or to different concepts across Standard Sets. More attention need to be paid to the terms and definitions of PROs to be measured. ICHOM has recently started to harmonize terminology across Standard Sets, which may improve this situation in the near future.
Many different PROMs are being recommended, even when assessing the same PRO. For some patient groups a disease-specific PROM is recommended, while for another patient group, a domain-specific or generic PROM is recommended to measure the same PRO. For example, to measure depression, disease-specific instruments (e.g. the Movement Disorders Society Unified Parkinson Disease Rating Scale (MDS-UPDRS) for Parkinson), a cancer-specific instrument (EORTC QLQ-C30), several domain-specific instruments (e.g. Hospital Anxiety and Depression Scale (HADS), PHQ-2, PHQ-9, WHO-5) and several generic instruments (e.g. PROMIS-29 Profile, PROMIS Global Health and 36-Item Short Form Health Survey (SF-36)) are being recommended. Furthermore, nine different PROMs are being recommended to measure fatigue. This variability may partly be explained by (lack of) available evidence to support the use of a particular PROM in a specific condition. It is time-consuming and costly to translate and validate so many PROMs across countries. Recommending different PROMs for the same PROs hampers outcome measurement in daily clinical practice and comparisons across patient groups with different conditions. Harmonization of PROMs can improve this situation. For example, a PROMIS Depression or Fatigue measure could be used across all patient groups for which these PROs are relevant. Research has shown that it may not be necessary to measure common symptoms like fatigue or depression with a different instrument, validated for each different patient group. A study in patients with rheumatoid arthritis, for example, found that up to 90% of patients with arthritis would rate their level of fatigue similarly when asked in a general sense about fatigue or when asked about the fatigue they attributed to their rheumatoid arthritis, which suggest that a generic PROM can be used instead of a disease-specific PROM [53]. Furthermore, evidence is growing for the validity of generic PROMIS measures across patient populations [47–52, 54].
Another issue we identified is that there is often not an exact match in the ICHOM sets between the PROs and PROMs. This means that not every PRO is measured with a separate (sub)scale. For example, in the ICHOM Standard Set for Overall Adult Health, five PROs are included that address mental health (general mental health, sleep, depression, vitality, and anxiety). Two PROMs are recommended (PROMIS Global Health and WHO5) for measuring these PROs. However, neither the PROMIS Global Health nor the WHO5 provide separate scores for sleep, depression, vitality, and anxiety. If PROMs are to be used in the consultation room, separate scores for each PRO may be more helpful.
We argue that there is room for harmonization of PROs and PROMs across ICHOM Standard Sets. Our study showed that many PROs that matter to patients are common across patient groups. It may not be necessary to use a different PROM to measure common PROs, like pain, fatigue, depression, in different patient populations. It is hard to identify the best PROM for a specific patient population because the number of validation studies of PROMs in specific patient populations is limited and evidence on important measurement properties (e.g. responsiveness) is often lacking (See for example [55–57]). It is too expensive and time-consuming to develop, validate, translate, and maintain different high quality PROMs for every patient group. It is also too expensive, complex and time-consuming to implement all these different PROMs in electronic health records, give the right PROM to the right patient, interpret the scores for every PROM in the correct way, and discuss them appropriately with patients in the consultation room. It is burdensome and confusing for patients with multi- or comorbidity to complete multiple, partly overlapping, PROMs for every health care professional they consult.
To go forward, we recommend a more universal and standardized approach to PRO and PROM selection. Much can be gained by selecting common PROs and PROMs across conditions wherever possible, for example using the conceptual framework and measures of PROMIS. There are several initiatives ongoing in this direction. One is the recently developed ICHOM adult overall health and pediatric overall health Standard Sets [58, 59]. These Standard Set contain 15 and 10 PROs respectively, to be measured in all adult or pediatric patients. Eleven of the 15 PROs for adults and eight of the 10 PROs for children the can be measured with PROMIS. The relevance and feasibility of measuring a common set of PROs in all patients needs to be evaluated. In the Netherlands, an alternative approach is being proposed. A national PROM working group developed a ‘menu’ of commonly relevant PROs and recommended PROMs, that can be used to select relevant PROs for a specific patient group(s) [60]. Both the ICHOM overall health Standard Sets and the Dutch ‘menu’ recommend measuring generic PROs where possible, supplemented with disease-specific PROs where needed. An approach which may be referred to as 'generic unless'. Both initiatives include PROMIS measures in their recommendations to standardize and reduce the number of PROMs being used. PROMIS measures are also included in 14 out of the 39 ICHOM Standard Sets. There is, however, much room left for improving the efficiency and validity of ICHOM Standard Sets.
PROMIS has several advantages over traditional PROMs. Since it is applicable across disease populations, it enables benchmarking, learning and improving quality of care in patients groups with multimorbidity, the main cost-drivers of healthcare, and across many of these patient groups. Moreover, it is also suitable for patients without a definite diagnosis or for patients with rare diseases, for which validated disease-specific PROMs are quite often not available. An additional advantage of PROMIS is the possibility of CAT [37]. With CAT the computer selects items from an item bank, based on answers to previous items. This yield highly reliable scores with a few relevant items only, which is an important benefit for using PROMs in clinical practice. Technical solutions for CAT application are currently available in a limited number of countries, but this is expected to increase in the near future. As long as these technical solutions are lacking, PROMIS short forms can be applied as an alternative. Finally, PROMIS is a sustainable system, maintained by the PROMIS Health Organization, an international network of researchers and clinicians across a large number of countries, who collaborate to facilitate widespread use and adoption of PROMIS in research and clinical practice.
A limitation of our study is that the classification of PROs was not done by raters independently. The initial classification was done by one rater, and then reviewed with confirmation by a second rater. Furthermore, classification was based on information in the ICHOM reference guides only. We did not map the recommended PROMs on item level to the PROMIS measures. Our classification may therefore be not completely correct. However, the exact numbers are not important for our call for a more universal and standardized approach to PRO and PROM selection.
Conclusion
We found considerable overlap in selected PROs across ICHOM Standard Sets, and large differences in terminology and recommended PROMs for the same PROs. For measuring 307 different PROs, covering 22 unique PRO concepts, a total of 114 different PROMs are currently being recommended. We recommend a more universal and standardized approach to the selection of PROs and PROMs. PROMIS offers an evidence-based conceptual framework of commonly relevant PROs and provides a sustainable set of validated PROMs, that are applicable across patient populations and medical specialties.
Acknowledgements
Part of this work has previously been presented at the virtual annual conferences of the PROMIS Health Organization (PHO) and the International Society for Quality of Life Research (ISOQOL) in 2020.
Abbreviations
- CAT
Computerized Adaptive Test
- EORTC QLQ-C30
European Organization for Research and Treatment for Cancer Quality of Life Questionnaire
- EQ-5D
EuroQol 5D
- HADS
Hospital Anxiety and Depression Scale
- ICHOM
International Consortium for Health Outcomes Measurement
- IRT
Item Response Theory
- MDS-UPDRS
Movement Disorders Society Unified Parkinson Disease Rating Scale
- PHQ
Patient Health Questionnaire (PHQ)
- PRO
Patient-Reported Outcome
- PROM
Patient-Reported Outcome Measure
- PROMIS
Patient-Reported Outcomes Measurement Information System
- SF-36
36-Item Short Form Health Survey
- WHODAS
World Health Organization Disability Assessment Schedule
Authors' contributions
CT, MZ, and HV were responsible for the study concept and design. MZ performed the data extraction, which was checked by CT. DC reviewed the classification on PROs. CT drafted the manuscript. LH and LR drafted the tables and figure. The manuscript was critically revised and approved by all authors. All authors read and approved the final manuscript.
Funding
No funding was received for this study.
Availability of data and materials
An excel file with all data extraction is available upon request from the corresponding author.
Declarations
Ethics approval
Ethics approval was not required for this study as no data in human subjects was collected.
Consent for publication
Consent for publication was not required for this study as no data in human subjects was collected.
Competing interests
CB Terwee and D Cella are members of the board of directors of the PROMIS Health Organization. CB Terwee, L Haverman and LD Roorda are representatives of the Dutch-Flemish PROMIS National Center. HE Vonkeman was part of the working group which developed the ICHOM Standard Set for inflammatory arthritis.
Footnotes
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Porter ME, Teisberg EO. Redefining health care: creating value-based competition on results. Boston: Harvard Business Review Press; 2006. [Google Scholar]
- 2.Porter ME. What is value in health care? N Engl J Med. 2010;363(26):2477–2481. doi: 10.1056/NEJMp1011024. [DOI] [PubMed] [Google Scholar]
- 3.Porter ME, Kaplan RS. How to pay for health care. Harv Bus Rev. 2016;94(7–8):88–98. [PubMed] [Google Scholar]
- 4.International Consortium for Health Outcomes Measurement (ICHOM) www.ichom.org.
- 5.ICHOM Methodology.
- 6.Akpan A, Roberts C, Bandeen-Roche K, Batty B, Bausewein C, Bell D, Bramley D, Bynum J, Cameron ID, Chen LK, et al. Standard set of health outcome measures for older persons. BMC Geriatr. 2018;18(1):36. doi: 10.1186/s12877-017-0701-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.de Roos P, Bloem BR, Kelley TA, Antonini A, Dodel R, Hagell P, Marras C, Martinez-Martin P, Mehta SH, Odin P, et al. A consensus set of outcomes for Parkinson's Disease from the international consortium for health outcomes measurement. J Parkinson's Dis. 2017;7(3):533–543. doi: 10.3233/JPD-161055. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Foust-Wright C, Wissig S, Stowell C, Olson E, Anderson A, Anger J, Cardozo L, Cotterill N, Gormley EA, Toozs-Hobson P, et al. Development of a core set of outcome measures for OAB treatment. Int Urogynecol J. 2017;28(12):1785–1793. doi: 10.1007/s00192-017-3481-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Kim AH, Roberts C, Feagan BG, Banerjee R, Bemelman W, Bodger K, Derieppe M, Dignass A, Driscoll R, Fitzpatrick R, et al. Developing a standard set of patient-centred outcomes for inflammatory bowel disease-an international, cross-disciplinary consensus. J Crohn's Colitis. 2018;12(4):408–418. doi: 10.1093/ecco-jcc/jjx161. [DOI] [PubMed] [Google Scholar]
- 10.McNamara RL, Spatz ES, Kelley TA, Stowell CJ, Beltrame J, Heidenreich P, Tresserras R, Jernberg T, Chua T, Morgan L, et al. Standardized outcome measurement for patients with coronary artery disease: consensus from the International Consortium for Health Outcomes Measurement (ICHOM) J Am Heart Assoc. 2015;4(5):e001767. doi: 10.1161/JAHA.115.001767. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Nijagal MA, Wissig S, Stowell C, Olson E, Amer-Wahlin I, Bonsel G, Brooks A, Coleman M, Devi Karalasingam S, Duffy JMN, et al. Standardized outcome measures for pregnancy and childbirth, an ICHOM proposal. BMC Health Serv Res. 2018;18(1):953. doi: 10.1186/s12913-018-3732-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Obbarius A, van Maasakkers L, Baer L, Clark DM, Crocker AG, de Beurs E, Emmelkamp PMG, Furukawa TA, Hedman-Lagerlof E, Kangas M, et al. Standardization of health outcomes assessment for depression and anxiety: recommendations from the ICHOM Depression and Anxiety Working Group. Qual Life Res. 2017;26(12):3211–3225. doi: 10.1007/s11136-017-1659-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Ong WL, Schouwenburg MG, van Bommel ACM, Stowell C, Allison KH, Benn KE, Browne JP, Cooter RD, Delaney GP, Duhoux FP, et al. A standard set of value-based patient-centered outcomes for breast cancer: the International Consortium for Health Outcomes Measurement (ICHOM) Initiative. JAMA Oncol. 2017;3(5):677–685. doi: 10.1001/jamaoncol.2016.4851. [DOI] [PubMed] [Google Scholar]
- 14.Oude Voshaar MAH, Das Gupta Z, Bijlsma JWJ, Boonen A, Chau J, Courvoisier DS, Curtis JR, Ellis B, Ernestam S, Gossec L et al. The International Consortium for Health Outcome Measurement (ICHOM) set of outcomes that matter to people living with inflammatory arthritis. Consensus from an international working group. Arthritis Care Res (Hoboken). 2018. [DOI] [PMC free article] [PubMed]
- 15.Rodrigues IA, Sprinkhuizen SM, Barthelmes D, Blumenkranz M, Cheung G, Haller J, Johnston R, Kim R, Klaver C, McKibbin M, et al. Defining a minimum set of standardized patient-centered outcome measures for macular degeneration. Am J Ophthalmol. 2016;168:1–12. doi: 10.1016/j.ajo.2016.04.012. [DOI] [PubMed] [Google Scholar]
- 16.Verberne WR, Das-Gupta Z, Allegretti AS, Bart HAJ, van Biesen W, Garcia-Garcia G, Gibbons E, Parra E, Hemmelder MH, Jager KJ, et al. Development of an international standard set of value-based outcome measures for patients with chronic kidney disease: a report of the International Consortium for Health Outcomes Measurement (ICHOM) CKD Working Group. Am J Kidney Dis Off J Natl Kidney Found. 2019;73(3):372–384. doi: 10.1053/j.ajkd.2018.10.007. [DOI] [PubMed] [Google Scholar]
- 17.Zerillo JA, Schouwenburg MG, van Bommel ACM, Stowell C, Lippa J, Bauer D, Berger AM, Boland G, Borras JM, Buss MK, et al. An international collaborative standardizing a comprehensive patient-centered outcomes measurement set for colorectal cancer. JAMA Oncol. 2017;3(5):686–694. doi: 10.1001/jamaoncol.2017.0417. [DOI] [PubMed] [Google Scholar]
- 18.Nano J, Carinci F, Okunade O, Whittaker S, Walbaum M, Barnard-Kelly K, Barthelmes D, Benson T, Calderon-Margalit R, Dennaoui J, et al. A standard set of person-centred outcomes for diabetes mellitus: results of an international and unified approach. Diabet Med J Br Diabet Assoc. 2020;37:2009–2018. doi: 10.1111/dme.14286. [DOI] [PubMed] [Google Scholar]
- 19.Zack R, Okunade O, Olson E, Salt M, Amodeo C, Anchala R, Berwanger O, Campbell N, Chia Y, Damasceno A, et al. Improving hypertension outcome measurement in low- and middle-income countries. Hypertension. 2019;73:990–997. doi: 10.1161/HYPERTENSIONAHA.118.11916. [DOI] [PubMed] [Google Scholar]
- 20.Martin NE, Massey L, Stowell C, Bangma C, Briganti A, Bill-Axelson A, Blute B, Catto J, Chen RC, D’Amico AV, et al. Defining a standard set of patient-centered outcomes for men with localized prostate cancer. Eur Urol. 2015;67:460–467. doi: 10.1016/j.eururo.2014.08.075. [DOI] [PubMed] [Google Scholar]
- 21.Mahmud I, Kelley T, Stowell C, Haripriya A, Boman A, Kossler I, Morlet N, Pershing S, Pesudovs K, Pin Goh P, et al. A Proposed minimum standard set of outcome measures for cataract surgery. JAMA Ophthamol. 2015;133:1247–1252. doi: 10.1001/jamaophthalmol.2015.2810. [DOI] [PubMed] [Google Scholar]
- 22.Morgans AK, van Bommel ACM, Stowell C, Abrahm JL, Basch E, Bekelman JE, Berry DL, Bossi A, Davis ID, de Reijke TM, et al. Development of a standardized set of patient-centered outcomes for advanced prostate cancer: an international effort for a unified approach. Eur Eurol. 2015;68:891–898. doi: 10.1016/j.eururo.2015.06.007. [DOI] [PubMed] [Google Scholar]
- 23.Allori AC, Kelley T, Meara JG, Albert A, Bonanthaya K, Chapman K, Cunningham M, Daskalogiannakis J, de Gier H, Heggie AA, et al. A standard set of outcome measures for the comprehensive appraisal of cleft care. Cleft Palate Craniofac J Off Publ Am Cleft Palate Craniofac Assoc. 2017;54(5):540–554. doi: 10.1597/15-292. [DOI] [PubMed] [Google Scholar]
- 24.Akpan A, Roberts C, Bandeen-Roche K, Batty B, Bausewein C, Bell D, Bramley D, Bynum J, Cameron ID, Chen LK, et al. Standard set of health outcome measures for older persons. BMC Geriatr. 2018;18:1–10. doi: 10.1186/s12877-017-0701-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Burns DJ, Arora J, Okunade O, Beltrame JF, Bernardez-Pereira S, Crespo-Leiro MG, Filippatos GS, Hardman S, Hoes AW, Hutchison S, et al. International Consortium for Health Outcomes Measurement (ICHOM): standardized patient-centered outcomes measurement set for heart failure patients. Jacc Heart Fail. 2020;8(3):212–222. doi: 10.1016/j.jchf.2019.09.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Butler DP, De la Torre A, Borschel GH, Hadlock TA, Beurskens C, Bogart K, Cárdenas Mejía A, Coombs C, Copeland J, Diels J, et al. An international collaborative standardizing patient-centered outcome measures in pediatric facial palsy. JAMA Fac Plast Surg. 2019;21(5):351–358. doi: 10.1001/jamafacial.2019.0224. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Clement RC, Welander A, Stowell C, Cha TD, Chen JL, Davies M, Fairbank JC, Foley KT, Gehrchen M, Hagg O, et al. A proposed set of metrics for standardized outcome reporting in the management of low back pain. Acta Orthop. 2015;86(5):523–533. doi: 10.3109/17453674.2015.1036696. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Mak KS, van Bommel AC, Stowell C, Abrahm JL, Baker M, Baldotto CS, Baldwin DR, Borthwick D, Carbone DP, Chen AB, et al. Defining a standard set of patient-centred outcomes for lung cancer. Eur Respir J. 2016;48(3):852–860. doi: 10.1183/13993003.02049-2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Reynish E, Burns A, Roberts C. Defining a standard set of patient-centered outcomes for patients with dementia. Innov Aging. 2017;1(Suppl 1):290. doi: 10.1093/geroni/igx004.1076. [DOI] [Google Scholar]
- 30.Rolfson O, Wissig S, van Maasakkers L, Stowell C, Ackerman I, Ayers D, Barber T, Benzakour T, Bozic K, Budhiparama N, et al. Defining an international standard set of outcome measures for patients with hip or knee osteoarthritis: consensus of the international consortium for health outcomes measurement hip and knee osteoarthritis working group. Arthritis Care Res (Hoboken) 2016;68(11):1631–1639. doi: 10.1002/acr.22868. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Salinas J, Sprinkhuizen SM, Ackerson T, Bernhardt J, Davie C, George MG, Gething S, Kelly AG, Lindsay P, Liu L, et al. An international standard set of patient-centered outcome measures after stroke. Stroke J Cereb Circ. 2016;47(1):180–186. doi: 10.1161/STROKEAHA.115.010898. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Riordain RN, Glick M, Mashhadani S, Aravamudhan K, Barrow J, Cole D, Crall JJ, Gallagher JE, Gibson J, Hegde S, et al. Developing a standard set of patient-centred outcomes for adult oral health: an international, cross-disciplinary consensus. Int Dent J. 2021;71(1):40–52. doi: 10.1111/idj.12604. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Hummel K, Whittaker S, Sillett N, Basken A, Berghammer M, Chalela T, Chauhan J, Garcia LA, Hasan B, Jenkins K, et al. Development of an international standard set of clinical and patient-reported outcomes for children and adults with congenital heart disease: a report from the International Consortium for Health Outcomes Measurement Congenital Heart Disease Working Group. Eur Heart J Qual Care Clin Outcomes. 2021;7:354–365. doi: 10.1093/ehjqcco/qcab009. [DOI] [PubMed] [Google Scholar]
- 34.Krause KR, Chung S, Adewuya AO, Albano AM, Babins-Wagner R, Birkinshaw L, Brann P, Creswell C, Delaney K, Falissard B, et al. International consensus on a standard set of outcome measures for child and youth anxiety, depression, obsessive-compulsive disorder, and post-traumatic stress disorder. Lancet Psychiatry. 2021;8(1):76–86. doi: 10.1016/S2215-0366(20)30356-4. [DOI] [PubMed] [Google Scholar]
- 35.Prevolnik Rupel V, Jagger B, Fialho LS, Chadderton LM, Gintner T, Arntz A, Baltzersen ÅL, Blazdell J, van Busschbach J, Cencelli M, et al. Standard set of patient-reported outcomes for personality disorder. Qual Life Res. 2021 doi: 10.1007/s11136-021-02870-w. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Dodd S, Clarke M, Becker L, Mavergames C, Fish R, Williamson PR. A taxonomy has been developed for outcomes in medical research to help improve knowledge discovery. J Clin Epidemiol. 2018;96:84–92. doi: 10.1016/j.jclinepi.2017.12.020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Cella D, Gershon R, Lai JS, Choi S. The future of outcomes measurement: item banking, tailored short-forms, and computerized adaptive assessment. Qual Life Res. 2007;16(Suppl 1):133–141. doi: 10.1007/s11136-007-9204-6. [DOI] [PubMed] [Google Scholar]
- 38.Khanna D, Krishnan E, Dewitt EM, Khanna PP, Spiegel B, Hays RD. The future of measuring patient-reported outcomes in rheumatology: Patient-Reported Outcomes Measurement Information System (PROMIS) Arthritis Care Res (Hoboken) 2011;63(Suppl 11):S486–S490. doi: 10.1002/acr.20581. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Cella D, Riley W, Stone A, Rothrock N, Reeve B, Yount S, Amtmann D, Bode R, Buysse D, Choi S, et al. The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005–2008. J Clin Epidemiol. 2010;63(11):1179–1194. doi: 10.1016/j.jclinepi.2010.04.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Cella D, Yount S, Rothrock N, Gershon R, Cook K, Reeve B, Ader D, Fries JF, Bruce B, Rose M. The Patient-Reported Outcomes Measurement Information System (PROMIS): progress of an NIH Roadmap cooperative group during its first two years. Med Care. 2007;45(5 Suppl 1):S3–S11. doi: 10.1097/01.mlr.0000258615.42478.55. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Thomas ML. Advances in applications of item response theory to clinical assessment. Psychol Assess. 2019;31:1442. doi: 10.1037/pas0000597. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Chang CH. Patient-reported outcomes measurement and management with innovative methodologies and technologies. Qual Life Res. 2007;16(Suppl 1):157–166. doi: 10.1007/s11136-007-9196-2. [DOI] [PubMed] [Google Scholar]
- 43.Nguyen TH, Han HR, Kim MT, Chan KS. An introduction to item response theory for patient-reported outcome measurement. Patient. 2014;7(1):23–35. doi: 10.1007/s40271-013-0041-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.http://www.healthmeasures.net/promis.
- 45.Worley SL. The extraordinary importance of sleep: the detrimental effects of inadequate sleep on health and public safety drive an explosion of sleep research. P & T Peer Rev J Formul Manag. 2018;43(12):758–763. [PMC free article] [PubMed] [Google Scholar]
- 46.Cella D, Choi SW, Condon DM, Schalet B, Hays RD, Rothrock NE, Yount S, Cook KF, Gershon RC, Amtmann D, et al. PROMIS(®) adult health profiles: efficient short-form measures of seven health domains. Value Health. 2019;22(5):537–544. doi: 10.1016/j.jval.2019.02.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Holden SK, Koljack CE, Prizer LP, Sillau SH, Miyasaki JM, Kluger BM. Measuring quality of life in palliative care for Parkinson's disease: a clinimetric comparison. Parkinsonism Related Disord. 2019;65:172–177. doi: 10.1016/j.parkreldis.2019.06.018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Katz P, Pedro S, Michaud K. Performance of the patient-reported outcomes measurement information system 29-item profile in rheumatoid arthritis, osteoarthritis, fibromyalgia, and systemic lupus erythematosus. Arthritis Care Res (Hoboken) 2017;69(9):1312–1321. doi: 10.1002/acr.23183. [DOI] [PubMed] [Google Scholar]
- 49.LaVela SL, Etingen B, Miskevics S, Cella D. Use of PROMIS-29® in US veterans: diagnostic concordance and domain comparisons with the general population. J Gen Intern Med. 2019;34(8):1452–1458. doi: 10.1007/s11606-019-05011-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Rawang P, Janwantanakul P, Correia H, Jensen MP, Kanlayanaphotporn R. Cross-cultural adaptation, reliability, and construct validity of the Thai version of the Patient-Reported Outcomes Measurement Information System-29 in individuals with chronic low back pain. Qual Life Res. 2020;29(3):793–803. doi: 10.1007/s11136-019-02363-x. [DOI] [PubMed] [Google Scholar]
- 51.Rose AJ, Bayliss E, Huang W, Baseman L, Butcher E, García RE, Edelen MO. Evaluating the PROMIS-29 v2.0 for use among older adults with multiple chronic conditions. Qual Life Res. 2018;27(11):2935–2944. doi: 10.1007/s11136-018-1958-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Tang E, Ekundayo O, Peipert JD, Edwards N, Bansal A, Richardson C, Bartlett SJ, Howell D, Li M, Cella D, et al. Validation of the Patient-Reported Outcomes Measurement Information System (PROMIS)-57 and -29 item short forms among kidney transplant recipients. Qual Life Res. 2019;28(3):815–827. doi: 10.1007/s11136-018-2058-2. [DOI] [PubMed] [Google Scholar]
- 53.Bartlett SJ, Gutierrez AK, Butanis A, Bykerk VP, Curtis JR, Ginsberg S, Leong AL, Lyddiatt A, Nowell WB, Orbai AM, et al. Combining online and in-person methods to evaluate the content validity of PROMIS fatigue short forms in rheumatoid arthritis. Qual Life Res. 2018;27(9):2443–2451. doi: 10.1007/s11136-018-1880-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54.Crins MHP, Terwee CB, Ogreden O, Schuller W, Dekker P, Flens G, Rohrich DC, Roorda LD. Differential item functioning of the PROMIS physical function, pain interference, and pain behavior item banks across patients with different musculoskeletal disorders and persons from the general population. Qual Life Res. 2019;28:1231–1243. doi: 10.1007/s11136-018-2087-x. [DOI] [PubMed] [Google Scholar]
- 55.Gabes M, Tischer C, Apfelbacher C. Measurement properties of quality-of-life outcome measures for children and adults with eczema: an updated systematic review. Pediatr Allergy Immunol Off Publ Eur Soc Pediatr Allergy Immunol. 2020;31(1):66–77. doi: 10.1111/pai.13120. [DOI] [PubMed] [Google Scholar]
- 56.Aiyegbusi OL, Kyte D, Cockwell P, Marshall T, Gheorghe A, Keeley T, Slade A, Calvert M. Measurement properties of patient-reported outcome measures (PROMs) used in adult patients with chronic kidney disease: a systematic review. PLoS ONE. 2017;12(6):e0179733. doi: 10.1371/journal.pone.0179733. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57.Haywood KL, Pearson N, Morrison LJ, Castrén M, Lilja G, Perkins GD. Assessing health-related quality of life (HRQoL) in survivors of out-of-hospital cardiac arrest: a systematic review of patient-reported outcome measures. Resuscitation. 2018;123:22–37. doi: 10.1016/j.resuscitation.2017.11.065. [DOI] [PubMed] [Google Scholar]
- 58.ICHOM Standard Set overall adult health. https://www.ichom.org/portfolio/adult-overall-health/.
- 59.ICHOM Standard Set overall pediatric health.
- 60.www.linnean.nl.
- 61.Administration USDoHaHSaD, (CDER) CfDEaR, (CBER) CfBEaR, (CDRH) CfDaRH. Guidance for industry patient-reported outcome measures: use in medical product development to support labeling claims; 2009. [DOI] [PMC free article] [PubMed]
- 62.PROMIS domain framework.
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
An excel file with all data extraction is available upon request from the corresponding author.