Validation of PROMIS Profile‐29 in adults with hemophilia in the Netherlands

Abstract

Background

The Patient‐Reported Outcomes Measurement Information System (PROMIS) Profile‐29 questionnaire is widely used worldwide, but it has not yet been validated in the Netherlands, nor in persons with hemophilia.

Objective

To validate the Dutch‐Flemish version of the PROMIS‐29 Profile v2.01 in adults with hemophilia.

Methods

Dutch males with hemophilia (all severities) completed questionnaires that contained sociodemographic and clinical characteristics, the PROMIS‐29, RAND‐36, and the Hemophilia Activities List (HAL). Structural validity of each subscale was assessed with confirmatory factor analysis (CFA). Internal consistency was calculated for each subscale with sufficient model fit in CFA. Construct validity was assessed by testing hypotheses about (1) correlations of each PROMIS‐29 subscale with corresponding scales of RAND‐36 and domains of HAL, and (2) mean differences in T‐scores between subgroups with different hemophilia severities, self‐reported joint impairment, and HIV infection status. We considered ≥75% of data in accordance with the hypotheses evidence for construct validity.

Results

In total, 770 persons with hemophilia participated in this cross‐sectional study. CFA revealed sufficient structural validity for five subscales: Physical Function, Depression, Sleep Disturbance, Ability to Participate in Social Roles and Activities, and Pain Interference. Internal consistency was high and Cronbach's alpha ranged from 0.79 for Sleep Disturbance to 0.96 for Pain Interference. Differences between clinical subgroups were in the expected direction. Construct validity was confirmed for Physical Function, Anxiety, Depression, Fatigue, Sleep Disturbance, and Pain Intensity.

Conclusion

This study revealed sufficient evidence for structural validity, internal consistency, and construct validity for most PROMIS Profile‐29 subscales among people with hemophilia in the Netherlands.

Essentials.

• The PROMIS Profile‐29 questionnaire offers precise assessment of patient‐relevant outcomes.

• PROMIS Profile‐29 was validated in a large sample of Dutch adults with hemophilia.

• Structural validity, internal consistency and construct validity were sufficient.

• The PROMIS Profile‐29 may be used to assess health status in persons with hemophilia.

1. INTRODUCTION

The congenital bleeding disorder hemophilia causes recurrent bleeds into joints and muscles because of a deficiency in coagulation factor VIII (hemophilia A) or factor IX (hemophilia B). The condition predominantly affects males and is classified into mild (0.05–0.40 IU/ml), moderate (0.01–0.05 IU/ml), and severe (<0.01 IU/ml) hemophilia, depending on the activity of factor VIII or IX. Individuals with severe hemophilia often suffer from spontaneous bleeds into joints and muscles, whereas those with mild hemophilia typically bleed when triggered by trauma or surgery. ¹ Treatment consists of coagulation factor replacement by intravenous injection to treat bleeds (episodic treatment) or to prevent bleeds (prophylaxis, defined as regular administration of a hemostatic agent, usually administered intravenously or subcutaneously). Recently, non‐factor replacement products have been marketed and gene therapy is under study. ¹

Early forms of treatment had devastating effects on the hemophilia community: through contaminated plasma‐derived blood products, many patients were infected with HIV in the 1980s and/or hepatitis C virus before the 1990s. ² The availability of treatment has resulted in a near‐normal life expectancy and improved outcomes, ³ but a potential side effect of factor replacement therapy is the development of neutralizing antibodies (“inhibitors”) against the infused coagulation factor. Regular prophylaxis with factor replacement products is not effective in patients with inhibitors, and recently, prophylaxis with non‐factor replacement products has helped reduce the burden of bleeding. ¹ In addition, joint damage (hemophilic arthropathy), pain, and disability are still relatively common, especially among older males affected by severe hemophilia because of recurrent joint bleeding. Large differences in joint status and pain exist between individuals. It is important to measure and monitor these outcomes in persons with hemophilia to personalize health care.

Patient‐reported outcomes (PROs) are any aspect of a patient's health that come directly from the patient without interpretation of the patient's responses by a physician or anyone else. ⁴ In hemophilia, PROs have been measured with hemophilia‐specific instruments such as the Hemophilia Activities List (HAL), ⁵ , ⁶ Haemo‐Quality of Life (QoL)‐A, ⁷ and Hemofilia‐QoL ⁸ as well as with generic instruments such as the RAND‐36 ⁹ or EQ‐5D. Two systematic reviews reported that the measurement properties of hemophilia‐specific instruments have not been studied sufficiently, in particular structural validity, responsiveness, and hypothesis testing. ¹⁰ , ¹¹ Whether to use disease‐specific or generic tools for hemophilia PROs depends on the goal of measuring such outcomes.

An alternative approach to measuring PROs is to use generic instruments based on Item Response Theory (IRT), which has several advantages over other generic instruments. First, instruments using IRT‐based scoring take the difficulty of items into account, thereby providing more valid and reliable scores. ¹² Second, IRT‐based item banks, consisting of large sets of questions, can be used as short forms of any length (consisting of the best performing items from an item bank) or as computerized adaptive tests (CAT). In a CAT, the computer selects relevant questions based on the answer to the previous question, resulting in even more efficient and precise, but comprehensive assessment of a construct of interest. The use of patient‐reported outcome measures (PROMs) in clinical practice is increasing. Using different PROMS for different patients and implementing many different PROMs in electronic health records may pose a burden on researchers and clinicians. Therefore, the availability of valid and precise generic PROMs for domains that are relevant across medical conditions (such as pain, Fatigue, Physical Function) would be highly beneficial.

The Patient‐Reported Outcomes Measurement Information System (PROMIS), developed in the United States, is the most extensively validated measurement system of item banks in the world. ¹³ , ¹⁴ , ¹⁵ PROMIS profiles have been developed that consist of a collection of short forms derived from IRT‐based item banks, covering seven patient‐relevant domains. Profiles offer quick assessment of several domains of health‐related QoL (HRQoL). ¹⁶ Available profiles are the Profile‐29, Profile‐43, and Profile‐57, which measure seven domains with four, six, or eight items, respectively. ¹⁶ As a generic tool, PROMIS‐29 has the advantage of making results comparable across diseases and the general population. ¹²

Before using an instrument in a new population or language, it should be validated ⁴ by assessing its measurement properties. The measurement properties can be divided into three domains: validity (content validity, construct validity, hypotheses‐testing), reliability (internal consistency, measurement error, and test‐retest reliability), and responsiveness. ¹⁷ A hierarchy of measurement properties can be defined. ¹⁸ Content validity is considered the most important measurement property, defined as the degree to which the content of an instrument is an adequate reflection of the construct to be measured. ¹⁸ It can be assessed in a qualitative study in which the relevance, comprehensiveness, and comprehensibility of the items of a PROM are assessed, for example by cognitive debriefing in the target population. ¹⁹ The next measurement properties that should be evaluated are structural validity and internal consistency. ¹⁷ Structural validity is the degree to which the scores of an instrument are an adequate reflection of the dimensionality of the construct to be measured ¹⁸ and is assessed with confirmatory factor analysis. ⁴ Internal consistency is the degree of interrelatedness of items ¹⁸ as assessed with Cronbach's alpha. ⁴ Finally, other measurement properties are to be evaluated, such as test‐retest reliability (the extent to which scores are stable over time in stable participants), construct validity (the degree to which the scores of an instrument are consistent with formulated hypotheses about relationships to scores of other instruments, or differences between relevant groups, based on the assumption that the instrument validly measures the construct to be measured), and responsiveness (the ability of an instrument to detect a change of the construct over time).

Item banks that underlie the PROMIS Profiles were translated into Dutch and showed sufficient linguistic, content, and conceptual equivalence. ²⁰ A next step is to evaluate the measurement properties of the item banks and their derivative short forms. PROMIS Profiles have been validated in several countries and in a number of conditions, ²¹ , ²² , ²³ but not yet in hemophilia.

Therefore, this study aimed to validate the Dutch‐Flemish version of the PROMIS‐29 Profile v2.01 in Dutch adults with hemophilia by assessing its structural validity, internal consistency, and construct (convergent and discriminative) validity.

2. METHODS

Data were collected as part of the Dutch nationwide Hemophilia in the Netherlands 6 study (HiN‐6). HiN‐6 is the latest in a series of six cross‐sectional studies that have been conducted since 1972. ³ , ²⁴ , ²⁵ Approval was obtained from the Medical Ethical Committee at Leiden University Medical Center, the Netherlands (registration number NL59114.058.17).

2.1. Participants and procedures

All adult males with mild, moderate, or severe congenital hemophilia A or B with levels of factor VIII or IX <0.40 IU/ml registered at one of the six Dutch hemophilia treatment centers were invited by letter to participate between June 2018 and July 2019.

Participants received a questionnaire through a secure e‐mail link or in hard copy, depending on their preference. Answers were stored in the Castor Electronic Data Capture system. ²⁶ Clinical characteristics were collected from electronic medical records. Participants signed written informed consent for extraction of data from electronic medical records, but this was not required for participation in the questionnaire.

2.2. Measures

Self‐reported sociodemographic and clinical data collected through the questionnaire were: age, education level (categorized in International Standard Classification of Education levels ²⁷ ), and perceived impairment in joint function. Joint impairment was assessed with a single question that was used in previous HiN surveys. Joint impairment was defined as “do you have any chronic joint problems due to hemophilia” (yes/no). Clinical characteristics collected from electronic medical records were type and severity of hemophilia, treatment type (prophylaxis, episodic), inhibitor status, and HIV and hepatitis C virus status. Clinical characteristics were taken from medical records if the participant had signed written informed consent for use of these data. If medical record data were not available, self‐reported data from the questionnaire were used. Hemophilia severity was known for all responders and nonresponders.

2.3. Dutch‐Flemish PROMIS‐29 profile v2.01

PROMIS Profiles are derived from full PROMIS item banks that were developed in the US general population and patient groups. ¹³ PROMIS Profiles were shown to be reliable and correlate highly with full item banks. ¹⁶ The PROMIS‐29 measures seven domains of HRQoL that are often considered important by patients: ¹⁶ Physical Function; Anxiety; Depression; Fatigue; Sleep Disturbance; Ability to Participate in Social Roles and Activities; and Pain Interference. Each domain is measured with four items. The PROMIS‐29 also contains a single item on Pain Intensity, resulting in a total of 29 items. Each item is scored from 1 to 5; a higher score indicates a higher degree of the construct being measured. For the subscales Physical Function and Ability to Participate in Social Roles and Activities, this means that a higher score indicates better HRQoL, whereas for the other subscales a higher score indicates worse HRQoL. ¹⁶ Domain scores were calculated as T‐scores using the Health Measures Scoring Service, ²⁸ resulting in a normalized score with a mean of 50 and a standard deviation (SD) of 10 in the reference population (the US general population). T‐scores were only calculated for a domain if at least one item of that domain was completed; T‐scores were considered missing if none of the items was completed.

2.4. RAND‐36

RAND‐36 version 1 is a generic measure that assesses health status using 36 items. It consists of eight health concepts with multi‐item scales: physical functioning (10 items); social functioning (two items); role limitations caused by physical health problems (four items); role limitations caused by emotional problems (three items); emotional well‐being (five items); pain (two items); general health perceptions (five items); energy/fatigue (four items); and an additional single item measuring change in perceived health during the past 12 months. ²⁹ Items were scored on a 3‐ to 6‐point Likert scale. As per the standard scoring instructions, subscale scores were calculated if a participant had completed at least half of the items of that subscale. ³⁰ If fewer than half of the items were completed, subscale scores were considered missing. Subscale scores were converted to a 0‐ to 100‐point scale. ⁹ A higher score indicates a better health status. The RAND‐36 was reported to have good internal consistency and discriminative validity in the Dutch general population ³¹ and in several hemophilia populations. ³² , ³³

2.5. Hemophilia activities list

HAL version 2.0 is a hemophilia‐specific instrument, developed in the Netherlands, that measures self‐perceived functional abilities in adults with hemophilia, in the previous month. It consists of 42 items in seven subdomains: lying/sitting/kneeling/standing (eight items), functions of the legs (nine items), functions of the arms (four items), use of transportation (three items), self‐care (five items), household tasks (six items), and leisure activities and sports (seven items). Items are scored on a 6‐point Likert scale. ⁵ , ⁶ Scores were calculated according to the standard instructions (i.e., a domain score was calculated if less than half of the items were missing) and converted to a 0‐ to 100‐point scale, with a higher score indicating better functional status. The HAL has sufficient content validity and construct validity but its structural validity is not known. ¹¹

2.6. Statistical analyses

Descriptive statistics (means, SD, N) were used to describe participant characteristics. Mean scores, SDs, the proportion of best and worst scores and percentage of missing scores for each domain or subscale were described for all measures. If proportions of best and worst scores were >30%, these were considered substantial ceiling or floor effects, respectively. ²¹

Structural validity, internal consistency, and construct validity were investigated as defined by the Consensus‐based Standards for the Selection of Health Measurement Instruments (COSMIN) taxonomy ¹⁸ and reported according to the COSMIN reporting guideline for studies on measurement properties. ³⁴ A sample size of at least 100 participants is considered adequate for these analyses. ³⁵

Structural validity was assessed with confirmatory factor analysis (CFA) for each PROMIS domain separately. Model parameters were estimated with the weighted least square mean and variance adjusted estimators for ordinal data. ³⁶ Model fit was assessed using the Comparative Fit Index (CFI), the Tucker‐Lewis Index (TLI), and the root mean squared error of approximation (RMSEA). Model fit was considered sufficient if CFI or TLI was >0.95 or RMSEA <0.06. ³⁷ Internal consistency was calculated for each domain with sufficient model fit and considered sufficient if Cronbach's alpha was ≥0.70. ³⁷

Hypotheses were formulated a priori for construct validity (convergent and discriminative) for each domain. We considered ≥75% of results in accordance with the hypotheses evidence for construct validity. ³⁷ Convergent validity was assessed with Pearson's correlations. We expected strong correlations (r ≥ .70 or r ≥ −.70) between similar subscales of PROMIS‐29 with RAND‐36 subscales and HAL domains, based on published literature ³⁸ , ³⁹ , ⁴⁰ and expert judgment (authors E.v.B. and S.G.), as shown in Table 1. All other correlations were expected to be ≤0.60.

TABLE 1.

Hypotheses for construct validity (convergent and discriminative)

PROMIS‐29 Subscale	Convergent Validity		MID	Discriminative Validity
	Pearson's r ≥ .70	Pearson's r ≤ .60		∆ T‐score ≥MID	∆ T‐score <MID
Physical Function	RAND‐36 physical functioning	All other RAND‐36 subscales (n = 8) All other HAL domains (n = 2)	2.0	Mild‐severe hemophilia No‐yes joint impairment	No‐yes HIV infection
	HAL LSKS
	HAL legs
	HAL arms
	HAL transportation HAL household
Anxiety	RAND‐36 emotional well‐being	All other RAND‐36 subscales (n = 8) All HAL domains	−2.3	NA	Mild‐severe hemophilia No‐yes joint impairment No‐yes HIV infection
Depression	RAND‐36 emotional well‐being	All other RAND‐36 subscales (n = 8) All HAL domains	−3.0	NA	Mild‐severe hemophilia No‐yes joint impairment No‐yes HIV infection
Fatigue	RAND‐36 energy/fatigue	All other RAND‐36 subscales (n = 8) All HAL domains	−2.0	No‐yes HIV infection	Mild‐severe hemophilia No‐yes joint impairment
Sleep Disturbance	NA	All RAND‐36 subscales All HAL domains	−1.0	NA	Mild‐severe hemophilia No‐yes joint impairment No ‐ yes HIV infection
Ability to Participate in Social Roles and Activities	RAND‐36 social functioning RAND‐36 role limitations‐physical RAND‐36 role limitations‐emotional HAL household HAL leisure and sports	All other RAND‐36 subscales (n = 6) All other HAL domains (n = 5)	1.0	No‐yes joint impairment	Mild‐severe hemophilia No‐yes HIV infection
Pain Interference	RAND‐36 physical functioning RAND‐36 pain HAL LSKS HAL legs HAL transportation HAL household	All other RAND‐36 subscales (n = 7) All other HAL domains (n = 3)	−2.0	Mild‐severe hemophilia No‐yes joint impairment	No‐yes HIV infection
Pain Intensity	RAND‐36 Pain	All other RAND‐36 subscales (n = 8) All HAL domains	−1.0	Mild‐severe hemophilia No‐yes joint impairment	No‐yes HIV infection

Abbreviations: HAL, hemophilia activities list; LSKS: lying/sitting/kneeling/standing; MID, minimal important difference; NA, not applicable.

Discriminative validity was assessed by comparing mean T‐scores between relevant clinical groups. Clinical subgroups were defined based on: hemophilia severity (mild compared with severe hemophilia); self‐reported joint impairment in one or more of the six main joints (left and right ankles, knees, elbows; no/yes); and HIV infection (no/yes). Mean differences between mild and severe hemophilia were adjusted for age; mean differences between absent and present joint impairment were adjusted for age and severity using UNIANOVA. The comparison of mean T‐scores for individuals with and without HIV were restricted to those born in 1985 or earlier, because the risk of HIV infection was considered negligible for younger patients. The following differences in mean T‐scores were considered relevant differences between groups, based on published minimally important differences (MID) or changes for other patient groups: ≥2 for Physical Function, ⁴¹ ≥−2.3 for Anxiety, ⁴² ≥−3.0 for Depression, ⁴² ≥−2 for Fatigue, ⁴³ ≥−1 for Sleep Disturbance, ⁴³ ≥1 for Ability to Participate in Social Roles and Activities, ⁴³ ≥−2.0 for Pain Interference, ⁴⁴ and ≥−1 for Pain Intensity. ⁴⁵ Because the MID is specific for each domain, a difference of, for example, 2 points may be a relevant difference in one domain, but not in another. Based on literature ⁴⁶ , ⁴⁷ and clinical experience (authors S.G., M.D.), we expected to find the following relevant differences: between mild and severe hemophilia and between absent and present joint impairment for Physical Function; between not HIV‐infected and HIV‐infected for Fatigue; between absent and present joint impairment for Ability to Participate in Social Roles and Activities; between mild and severe and between absent and present joint impairment for Pain Interference; and for Pain Intensity (Table 1).

All analyses were performed with IBM SPSS version 25, except for CFA, which was performed in R, version 3.6.1 (package “lavaan”).

3. RESULTS

3.1. Participants

Of 1746 Dutch adults with hemophilia who were invited to participate, 808 completed the questionnaires partially or in full (response 46.3%). The final sample for analysis consisted of 770 participants for whom one or more PROMIS‐29 T‐scores were calculated. For 598 of 770 participants (77.7%), clinical data from electronic medical records were available. Mean age was 48.9 (SD 17.2) years. Half of the participants (49.9%) had mild hemophilia, 15.6% had moderate, and 34.5% had severe hemophilia, which is representative of the total Dutch hemophilia population (55.8%, 13.2%, and 30.1%, respectively). Clinical and sociodemographic characteristics are shown in Table 2.

TABLE 2.

Participant characteristics (n = 770)

Clinical Characteristics	N	%
Hemophilia severity a
Mild	384	49.9
Moderate	120	15.6
Severe	266	34.5
Type of hemophilia
Hemophilia A	669	86.9
Hemophilia B	92	11.9
No hemophilia a	3	0.4
Unknown b	6	0.7
Prophylaxis (severe hemophilia)
Yes	233	87.6
No	30	11.3
Missing	3	1.1
HIV infection
Yes	22	2.9
No	721	93.6
Unknown	27	3.5
HCV infection
Never infected	418	54.3
Past infection	231	30.0
Current infection	8	1.0
Past or current infection c	2	0.6
Unknown	111	14.4
Inhibitor
Never	637	82.7
Past	68	8.8
Current	12	1.6
Unknown d	53	6.9
Joint impairment e
Yes	338	43.9
No	379	49.2
Unknown	53	6.9

Demographic characteristics	Mean	SD
Age in years f	48.9	17.2
Education g	N	%
Primary education	44	5.7
Secondary education	397	51.6
Tertiary education	298	38.7
Missing/prefer not to say	31	4.0

Clinical characteristics were taken from electronic medical records if participant had provided informed consent for extraction of data. If electronic medical record data were not available and participants did not complete the questions, status is unknown. Hemophilia severity was available from electronic medical records for all eligible persons (responders and nonresponders).

^a Three participants indicated on the questionnaire that they no longer had hemophilia, which might be because of a liver transplant (n = 1) or participation in a gene therapy trial, but the exact reason is unknown.

^b Five participants did not know their type of hemophilia (A or B), and one person skipped this question. Medical record data were missing for these individuals.

^c Five individuals had a past or current hepatitis C virus infection, but current infection status could not be established.

^d Inhibitor data from the medical record were not available for 53 participants because they did not provide informed consent for extraction of data.

^e Joint impairment was self‐reported chronic joint impairment in any joint (yes‐no).

^f For three participants, age was missing and no electronic medical record was available.

^g Education level was categorized according to International Standard Classification of Education (ISCED) levels: primary education (ISCED level 1), secondary education (ISCED levels 2 and 3), tertiary education (ISCED levels 6 and 7).

3.2. Description of measures

Table 3 shows mean, minimum, and maximum scores, SDs, floor and ceiling effects, and percentage of missing scores of all measures from the questionnaires. Mean T‐scores for PROMIS‐29 were better than the US general population average for all subscales except Physical Function, which was worse (48.9). Distributions of all PROMIS‐29 domain scores were skewed toward better scores (i.e., scores >50 for the subscales Physical Function and Ability to Participate in Social Roles and Activities, and <50 for all other subscales) (Figure 1). Five of seven PROMIS‐29 subscales and Pain Intensity showed substantial ceiling effects of >30% patients with the best scores, whereas this was the case for five of eight RAND subscales and for all HAL‐domains. PROMIS‐29 had fewer missing answers than RAND‐36 and HAL.

TABLE 3.

Characteristics of PROMIS‐29, RAND‐36, and HAL for adult men with hemophilia

	N a	Mean (SD) b	Range (min‐max)	Worst Score (%) c	Best Score (%) c	Missing (%) d
PROMIS‐29
Physical Function	765	48.9 (9.6)	22.9–56.9	1.3	51.9	0.6
Anxiety	744	48.0 (8.2)	40.3–81.4	0.1	43.2	3.4
Depression	744	46.4 (7.8)	41.0–79.3	0.3	59.1	3.4
Fatigue	738	46.6 (9.6)	33.7–75.8	0.5	21.0	4.2
Sleep Disturbance	738	46.5 (7.9)	32.0–73.3	0.3	5.6	4.2
Ability to Participate in Social Roles and Activities	729	54.2 (8.9)	27.5–64.2	0.6	30.6	5.3
Pain Interference	726	49.6 (9.0)	41.6–75.6	0.6	47.4	5.7
Pain Intensity	724	2.4 (2.5)	0–10	0.1	31.6	6.0
RAND‐36
Physical functioning	734	77.9 (27.4)	0–100	0.8	31.9	2.3
Social functioning	705	83.5 (20.7)	0–100	0.5	43.0	8.4
Role limitations‐physical	710	76.5 (37.5)	0–100	13.1	61.7	7.7
Role limitations‐emotional	702	84.9 (31.6)	0–100	8.1	71.8	8.7
Emotional well‐being	698	77.2 (15.6)	0–100	0.1	3.6	9.2
Energy/fatigue	698	64.7 (17.8)	0–100	0.3	1.2	9.1
Pain	698	77.4 (22.5)	0–100	0.5	31.6	9.0
General health perceptions	694	64.5 (22.3)	0–100	0.6	4.3	0.0
Change in health	763	50.4 (19.8)	0–100	2.7	4.8	0.9
HAL
Lying/sitting/kneeling/standing	709	77.6 (26.5)	7.5–100	0.0	37.3	7.1
Functions of the legs	694	74.0 (31.3)	0–100	1.6	38.8	9.1
Functions of the arms	688	83.9 (24.5)	0–100	0.6	50.9	10.3
Use of transportation	680	85.8 (24.7)	0–100	0.4	55.6	11.6
Self‐care	681	90.8 (18.3)	5–100	0.0	59.0	11.4
Household tasks	647	87.4 (21.8)	0–100	0.4	51.7	12.5
Leisure activities and sports	614	82.0 (24.9)	0–100	0.5	39.1	13.1

Abbreviations: HAL, Hemophilia Activities List; PROMIS, Patient‐Reported Outcomes Measurement Information System; SD, standard deviation

^a The number of participants for whom a score could be computed as described in the Methods section.

^b Higher scores on RAND‐36 and HAL indicate better health status and better physical functioning, higher scores on PROMIS‐29 indicate more of the construct being measured (e.g., more Physical Function and Ability to Participate in Social Roles and Activities, or more Anxiety, Depression, Fatigue, Sleep Disturbance and Pain).

^c Worst and best possible scores were calculated if at least one item had been completed. Floor and ceiling effects are defined as the percentage of participants with the worst and the best scores possible. Floor and ceiling effects are considered present if >30% (in bold).

^d Percentage of participants for whom all items on a domain are missing.

FIGURE 1.

Distribution of T‐sc frequencies of T‐scores for each PROMIS domain, and level of pain for Pain Intensity. The black curve indicates the normal distribution based on the frequencies. A higher score indicates more of the construct being measured. PROMIS, Patient‐Reported Outcomes Measurement Information System

3.3. Structural validity

PROMIS‐29 showed sufficient CFA model fit (CFI or TLI > 0.95, or RMSEA < 0.06) for Physical Function (CFI: 0.95, TLI: 0.85, RMSEA: 0.13), Depression (CFI: 1.00, TLI: 0.99, RMSEA: 0.02), Sleep Disturbance (CFI: 0.94, TLI: 0.82, RMSEA: 0.05), Ability to Participate in Social Roles and Activities (CFI: 1.00, TLI: 1.00, RMSEA: 0.00), and Pain Interference (CFI: 0.99, TLI: 0.98, RMSEA: 0.05). The subscales Anxiety and Fatigue did not show sufficient model fit (Table 4).

TABLE 4.

Structural validity and internal consistency of PROMIS‐29

	N	CFI	TLI	RMSEA	Cronbach's Alpha
PROMIS‐29
Physical Function	752	0.95	0.85	0.13	0.94
Anxiety	735	0.88	0.63	0.15	‐
Depression	727	1.00	0.99	0.02	0.93
Fatigue	728	0.85	0.56	0.24	‐
Sleep Disturbance	713	0.94	0.82	0.05	0.79
Ability to Participate in Social Roles and Activities	717	1.00	1.00	0.00	0.93
Pain Interference	715	0.99	0.98	0.05	0.96

Good internal consistency is defined as Cronbach's alpha ≥ 0.70. Fit parameters were rounded to two decimal places.

Abbreviations: CFI, comparative fit index; RMSEA, root mean square error of approximation, Sufficient fit; TLI, Tucker‐Lewis Index; indicated in bold: CFI or TLI > 0.95, or RMSEA < 0.06.

3.4. Internal consistency

Internal consistency was sufficient (Cronbach's alphas ≥ 0.70) for all five PROMIS‐29 subscales with sufficient model fit in CFA. For four of them, Cronbach's alphas were ≥0.90: Physical Function, Depression, Ability to Participate in Social Roles and Activities, and Pain Interference (Table 4). No Cronbach's alphas were calculated for Anxiety and Fatigue because model fit was not sufficient.

3.5. Construct validity

Results for convergent validity are shown in Table 5. For the subscales Anxiety, Depression, Fatigue, Sleep Disturbance, and Pain Intensity, all correlations were in accordance with the hypotheses for convergent validity. For the subscales Physical Function, 12 of 16 correlations were as hypothesized, whereas for Ability to Participate in Social Roles and Activities, this was the case for 11 of 16 correlations. Nine of 16 correlations were in accordance with the hypotheses for Pain Interference.

TABLE 5.

Pearson's r for correlations between RAND‐36 and PROMIS‐29 subscales (convergent validity)

		PROMIS−29
		Physical Function	Anxiety	Depression	Fatigue	Sleep Disturbance	Ability to Participate	Pain Interference	Pain Intensity
RAND‐36	Physical functioning	0.91	−0.31	−0.36	−0.37	−0.28	0.59	−0.70	−0.59
	Social functioning	0.52	−0.57	−0.60	−0.58	−0.42	0.70	−0.59	−0.53
	Role limitations‐physical	0.63	−0.36	−0.40	−0.48	−0.32	0.62	−0.66	−0.57
	Role limitations‐emotional	0.33	−0.48	−0.56	−0.44	−0.37	0.50	−0.37	−0.33
	Emotional well‐being	0.28	−0.74	−0.75	−0.55	−0.50	0.56	−0.35	−0.33
	Energy/fatigue	0.40	−0.55	−0.59	−0.72	−0.50	0.60	−0.46	−0.42
	Pain	0.63	−0.33	−0.38	−0.46	−0.31	0.55	−0.82	−0.80
	General health perceptions	0.59	−0.46	−0.47	−0.54	−0.43	0.62	−0.63	−0.55
	Change in health	0.29	−0.14	−0.16	−0.17	−0.11	0.27	−0.28	−0.28
HAL	Lying/sitting/kneeling/standing	0.79	−0.24	−0.27	−0.29	−0.28	0.53	−0.63	−0.53
	Functions of the legs	0.85	−0.23	−0.28	−0.28	−0.25	0.53	−0.65	−0.56
	Functions of the arms	0.73	−0.30	−0.31	−0.35	−0.28	0.56	−0.64	−0.54
	Use of transportation	0.77	−0.25	−0.30	−0.30	−0.25	0.54	−0.58	−0.48
	Self‐care	0.66	−0.30	−0.32	−0.33	−0.27	0.54	−0.60	−0.53
	Household tasks	0.80	−0.31	−0.35	−0.36	−0.29	0.60	−0.70	−0.56
	Leisure activities and sports	0.80	−0.29	−0.34	−0.36	−0.29	0.60	−0.70	−0.57
Hypotheses confirmed (%)		75	100	100	100	100	69	56	100

Correlations in bold were expected to be ≥0.70 or ≥ −0.70. All other correlations were expected to be ≤0.60.

Abbreviations: HAL, Hemophilia Activities List; PROMIS, Patient‐Reported Outcomes Measurement Information System.

Unadjusted and adjusted differences in mean T‐scores between clinical groups (discriminative validity) are shown in Table 6. All differences between groups were in the expected direction (i.e., participants with mild hemophilia, no joint damage, and no HIV infection had better scores for all subscales). Adjusting for age resulted in a larger difference between mild and severe hemophilia, and adjusting for age and disease severity resulted in smaller differences between individuals with and without joint impairment. Finally, differences became smaller when HIV‐infected participants were compared with noninfected participants with severe hemophilia born in or before 1985.

TABLE 6.

Differences in mean PROMIS‐29 T‐scores for clinical subgroups (discriminative validity)

	MID	Severe‐mild Hemophilia		Yes‐no joint Impairment		Yes–no HIV Infection		Hypotheses Confirmed (%)d
		Unadjusted (95% CI)	Adjusted (95% CI)a	Unadjusted (95% CI)	Adjusted (95% CI)b	Unadjusted (95% CI)	Adjusted (95% CI)c
Physical Function	2.0	8.6 (7.0–10.0)	10.2 (9.1–11.4)	10.8 (9.6–11.9)	6.1 (4.8–7.4)	10.5 (6.4–14.6)	1.5 (−1.9 to 4.8)	100
Anxiety	−2.3	−0.4 (−1.7 to 0.9)	−0.8 (−2.1 to 0.5)	−2.5 (−3.7 to −1.3)	−2.7 (−4.2 to −1.2)	−2.6 (−6.2 to 1.0)	−1.7 (−5.8 to 2.3)	67
Depression	−3.0	−1.7 (−2.9 to −0.4)	−2.1 (−3.3 to −0.8)	−2.7 (−3.9 to −1.6)	−1.8 (−3.3 to −0.4)	−3.4 (−6.7 to −0.1)	−1.7 (−5.6 to 2.2)	100
Fatigue	−2.0	−1.9 (−3.4 to −0.3)	−2.2 (−3.7 to −0.6)	−3.6 (−5.0 to −2.1)	−3.5 (−5.3 to −1.7)	−4.7 (−8.8 to −0.7)	−2.5 (−6.9 to 1.9)	33
Sleep Disturbance	−1.0	−1.2 (−2.5 to 0.0)	−1.4 (−2.7 to −0.2)	−2.2 (−3.3 to −1.0)	−1.8 (−3.2 to −0.3)	−3.7 (−7.0 to −0.3)	−2.8 (−6.5 to 1.0)	0
Ability to Participate in Social Roles and Activities	1.0	4.4 (3.0–5.8)	5.4 (4.0–6.7)	5.7 (4.4–7.0)	3.1 (1.6–4.6)	8.7 (5.0–12.5)	3.5 (−0.2 to 7.3)	33
Pain Interference	−2.0	−6.0 (−7.4 to −4.7)	−7.2 (−8.5 to −5.8)	−8.2 (−9.4 to −7.0)	−5.7 (−7.2 to −4.3)	−7.3 (−11.1 to −3.5)	−0.9 (−4.7 to 3.0)	100
Pain Intensity	−1.0	−1.5 (−1.9 to −1.2)	−1.8 (−2.1 to −1.4)	−1.9 (−2.3 to −1.6)	−1.4 (−1.8 to −0.9)	−1.9 (−2.9 to −0.8)	−0.6 (−1.7 to 0.5)	100

Differences in bold were hypothesized to be >MID. For the subscales Physical Function and Ability to Participate in Social Roles and Activities a positive difference means that persons with mild hemophilia, no joint impairment, or no HIV infection have more of these constructs than persons with severe hemophilia, joint impairment, or HIV infection. For the other subscales, negative differences indicate less of these constructs for persons with mild hemophilia, no joint impairment, or no HIV infection.

Abbreviations: 95% CI, 95% confidence interval; MID, minimally important difference; PROMIS, Patient‐Reported Outcomes Measurement Information System.

^a Adjusted for age.

^b Adjusted for age and severity.

^c Mean difference between individuals with severe hemophilia born in 1985 or earlier, with or without HIV.

^d Hypotheses confirmed for discriminative validity.

The evidence for discriminative validity was strongest for Physical Function, Depression, Pain Interference, and Pain Intensity: all differences between subgroups were as hypothesized. For Anxiety, two of three differences between groups were as hypothesized, and for Fatigue and Ability to Participate in Social Roles and Activities one difference was as hypothesized. None of the differences between groups were in accordance with the hypotheses for Sleep Disturbance.

In total, six subscales showed evidence for construct validity (≥75% hypotheses confirmed): Physical Function, (79%), Anxiety (95%), Depression (100%), Fatigue (89%), Sleep Disturbance (84%), and Pain Intensity (100%). Two subscales did not meet the criterium for ≥75% of hypotheses confirmed: for Ability to Participate in Social Roles and Activities and Pain Interference, 63% of hypotheses were confirmed.

Table 7 summarizes the evidence for structural validity, internal consistency, and construct validity.

TABLE 7.

Summary of the evidence for structural validity, internal consistency, and construct validity (convergent and discriminative)

PROMIS‐29 Subscale	Structural Validity	Internal Consistency	Construct Validity
Physical Function	+	+	+
Anxiety	−	0	+
Depression	+	+	+
Fatigue	−	0	+
Sleep Disturbance	+	+	+
Ability to Participate in Social Roles and Activities	+	+	−
Pain Interference	+	+	−
Pain Intensity	NA	NA	+

+, evidence for the measurement property according to prespecified criteria; −, the evidence for the measurement property did not meet prespecified criteria; 0, not assessed because of limited structural validity; NA, measurement property not applicable (1 item).

4. DISCUSSION

This study is the first validation of the Dutch‐Flemish version of the PROMIS Profile‐29, as well as the first validation of this profile among persons with hemophilia. Using consensus‐based standards for evaluating validity, we aimed to assess structural validity, internal consistency, and construct validity of the PROMIS‐29 Profile v2.01 in Dutch adults with hemophilia. In a representative sample of the Dutch hemophilia population, our analyses showed sufficient evidence for structural validity and internal consistency for five of seven subscales and sufficient evidence for construct validity for five subscales and for Pain Intensity.

In the confirmatory factor analysis, model fit was not sufficient for Anxiety and Fatigue, potentially indicating a lack of unidimensionality ⁴⁸ (i.e., that these subscales may measure more than one construct for people with hemophilia). An explanation may be that CFA modelling assumes a normal distribution of the data. Our results, however, showed skewed distributions for all subscales. This may have influenced fit statistics. ⁴⁸ In contrast to our findings, a previous validation of PROMIS‐29 among kidney transplant recipients found excellent structural validity for all subscales, ²³ even with similarly skewed distributions.

We found evidence for sufficient internal consistency for five subscales, with Cronbach's alphas >0.90 for the subscales Physical Function, Depression, Ability to Participate in Social Roles and Activities, and Pain Interference. Consistent with our findings, two previous studies in kidney transplant recipients and populations with rheumatoid arthritis, osteoarthritis, and systemic lupus erythematosus reported similarly high Cronbach's alphas for all subscales. ²¹ , ²³

Overall, the subscales Physical Function, Anxiety, Depression, Fatigue, Sleep Disturbance, and Pain Intensity showed evidence for construct validity (i.e., >75% of results in accordance with the hypotheses). Fewer hypotheses were confirmed for the subscales Ability to Participate in Social Roles and Activities and Pain Interference (63%).

Correlations lower than the expected 0.70 were found for Ability to Participate in Social Roles and Activities with the RAND‐36 role limitations caused by physical or emotional health problems (0.62 and 0.50, respectively). The hypothesis for the former correlation was based on a Dutch study among 30 abdominal surgery patients that reported a correlation of 0.72 between the SF‐36 subscale Role limitations caused by physical health problems and the eight‐item PROMIS Ability to Participate in Social Roles and Activities short form. ⁴⁰ Though the correlation we report is below the 0.70 threshold, it is of the same order of magnitude and the difference may be due to random variation or to differences in the underlying constructs being measured.

Lower correlations were also found between PROMIS‐29 Ability to Participate in Social Roles and Activities with HAL household tasks (0.60) and leisure and sports (0.60). This may mean that these constructs differ more than anticipated, resulting in fewer hypotheses for convergent validity confirmed. Indeed, HAL subscales measure several aspects of self‐perceived functional ability, whereas PROMIS Ability to Participate in Social Roles and Activities measures participation.

Some subscales that were not expected to correlate highly with RAND‐36 and HAL (i.e., expected to be ≤0.60) showed correlations above the threshold of 0.60. This was the case for the correlation between Physical Function with RAND‐36 pain (0.63), RAND‐36 role limitations caused by physical health problems (0.63) and HAL self‐care (0.66), and for Ability to Participate in Social Roles and Activities with RAND‐36 general health perceptions (0.62), and for Pain Interference with RAND‐36 role limitations caused by physical health problems (−0.66), with RAND‐36 general health perceptions (−0.63), and with HAL functions of the arms (−0.64). We used a relatively low expected correlation of ≤0.60 between subscales that do not measure the same construct to distinguish them from the correlations ≥0.70 expected between subscales that measure the same construct, but this resulted in fewer hypotheses confirmed (especially for Ability to Participate in Social Roles and Activities and Pain Interference), and thus lower evidence of construct validity. This strict criterium may have led to quite conservative conclusions.

Also interesting is that most correlations between Pain Interference and HAL subscales were of similar strength, between −0.58 and −0.66. Though below the 0.70 threshold, the subscales perceived functional ability (HAL) ⁵ , ⁶ and Pain Interference with functional ability (PROMIS) ¹⁶ may measure similar constructs after all.

We found unexpected differences larger than the MID for some subscales. For example, differences between all clinical groups were larger than expected for Sleep Disturbance. Sleep Disturbance is not routinely studied in hemophilia, but a qualitative study reported that pain may affect Sleep Disturbance. ⁴⁹ Persons with severe hemophilia, joint impairment, and HIV are more likely to experience pain because of recurrent bleeding, which may explain part of the observed differences. However, confidence intervals of the observed differences were wide. Also, the correlation between PROMIS‐29 Sleep Disturbance and RAND‐36 pain was low (−0.31), making a substantial influence of pain on Sleep Disturbance less likely. Differences between mild and severe hemophilia and for different HIV infection status were also larger than expected for Ability to Participate in Social Roles and Activities, whereas we only expected to find differences for joint impairment. Because effective treatment is available, persons with severe hemophilia should be able to lead near‐normal lives, and for this reason were expected to have similar levels of social participation as individuals with mild hemophilia. Our results indicate that this may not be the case. Indeed, hemophilia is reported to have a negative impact on employment and education, ⁵⁰ and may also have affected the Ability to Participate in Social Roles and Activities. Individuals with HIV infection may have a more severe bleeding phenotype than those without HIV: persons with a more severe bleeding phenotype may have received more plasma‐derived treatment products in the past, and contracted HIV as a result, compared with persons with severe hemophilia with a milder bleeding phenotype. A more severe bleeding phenotype may also have resulted in more joint impairment and lower participation. Unfortunately, we did not have reliable information on bleeding phenotype and were therefore unable to correct for this confounder. The number of individuals with HIV was small (n = 22), resulting in less reliable estimates of T‐scores in this subgroup.

A potential limitation of this study is that the response rate of the HiN‐6 study was limited (46.3%). This may have led to some bias. First, fewer people had only primary education (5.7%) and more had secondary education (51.6%) compared with the general Dutch population (21% and 40%, respectively). ⁵¹ If people with a higher education were better able to manage their hemophilia, this could have resulted in higher scores on PROMIS subscales. This may, in part, explain our finding that mean scores on many PROMIS‐29 subscales were higher than the general population average of 50. Second, persons with more health‐related problems from hemophilia may have been more likely to participate because they were more motivated to complete a questionnaire about their health. This would have resulted in low scores; however, our results showed large proportions of participants with the highest scores on several subscales, indicating few health problems. Therefore, we believe selection bias because of health problems was unlikely to have affected the findings of this study.

Content validity of PROMIS‐29 was reported to be good in several other populations. ¹³ , ¹⁴ Our results also provide some evidence for content validity of PROMIS‐29 among persons with hemophilia: the number of missing answers was low, which may indicate that items were relevant to participants. ⁵² On the other hand, PROMIS‐29 showed large proportions of best scores for most subscales, which may indicate a lack of content validity: best scores may indicate that items were not relevant to measure the domain for this population and that more “difficult” items may be missing. ⁵² The large proportion of best scores on most subscales (except Fatigue and Sleep Disturbance) leads to a loss in measurement precision in well‐functioning individuals. The four‐item short forms that comprise PROMIS‐29 may therefore not be optimal for persons with hemophilia. Because PROMIS item banks are IRT‐based, they are flexible and another selection of items can be considered. For example, a longer or a custom short form with more “difficult” items from the item bank or a CAT may solve these ceiling effects and still yield comparable results. ¹² Unfortunately, Dutch CATs were not available at the time of our study, but have become available recently. ⁵³ , ⁵⁴

In our study, five subscales met all criteria for structural validity and internal consistency and five and Pain Intensity met all the criteria for hypotheses‐testing for construct validity. Small changes in the methods regarding the cutoffs of correlations and the percentage of hypotheses confirmed may have had profound effects on the conclusions.

Other studies that validated PROMIS‐29 in different populations did not formulate hypotheses for construct validity, which may lead to less transparent and less consistent interpretation of the results. ²¹ , ²² , ²³ Yet, hypothesis‐testing for construct validity depends on sufficient knowledge about the constructs being measured with all subscales. However, limited literature was available that quantified correlations with other instruments or differences between groups. Despite the lack of explicit hypotheses in other studies, the magnitude of differences between relevant subgroups is similar. ²¹ , ²² , ²³ This indicates generalizability across diseases.

Ideally, PROMs are used that measure the most relevant outcomes for a specific population. A consensus‐based standard set of relevant outcomes for persons with hemophilia was published recently, ⁵⁵ along with recommendations for instruments to measure these outcomes. The set included the five PROs ability to engage in normal daily activities, chronic pain, sustainability of physical functioning, social functioning, and mental health. The latter four can be measured with the PROMIS Profile‐29 subscales that were validated in the current study: Pain Interference and Pain Intensity; Physical Function; Ability to Participate in Social Roles and Activities; and Anxiety and Depression, respectively. For an even more comprehensive assessment, social functioning may be measured with the PROMIS domain self‐efficacy for managing social interactions, and mental health with the subscales general life satisfaction and positive affect. Ability to engage in normal daily activities may be measured with PROMIS self‐efficacy for managing chronic conditions ‐ managing daily activities. PROMIS item banks or short forms for these subscales may be validated for comprehensive assessment of the standard set of outcomes for hemophilia. The standard set of outcomes did not include the domains Fatigue and Sleep Disturbance, which may not need to be prioritized for measurement, though they may still be important in some patients or certain situations.

Which tools to use (disease‐specific or generic) depends on the goal of measuring outcomes and the type of outcomes. Some outcomes, such as degree of hemophilic arthropathy, are disease‐specific and need to be assessed with disease‐specific instruments. Functional outcomes such as those measured with PROMIS item banks (e.g., Physical Function, Fatigue) are of a more generic nature. For clinical care aimed at improving outcomes, generic tools may be the most suitable, whereas in other cases disease‐specific tools may be necessary. Still, in many cases, a combination of generic tools where possible, supplemented with disease‐specific tools where needed, may be the most suitable for comprehensive measurement of all outcomes that are relevant for hemophilia.

5. CONCLUSION

This study found sufficient evidence for structural validity, internal consistency, and construct validity of the PROMIS‐29 subscales Physical Function, Depression, and Sleep Disturbance in adult persons with hemophilia in the Netherlands. Construct validity was also sufficient for Anxiety, Fatigue, and Pain Intensity. These results indicate that PROMIS short forms that measure these domains may be used in hemophilia populations. Future studies should explore whether the use of custom short forms or CATs can solve observed ceiling effects.

CONFLICT OF INTEREST

E.C. van Balen, L. Haverman, S. Hassan, E.M. Taal, C. Smit, M.H. Driessens, E.A.M. Beckers, H.L. Hooimeijer, S.E.M. Schols, C.B. Terwee, and F.R. Rosendaal have no conflicts of interest to disclose. M. Coppens has received financial support for research from Bayer, CSL Behring, Daiichi Sankyo, Portola/Alexion, Roche, Sanquin Blood Supply, and UniQure and consultancy or lecturing fees from Bayer, CSL Behring, Medcon International, MEDtalks, NovoNordisk, Pfizer, and Sobi. J. Eikenboom received research support from CSL Behring and has been a teacher on educational activities of Roche. F.W.G. Leebeek received unrestricted research grants from CSL Behring, Takeda, UniQure and Sobi, is a consultant for UniQure, Novo Nordisk, Biomarin, and Takeda, of which the fees go to the institution, and has received a travel grant from Sobi. L.F.D. van Vulpen received a research grant form CSL Behring, is a consultant for Sobi and Tremeau; all fees go to the institution. He is also a DSMB member for a study by Roche. J.G. van der Bom has been a teacher on the educational activities of Bayer. S.C. Gouw has received unrestricted research grants from Sobi.

AUTHOR CONTRIBUTIONS

Erna C. van Balen wrote the analysis plan, conducted the analyses with the help of Elisabeth M. Taal, and wrote the manuscript. Lotte Haverman, Caroline B. Terwee, Johanna G. van der Bom, and Samantha C. Gouw provided critical comments on design and analysis of the study. All authors provided written feedback and approved the final manuscript.

van Balen EC, Haverman L, Hassan S, et al. Validation of PROMIS Profile‐29 in adults with hemophilia in the Netherlands. J Thromb Haemost. 2021;19:2687–2701. 10.1111/jth.15454