Reliability and validity of Chinese version of a tool to assess the quality of life in idiopathic pulmonary fibrosis in patients with interstitial lung disease

Abstract

Objective

This paper aims to determine the reliability and validity of the Chinese version of a tool that assesses the quality of life in idiopathic pulmonary fibrosis (cATAQ-IPF) in patients with interstitial lung disease (ILD).

Methods

We used the process of scale introduction to establish cATAQ-IPF. The content validity of the scale was evaluated by six experts. A total of 92 patients with ILD completed the cATAQ-IPF, St. George's respiratory questionnaire (SGRQ), and The Medical Research Council dyspnoea scale at the baseline, and 15 patients completed cATAQ-IPF at the follow-up period 2 weeks later. Thus, yielding data were used to assess various psychometric properties of cATAQ-IPF. Intraclass correlation coefficient (ICC), Cronbach's α coefficient, content validity index (CVI), item-level CVI (I-CVI), Pearson's coefficients, criterion-relation validity, and known-group validity were used for data analysis.

Results

The cATAQ-IPF showed excellent test–retest reliability (ICC = 0.95), except for the therapy domain (Cronbach's α = 0.60) and acceptable internal consistency (Cronbach's α = 0.96 for the total). The scale-level CVI was 0.80, and the I-CVI was in the range of 0.78–1.00. The total cATAQ-IPF score was strongly correlated with the SGRQ total score (r = 0.71, P < 0.01). The cATAQ-IPF score of patients with ILD was 250.74 ± 47.39, and that of patients with IPF was 287.90 ± 22.56. Patients with IPF possessed considerable impairments in health-related quality of life according to the cATAQ-IPF score (t = 4.94, P < 0.01).

Conclusions

The cATAQ-IPF is a reliable and valid instrument for the evaluation of quality of life of Chinese patients with various forms of ILD.

Highlights

• •Translation from English to Chinese by two individual translators.
• •Retranslation from Chinese into English and reconciliation to obtain Chinese version.
• •Questionnaire survey of Chinese patients with ILD and a follow-up interview regarding the appropriateness of expression in the Chinese questionnaire.

1. Introduction

Interstitial lung diseases (ILDs), particularly idiopathic pulmonary fibrosis (IPF) that is one of the most common forms, are chronic, progressive conditions with remarkable morbidity and mortality. The median survival for IPF ranges from 3 years to 5 years, with half of mortality resulting from progressive respiratory failure [1,2]. Regardless of the cause, as ILD progresses, patients become severely limited in their activities and dependent on supplemental oxygen [3]. Fatigue, dyspnoea, and cough, which are disabling and impair health-related quality of life (HRQL) are the main symptoms of ILD [4]. Thus, health status and HRQL should be measured to assess the effect of disease and treatment efficacy [5].

Several instruments, such as St. George's respiratory questionnaire (SGRQ) and The Medical Outcomes Short Form 36 (SF-36), have been used to measure HRQL in patients with ILD. However, instruments that have been developed specifically for patients with ILD are lacking. Therefore, developing a scale for patients with ILD to measure their HRQL is necessary. In 2010, Swigris [6] developed a questionnaire specifically to assess HRQL in patients with IPF. Questionnaire called a tool to assess quality of life in idiopathic pulmonary fibrosis (ATAQ-IPF) consists of 74 items comprising 13 domains and covers an extensive range of life quality. Although ATAQ-IPF was developed specifically for patients with IPF, it possesses face validity for all ILDs. To our knowledge, the reliability and validity of ATAQ-IPF in Chinese IPF population was demonstrated in 2013 but have not been administered to Chinese patients with ILD. This study aimed to introduce, administer, and test a Chinese version of ATAQ-IPF (cATAQ-IPF) in Chinese patients with ILD.

2. Methods

2.1. Study design

This work is an instrument translation and validation study. Access to the English version of ATAQ-IPF was granted by its developer (personal communication with Jeffrey J. Swigris). We used scale introduction methodology (translation, reverse translation, and culture adaptation) with input and oversight provided by Chinese ILD experts and Dr. Swigris to introduce cATAQ-IPF. cATAQ-IPF, SGRQ, and the Medical Research Council (MRC) dyspnoea scale were administered to the subjects, and analyses were conducted to assess the psychometric properties of cATAQ-IPF. This research was approved by the Institutional Review Board, and each participant provided a written informed consent [7,8].

2.2. Subjects

All patients with ILD evaluated at a tertiary hospital from January 2014 to August 2015 were considered for inclusion. ILD was diagnosed by integrating data on medical history, chest radiograph or high-resolution computed tomography scan, and transbronchial or surgical lung biopsy according to the published guidelines [9]. Patients with heart, liver, kidney, or mental diseases that precluded data collection or dominated their clinical picture were excluded. The final cohort included 92 subjects, 20 of which were diagnosed with IPF and 72 of which with other ILDs (non-IPF).

2.3. Cross-cultural adaptation

The cATAQ-IPF was produced using forward and back translations, with reconciliation by a panel of experts and cognitive interviewing in a small sample size of patients with ILD. The processes of translation and cross-cultural adaptation were performed in five stages, as follows: forward translation, translation synthesis, translation back to English, expert committee, and pretesting [10] (See Fig. 1).

Fig. 1.

Linguistic validation procedure.

2.4. Linguistic validation

In the medical review of the cATAQ-IPF, the plainest Chinese expressions were selected for every respondent to understand the questionnaire. Changes in the text of the cATAQ-IPF were unnecessary from the viewpoint of adaption to Chinese culture or translation to the Chinese language. In the pilot study, respondents understood the cATAQ-IPF without difficulty.

3. Measures

All questionnaires were completed using paper and pencil in a private room without assistance from a research personnel. The cATAQ-IPF, SGRQ, and MRC dyspnoea scale were completed at the baseline.

3.1. The cATAQ-IPF

The English version was developed by Swigris [6] and is composed of 74 items, comprising 13 domains, as follows: cough, dyspnoea, forethought, sleep, mortality, exhaustion, emotional well-being, social participation, finances, independence, sexual health, relationships, and therapies. All items used a five-point Likert response format from strongly disagree to strongly agree. The total score is in the range of 74–370, and high scores correspond to considerable impairment.

3.2. SGRQ

SGRQ is a respiratory-specific instrument developed for patients with COPD and widely used standardized assessment tool with three components, that is, symptoms, activity, and impacts. Scores for each component and a total score were calculated from the three components ranging from 0 to 100, with high scores indicating considerable disability [11]. Although designed for patients with obstructive disease, SGRQ is also a valid HRQL measurement tool for patients with restrictive lung disease [12].

3.3. MRC dyspnoea scale

The MRC dyspnoea scale has been in use for many years for grading the effect of breathlessness on daily activities. This scale measures perceived respiratory disability. It is consists of five statements on perceived breathlessness (scores are in the range of 1–5, with high scores indicating considerable dyspnoea). For the purposes of this study, scores of 1 or 2 were classified as mild, and scores of 3–5 were classified as moderate to severe.

4. Statistical analysis

4.1. Reliability

For test–retest reliability, a randomly selected group of 15 subjects completed the cATAQ-IPF at the baseline. At 2 weeks after completing the cATAQ-IPF, each patient confirmed a subjective stability in their disease prior to the completion of the cATAQ-IPF for the second time in an informal interview. This commonly used time interval was considered long enough for participants to avoid recalling and simply reiterating their baseline responses but also short enough to ensure that their clinical condition has remained stable. Test–retest reliability was examined using the intraclass correlation coefficient (ICC). The internal consistency reliability of the cATAQ-IPF was assessed using Cronbach's α coefficient.

4.2. Validity

Content validity index (CVI) was assessed using scale- (S-CVI) and item-level CVI (I-CVI). To quantify CVI, experts should assess and quantize each item separately; note that each item should be scored between 1 and 4 (1, irrelevant; 2, unable to assess relevance without item revision or item is in need of such revision for it to be irrelevant; 3, relevant but needs minor revision; 4, very relevant and succinct). I-CVI is determined by the proportion of experts who gave a rating of 3 or 4 for every item, while S-CVI is determined by the proportion of all experts who gave a rating of 3 or 4 for all items [13,14].

Correlation between certain SGRQ and certain cATAQ-IPF scores was examined using Pearson's coefficients. All statistical analyses were performed with SPSS version 18.0. Baseline data are presented as percentages or means with standard deviation as appropriate. We considered P < 0.05 to indicate statistical significance.

5. Results

5.1. Sample characteristics

A total of 92 surveys were administered. The overall response rate was 100%. The sample size was composed of 47.8% male (n = 44), and the average age was 55.80 ± 11.78 years. Most of the participants were diagnosed with ILD for <3 years. Nearly 40% of the participants had moderate to severe dyspnoea according to MRC dyspnoea scale (Table 1).

Table 1.

Characteristics of the participants [n (%)].

Characteristics	Total patients (n = 92)	IPF (n = 20)	Non-IPF (n = 72)
Age, Mean ± SD	55.80 ± 11.78	58.25 ± 10.58	55.13 ± 12.07
Gender
Male	44 (47.8)	14 (70.0)	30 (41.7)
Female	48 (52.2)	6 (30.0)	42 (58.3)
Smoking status
Smoked	30 (32.6)	10 (50)	20 (72.2)
Never	62 (67.4)	10 (50)	52 (27.8)
Disease course
≤3 years	74 (80.4)	12 (60)	62 (86.1)
>3 years	18 (19.6)	8 (40)	10 (13.9)
MRC grades
0	4 (4.3)	0 (0)	4 (5.6)
1	21 (22.8)	1 (5)	20 (27.6)
2	31 (33.7)	7 (35)	24 (33.3)
3	23 (25.0)	7 (35)	16 (22.2)
4	13 (14.1)	5 (25)	8 (11.2)
SGRQ score, Mean ± SD
Symptoms	64.37 ± 26.89	80.25 ± 18.97	59.96 ± 27.20
Activity	70.20 ± 24.50	85.45 ± 14.77	65.96 ± 25.04
Impact	56.77 ± 24.59	74.85 ± 12.02	51.75 ± 24.87
Total score	62.08 ± 21.75	78.65 ± 10.84	57.47 ± 21.81

5.2. Reliability

Internal consistency and test–retest reliability results for the cATAQ-IPF are listed on Table 2. Internal consistency was 0.96 for the total score, with the dimensions in the range of 0.60–0.95. Only the therapy domain was lower than the acceptable 0.70 cut-off, and the 2-week test–retest reliability was 0.950 for the total score, with the dimensions in the range of 0.849–0.984. The corrected item-total correlations were in the range of 0.853–0.923. Most item-total correlations were >0.420 (Table 2). Some corrected item-total correlations were 0.200, 0.248, and 0.267 in the dimensions of finances, relationships, and therapies possibly due to the difference in insurance, culture difference, and medical level between China and the US. Ceiling or floor effects were nonexistent for the total cATAQ-IPF.

Table 2.

Reliability of the cATAQ-IPF in patients with ILD.

Dimension	Items (n)	Cronbach's α	Total dimension	Corrected item-total correlation	Test–retest reliability (ICC)
1. Cough	6	0.95	0.63**	0.853–0.923**	0.946**
2. Dyspnea	6	0.88	0.78**	0.752–0.835**	0.937**
3. Forethought	5	0.92	0.73**	0.839–0.892**	0.941**
4. Sleep	6	0.88	0.65**	0.480–0.597**	0.912**
5. Mortality	6	0.85	0.66**	0.420–0.879**	0.935**
6. Exhaustion	5	0.80	0.82**	0.468–0.868**	0.935**
7. Emotional well-being	7	0.92	0.78**	0.757–0.893**	0.957**
8. Social participation	5	0.73	0.74**	0.520–0.781**	0.849**
9. Finance	6	0.81	0.66**	0.200–0.914**	0.982**
10. Independence	5	0.77	0.78**	0.517–0.838**	0.914**
11. Sexual health	5	0.84	0.24*	0.464–0.828**	0.914**
12. Relationships	6	0.72	0.61**	0.248–0.790**	0.984**
13. Therapies	6	0.60	0.61**	0.267–0.714**	0.887**
Total		0.96		0.853–0.923**	0.950**

Note: **P < 0.01, *P < 0.05.

5.3. Validity

5.3.1. Content validity

The CVI of the cATAQ-IPF was evaluated by six medical experts. The S-CVI was 0.80, and the I-CVI ranged from 0.78 to 1.00.

5.3.2. Criterion-relation validity

The criterion-relation validity of the cATAQ-IPF was acceptable. The correlation coefficient between the total cATAQ-IPF score and the total SGRQ score was 0.709 (P < 0.01). The score of the sexual health dimension was correlated insignificantly with any dimension of the total SGRQ score (Table 3).

Table 3.

Criterion-relation validity of the cATAQ-IPF.

Dimension	Symptoms	Activities	Impacts	Total SGRQ (r)	MRC (r)
1. Cough	0.356**	0.370**	0.523**	0.516**
2. Dyspnea	0.405**	0.539**	0.675**	0.670**	0.609**
3. Forethought	0.408**	0.574**	0.611**	0.645**
4. Sleep	0.192	0.367**	0.421**	0.412**
5. Mortality	0.205*	0.377**	0.431**	0.429**
6. Exhaustion	0.325**	0.490**	0.653**	0.619**
7. Emotional well-being	0.122	0.425**	0.549**	0.487**
8. Social participation	0.426**	0.434**	0.498**	0.523**
9. Finance	0.274**	0.258*	0.366**	0.354**
10. Independence	0.366**	0.386**	0.511**	0.509**
11.Sexual health	0.044	−0.014	0.092	0.052
12. Relationships	0.553**	0.317**	0.330**	0.417**
13. Therapies	0.218*	0.339**	0.490**	0.444**
ATAQ-IPF	0.440**	0.568**	0.719**	0.709**	0.555**

Note: **P < 0.01, *P < 0.05.

5.3.3. Known-group validity

The cATAQ-IPF showed good known-group technique for IPF and non-IPF groups. The total scores of the cough, dyspnoea, forethought, sleep, exhaustion, emotional well-being, and relationship dimensions were significantly different in patients with and without IPF (P < 0.001; Table 4).

Table 4.

Comparison of quality of life in patients with different diseases.

Dimension	IPF (n = 20)	Non-IPF (n = 72)	t value	P-valuea
Total score	287.90 ± 22.56	250.74 ± 47.39	4.938	<0.001
1. Cough	24.70 ± 4.66	17.58 ± 7.80	5.122	<0.001
2. Dyspnea	26.60 ± 3.95	21.76 ± 6.11	3.342	0.001
3. Forethought	22.25 ± 3.40	19.61 ± 5.24	2.126	0.036
4. Sleep	23.15 ± 4.63	19.67 ± 6.80	2.662	0.011
5. Mortality	20.90 ± 4.77	18.43 ± 5.71	1.769	0.080
6. Exhaustion	21.30 ± 2.32	16.71 ± 4.86	5.943	<0.001
7. Emotional well-being	27.55 ± 4.37	24.90 ± 7.39	2.023	0.048
8. Social participation	22.10 ± 2.22	20.39 ± 3.90	1.875	0.064
9. Finance	23.30 ± 3.95	22.57 ± 5.22	0.581	0.563
10. Independence	20.60 ± 3.71	18.88 ± 4.16	1.679	0.097
11. Sexual health	18.15 ± 3.82	16.68 ± 3.01	1.818	0.072
12. Relationships	19.00 ± 2.71	16.04 ± 4.04	3.518	0.001
13. Therapies	18.30 ± 4.00	17.51 ± 4.17	0.753	0.454

Note: ^aP-values from Student's t-tests or correlations as appropriate.

6. Discussion

For chronic, disabling illnesses, HRQL is an important outcome to examine when considering whether therapeutic interventions are beneficial. HRQL instruments can be generic or disease-specific. Although generic instruments can be applied in various clinical settings and also provide useful information for either ill or healthy populations by including items most relevant to patients with a particular disease, condition or disease-specific instruments are likely to be more sensitive than generic instruments to the changes in the status that result from therapeutic interventions.

In the present study, we translated the original ATAQ-IPF version, that is, the only IPF-specific HRQL instrument published to date, into Chinese and found that the cATAQ-IPF was a reliable and valid questionnaire useful in assessing HRQL in Chinese patients with various ILDs, not only IPF. We used linguistic validation to develop the cATAQ-IPF and confirmed that the instrument can be translated and adapted to Chinese patients.

Reliability is a generic term used to indicate both the homogeneity (internal consistency) of items composing a scale and the reproducibility (test–retest reliability) of a scale [15]. The ICC was 0.95, with values for the domains in the range of 0.85–0.98, thereby suggesting that cATAQ-IPF is a stable and reliable instrument in this population.

A cut-off of 0.70 is considered indicative of the acceptable internal consistency and test–retest reliability [16]. The internal consistency of 12 domains met this benchmark, except only for the therapy domain. Therapy domain showed low internal consistency reliability possibly due to the differences in insurance and medical treatment. All domain show significant statistically correlation except sexual health domain. It might be cultural differences between Eastern and Western countries that lead to this. In a study by Yunrong Liu [17], the main problem with administering surveys regarding sex to Chinese people was that the respondents either outright refuse to respond to these items or are unable to answer questions truthfully.

Incorporating the opinions from a committee of experts increases confidence that the cATAQ-IPF covers topics and domains relevant to Chinese patients with ILD. Among CVIs, the most widely used are the CVIs introduced by Hambleton and Martuza [18]; these CVIs include I-CVI and S-CVI, which are used to assess the content validity of each item and the whole scale, respectively. Some studies have shown that when the number of experts is ≥ 6, the I-CVI should be ≥ 0.78 [14]. Davis [15] recommended in his study that the S-CVI should be ≥ 0.8. In the present study, six experts were consulted. The I-CVI ranged from 0.78 to 1.00, and a S-CVI of >0.8 indicated good content validity.

SGRQ was chosen as a criterion variable. Although it was developed for patients with obstructive lung diseases, SGRQ has frequently been used to measure HRQL in patients with IPF and other ILDs [[19], [20], [21], [22]]. Significant correlations in the expected direction between the cATAQ-IPF and SGRQ scores supported the criterion validity of the cATAQ-IPF, while only the moderate strength of those correlations confirmed that each issue provided unique information. A medium correlation between cATAQ-IPF and SGRQ was attributed to the fact that SGRQ mainly described dyspnoea, while ATAQ-IPF possessed 13 dimensions, that making a comprehensive review on the quality of life of patients with ILD.

7. Problems in the scale usage

cATAQ-IPF composed many items, thereby requiring 40 min to complete. However, the advantage of using a multidimensional questionnaire, that is, one that captures all domains relevant to a patient group (e.g., cATAQ-IPF), was that other questionnaires were unnecessary. For instance, a dyspnoea scale was unnecessary because the cATAQ-IPF contains one that was relevant to this disease. A cough questionnaire was unnecessary because the cATAQ-IPF includes a cough scale. Considering cultural differences, although items were translated well, some of these items were inapplicable to Chinese patients (e.g., item 42: “I limit the amount that I travel because I have IPF.“). Most of the subjects were hospitalized; thus, their responses may not reflect Chinese patients with ILD in general. The limitation of this study included the lack of factor analysis for retesting the subscale structure and cultural differences. Factor analysis can be added in the future study. The number of cases in this study was limited because it was difficult to collect. Contrasted-group and known-group validity should be added in the future study. The cATAQ-IPF needs to be revised on the sexual health dimension in the studies in the future, such as qualitative research of sexual health between Eastern and Western countries in ILD.

In conclusion, our study introduced ATAQ-IPF for the first time in China. The cATAQ-IPF demonstrated acceptable test–retest reliability, internal consistency reliability, and aspects of validity among hospitalized patients with ILD. Thus, the cATAQ-IPF should be considered as a useful tool to assess HRQL among Chinese patients with ILD.

Conflicts of interest

The authors have no conflicts of interest to declare.

Funding

The authors are grateful for financial support from the China Medical Board (NO. 10-020-201211).

Ethical approval

Ethical approval from Peking Union Medical College Hospital research ethnics committee (NO. S-K104) has been obtained.

Appendix A. Supplementary data

The following are the Supplementary data to this article: