Inter-rater reliability of a wright respirometer to measure vital capacity in neuromuscular disorders

Abstract

[Purpose] We examined the inter-rater reliability of the Wright respirometer between therapists who assessed the vital capacity of patients with neuromuscular disorders. [Participants and Methods] We examined 18 patients with neuromuscular disorders. We performed a test-retest method after measuring their vital capacities. Two physical therapists experienced in the use of the Wright respirometer, specifically, therapist A with six years of clinical experience and therapist B with one, each took one measurement of the vital capacity of the same patients. The measurements between the therapists were taken at intervals of 3–7 days. We made a manual to standardize the measurements between the therapists. [Results] The vital capacities were 905 ± 490 mL for therapist A and 897 ± 483 mL for therapist B. The inter-rater reliability of ICC_2,1 was 0.96 (95% CI: 0.89–0.98). Bland-Altman analyses revealed neither a fixed nor proportional bias. [Conclusion] These results demonstrated good to excellent inter-rater reliability of the Wright respirometer for patients with neuromuscular disorders using a manual and instructions.

INTRODUCTION

Neuromuscular disorders (NMD) are characterized by the progressive weakening of skeletal, respiratory, and/or bulbar-innervated muscles^1,2,3). Patients with NMDs have decrements in vital capacity (VC), lung and chest wall compliance, and coughing capacity⁴⁾. The pulmonary rehabilitation of patients with NMDs is designed to maintain lung and thorax mobility and ameliorate the elasticity reduction that are associated with reduced VC⁵⁾. Pulmonary rehabilitation for NMD patients often includes the introduction of maximum insufflation capacity (MIC) instead of deep inspiration to maintain the patients’ lung and thorax function from an early stage⁶⁾. The MIC is attained when the patient takes a deep breath and holds it in; the air is then stacked using a bag-valve mask device⁷⁾. Measurements of the VC and MIC in the pulmonary rehabilitation of patients with NMD are important to accurately grasp the progress of their disease.

NMD guidelines for respiratory management recommend measuring the MIC when the patient’s %VC is <50%⁸⁾. For measurements of the MIC and VC in clinical settings, a Wright respirometer is frequently used⁹⁾. The principle of the Wright respirometer is that the air flow hits the impeller and the volume is measured by the number of rotations¹⁰⁾. The Wright respirometer can be used for NMD patients who cannot hold the mouthpiece of a spirometer in their mouth. When a Wright respirometer is used clinically for a patient, a face mask is attached to the respirometer for measurement. However, this evaluation may involve an air leak from the face mask. The reliability of the Wright respirometer had not been well studied. In a previous study, we investigated the intra-rater reliability of the Wright respirometer to determine the variability of the results obtained between measurements taken by a single physical therapist¹¹⁾, and we observed good-to-excellent results. The inter-rater reliability of the Wright respirometer for patients with NMDs has not been known.

Our hypothesis was that the Wright respirometer measurements would show high inter-rater reliability among examiners. We conducted the present study to examine the inter-rater reliability of the Wright respirometer between therapists assessing the VC of patients with NMDs.

PARTICIPANTS AND METHODS

The participants were NMD patients who were admitted to our hospital or outpatient clinic. The eligibility criteria were (1) diagnosed with NMD, (2) VC <2,000 mL, and (3) %VC <50%. The exclusion criteria were (1) tracheotomy and (2) patients judged to be unable to voluntarily control their respiratory movements. The study protocol was approved by the institutional review board of National Hospital Organization Miyazaki Higashi Hospital (approval no. 25-13). All patients signed an informed consent form.

We performed a test-retest method after measuring each patient’s VC. Two physical therapists experienced in the use of the Wright respirometer, i.e., therapist A (sixth year of clinical experience) and therapist B (first year of clinical experience) each took one measurement of the VC of the same patients. The measurement sequence of therapists A and B was randomized using a random number table. The measurements between the therapists were taken at intervals of 3–7 days. We used the Wright respirometer (Halo/scale Wright respirometer standard type, nSpire Health, Hertford, UK), with a silicone face mask (No.4/5 + Multifunctional mask cover, Laerdal Medical Japan, Tokyo) (Fig. 1). First, the measurement procedure and precautions regarding the measurement method were explained to the patient. The patient then practiced a few times with the respirometer, and after confirming that there was no problem in the calling method, the VC measurement was started. We created a manual to standardize the measurements among therapists¹¹⁾ (Fig. 2). A set of standard instructions was given to all patients. The patient’s posture during the measurements was in a wheelchair sitting position in all cases. The measurement location was a rehabilitation room or at the patient’s bedside. The patients were not informed of the measured values.

Fig. 1.

Wright respirometer.

Fig. 2.

Voice credit at the time of the vital capacity (VC) measurements used in this study.

All measurements took the following four points into account. First, the patient should have no fever or abnormal vital signs. Second, measurements should be in the same environment and at the same time of day as much as possible. Third, we did not implement inspiratory muscle training, which causes fatigue in the respiratory muscles. Fourth, the patient had had enough rest between the measurements.

We analyzed the relative and absolute inter-rater reliability using the intraclass correlation coefficient (ICC_2,1) with 95% confidence intervals (95% CIs). We conducted Bland-Altman analyses¹²⁾ to identify systematic bias between the two therapists. We investigated the measurement variability by calculating the standard error of measurement (SEM) and the 95% CI of the minimal detectable change (MDC). The MDC exhibits the smallest change that indicates a real improvement for a single individual. The ICC criteria were as follows: ≥0.9 as ‘excellent’, ≥0.8 and <0.9 as ‘good’, ≥0.7 and <0.8 as ‘normal’, ≥0.6 and <0.7 as ‘possible’, and <0.6 as ‘reconsidered’^{13, 14)}. The significance level was <5% on both sides in all analyses. All statistical analyses were done with R 2.8.1 (CRAN, free software).

RESULTS

We examined 18 patients (age median, range: 51, 14–83 years; 12 females, 6 males; height 151.8 ± 7.2 cm; weight 45.1 ± 12.0 kg; body mass index [BMI] 19.5 ± 4.5 kg/m²) with NMDs including myotonic dystrophy (MD) (n=6), Duchenne muscular dystrophy (DMD) (n=3), limb girdle muscular dystrophy (LGMD) (n=2), progressive muscular dystrophy (n=2), myopathy (n=2), Fukuyama congenital muscular dystrophy (FCMD) (n=1), facioscapulohumeral muscular dystrophy (FSHD) (n=1), and Gene carriers of DMD (n=1) (Table 1). Eleven patients were fitted with noninvasive positive pressure ventilation (NPPV), and the median wearing time (interquartile range) was 24 (9–24) hr. The measured VC values were 905 ± 490 mL by therapist A and 897 ± 483 mL by therapist B. The inter-rater reliability of ICC_2,1 was 0.96 (95% CI: 0.89–0.98) (Table 2). The Bland-Altman analyses revealed neither fixed bias nor proportional bias (Fig. 3). The values for SEM and MDC between therapists A and B were 102 mL and 282 mL, respectively.

Table 1. Characteristics of the 18 patients with neuromuscular disorders.

No.	Category	Disease	Age (years)	Gender	Height (cm)	Weight (kg)	BMI (kg/m2)	Stage	NPPV time (hr)	Vital capacity (mL)		Sequence (random)
										Examiner A	Examiner B
1	Hospitalization	MD	62	female	143.0	43.1	21.1	Ⅶ	9	350	440	A→B
2	Hospitalization	PMD	65	female	151.0	37.4	16.4	Ⅶ	0	1,110	1,120	A→B
3	Outpatient	DMD	14	male	135.0	36.3	19.9	Ⅶ	9	1,220	1,500	B→A
4	Hospitalization	Gene carriers of DMD	41	female	150.0	39.7	17.6	Ⅶ	24	320	330	B→A
5	Hospitalization	MD	52	female	150.0	57.6	25.6	Ⅶ	24	1,000	1,030	B→A
6	Hospitalization	MD	46	female	153.8	58.5	24.7	Ⅶ	20	730	880	B→A
7	Outpatient	FCMD	15	female	140.0	40.0	20.4	Ⅶ	2	1,260	1,210	B→A
8	Hospitalization	DMD	32	male	158.0	33.2	13.3	Ⅶ	24	160	220	B→A
9	Outpatient	FSHD	58	female	154.2	43.7	18.4	Ⅳ	0	1,900	1,750	A→B
10	Outpatient	LGMD	54	female	160.0	65.6	25.6	Ⅶ	0	1,130	1,210	B→A
11	Hospitalization	MD	60	male	152.7	30.8	13.2	Ⅶ	24	670	550	A→B
12	Outpatient	DMD	31	male	160.0	70.2	27.4	Ⅶ	24	320	310	A→B
13	Hospitalization	MD	59	male	151.5	38.9	17.0	Ⅶ	0	1,200	850	B→A
14	Outpatient	MD	50	female	158.0	58.0	23.2	Ⅴ	9	790	790	A→B
15	Outpatient	LGMD	28	female	146.3	40.8	19.1	Ⅰ	0	1,870	1,900	A→B
16	Hospitalization	Myopathy	34	female	154.0	28.9	12.2	Ⅶ	12	840	720	A→B
17	Hospitalization	PMD	83	female	152.0	47.8	20.7	Ⅶ	0	600	720	B→A
18	Hospitalization	Myopathy	61	male	163.0	40.8	15.4	Ⅶ	0	820	620	A→B

BMI: Body mass index; stage: Ministry of Health and Welfare research group functional classification; NPPV: noninvasive positive pressure ventilation; MD: myotonic dystrophy; PMD: progressive muscular dystrophy; DMD: Duchenne muscular dystrophy; FCMD: Fukuyama congenital muscular dystrophy; LGMD: Limb girdle muscular dystrophy; FSHD: facioscapulohumeral muscular dystrophy.

Table 2. The inter-rater reliability of the vital capacity measurement between therapists A and B.

	Bland-Altman analysis
ICC2,1 (95% CI)	Fixed bias			Proportional bias		SEM (mL)	MDC (mL)
	95% CI of the d	p value		Slope of regression line	p value
0.96 (0.89–0.98)	−79 to 64	p=0.82		−0.02	p=0.84	102	282

ICC: intraclass correlation coefficients; CI: confidence interval; d: the mean of difference between the two therapists; SEM: standard error of measurement; MDC: 95% confidence interval of the minimal detectable change.

Fig. 3.

The Bland-Altman plots showing the differences between measures of VC (mL) from the two therapists’ sessions. The upper and lower horizontal lines represent the mean of the differences and 95% limits of agreement, respectively.

DISCUSSION

The results of the analyses support our hypothesis, i.e., that measurements of NMD patients’ VC by the Wright respirometer would show high inter-rater reliability. The ICC of this study was 0.89–0.98. Our present results were thus good-to-excellent. In our previous investigation, we observed that the intra-rater reliability of the ICC by the same method as that used herein was nearly the same at 0.86–0.98¹¹⁾. On the other hand, another research group reported that the ICC of VC measurements taken with an electronic spirometer was 0.98¹⁵⁾. Compared to that study, the reliability of the Wright respirometer observed herein is slightly inferior to that of an electronic spirometer.

The results of the present Bland-Altman analysis did not reveal systematic bias, i.e., fixed bias and proportional bias. According to the Japanese Society of Respiratory Science spirometry handbook, pulmonary function measurements are to be repeated at least twice, ideally three times, until the difference is within 100 mL or within 5%¹⁶⁾. In the present investigation, the SEM was 102 mL. Since our Wright respirometer measurement method’s SEM is nearly within the standard value, the method can be used for evaluations among multiple therapists. However, it is desirable for the measurer to take multiple measurements in order to reduce the SEM. The MDC in this study was 282 mL. We also reported that the intra-examiner reliability MDC was 233 mL¹¹⁾. It is necessary to judge the inter-rater reliability by estimating the clinical effect more strictly than the intra-rater reliability.

The strength of this study is that the creation of a unified manual gives a high degree of reliability among therapists (even first-year physical therapists). The small and lightweight Wright respirometer does not require electricity, and it is thus not easily affected by the environment. The evaluation method described herein is very useful for NMD patients living at home because it can be conducted in a short time and easily.

This study has two limitations. First, in the present measurement method, the instructions/script (Fig. 2) is performed only under a standardized condition, and it is not clear whether the reliability actually decreases without the instructions/script. Second, the MDC in this study is a statistical judgment (by a distribution-based method), and in the future it will be necessary to consider including the anchor-based method.

In conclusion, our findings demonstrated the good-to-excellent inter-rater reliability in the use of the Wright respirometer for NMD patients along with the use of our new manual and the instructions.

Co nference presentation

We presented this study for the World Confederation for Physical Therapy Congress 2015 in Singapore.

Co nflict of interest

The authors declare that there is no conflict of interest.