Intra- and Inter-observer Reliability of Quadriceps Muscle Thickness Measured with Bedside Ultrasonography by Critical Care Physicians

Abstract

Background:

Muscle wasting is common among critically ill patients with sepsis and has a significant effect on clinical outcome. However, appropriate tool for measurement of muscle loss is debatable. Ultrasonography (USG) has been used for objective assessment of quadriceps muscle thickness among these patients; however, there is limited data on its reliability.

Aims and Objective:

This study was aimed to assess the reliability of quadriceps muscle thickness as measured by critical care physicians.

Methodology:

This cross-sectional study included twenty patients with sepsis. Quadriceps muscle thickness was measured on right mid-thigh at a predefined point by two critical care fellows using bedside USG. Intra- and inter-observer reliability of the measurements was assessed by intra-class correlation coefficient (ICC).

Results:

Hundred and twenty quadriceps muscle thickness measurements, three by each of the two critical care fellows, were done in twenty patients with sepsis. First, second, and third measurements (mean ± standard deviation) taken by the first observer (RK) were 35.030 ± 3.546 mm, 35.055 ± 3.307 mm, and 35.245 ± 3.027 mm, respectively. The three values recorded by the second observer (AD) were 35.585 ± 3.746 mm, 35.1 ± 3.006 mm, and 34.89 ± 2.556 mm, respectively. ICC for observer 1 and 2 was 0.925 (95% confidence interval [CI]: 0.851–0.967) and 0.835 (95% CI: 0.689–0.925), respectively. The mean difference of measurement between two observers was 0.082 mm (95% CI: −1.194–1.031). The mean ICC (95% CI) for inter-observer reliability was 0.992 (0.979–0.997); P < 0.001.

Conclusions:

This study shows that ultrasound is a reliable tool for the measurement of quadriceps muscle thickness by critical care physicians with excellent inter- and intra-class reliability.

INTRODUCTION

Loss of muscle mass commonly occurs among critically ill patients admitted in intensive care unit (ICU).[1,2,3] In critically ill patients, loss of muscle mass adversely affects the short-term (prolonged mechanical ventilation, re-intubation, mortality, and increased cost of care) as well as long-term clinical outcomes.[1,2,4] The loss of muscle mass is multifactorial and is attributable to inactivity, altered nutrition, and presence of systemic inflammation.[3,4] All these factors lead to altered muscle metabolism characterized by decreased synthesis and increased degradation of the muscle proteins resulting in decreased muscle mass.[1,3]

Accurate, objective, and reliable evaluation of muscle function is the first step toward development of any preventive or therapeutic strategy for loss of muscle mass in ICU. The Medical Research Council (MRC) score and anthropometry are two most commonly used methods for this purpose. MRC score assessment requires patients’ full alertness and cooperation, which may not be possible in ICU.[5] Similarly, anthropometry is dependent on normal hydration status and is not reliable in ICU patients who have subcutaneous edema due to immobility and poor nutrition.[6] Thus, both of these tools are insensitive and unreliable for evaluation of patients in ICU. There are studies which have shown that muscle thickness can be used as a surrogate of muscle strength.[7,8] Muscle thickness may be measured objectively and accurately, independent of patients’ level of alertness and effort, and serial monitoring allows us to assess for loss of muscle mass. Different techniques which can be used for the measurement of muscle thickness include dual-energy X-ray absorptiometry (DEXA) scan, magnetic resonance imaging (MRI), and computed tomography (CT) scan.[3,4] All these are reliable and accurate; however, it is not always feasible to perform these investigations repeatedly for many patients admitted in the ICU.

Recently, ultrasonography (USG) has been shown to be a promising tool for the assessment of quadriceps muscle thickness.[9,10,11,12] USG is a bedside tool which can be easily performed repeatedly to study muscle thickness and loss of muscle mass. The major disadvantage of USG is that it is operator dependent. Therefore, before using this for such assessments, its reproducibility should be established. Hence, we conducted this study to assess intra- and inter-observer reliability of USG for the measurement of quadriceps muscle thickness in patients admitted with sepsis.

METHODOLOGY

The study was conducted in the department of pulmonary medicine at a tertiary care hospital in North India. Standard guidelines for studies involving human subjects were followed.[13,14] The study protocol was approved by institute's Ethics Committee.

Patients

This study included adult patients (age ≥18 years) admitted in pulmonary medicine services with a diagnosis of sepsis between March and May 2016. The diagnosis of sepsis was based on the criteria suggested by SCCM/ESICM/ACCP/ATS/SIS International Sepsis Definitions Conference.[15] Exclusion criteria were presence of primary neuromuscular diseases (such as myopathy, neuropathy, and stroke), an amputated limb, and refusal to consent.

Equipment

Quadriceps muscle thickness measurements were taken by two critical care fellows (RK and AD). The measurements were done using B-mode USG using 5.0–13.0 MHz linear array probe (VF 13-5) on Siemens ACUSON X300™ Machine.

Muscle thickness measurements

Right thigh was used for all measurements of quadriceps muscle thickness. The procedure adopted for muscle thickness measurement for this study was similar to that previously described by Campbell et al.[6] The measurements were done with the patient lying supine with knee extended and toe facing the roof. The same posture was maintained throughout the USG measurements. A circumferential mark was put at the midway between the tip of the greater trochanter and the lateral joint line of the knee. The linear probe was placed on this circumferential line, perpendicular to the skin and moved along the line till a suitable image was obtained. Then, the point corresponding to the center of the probe was marked with a vertical line. This point was used as the reference point for all subsequent measurements of quadriceps thickness. The skeletal muscle thickness of the quadriceps muscle between the superficial fat–muscle interface and the femur was measured anteriorly [Figure 1]. Minimal pressure was applied during USG measurements. Three measurements were taken by each observer. Both observers were blinded to each other's results.

Figure 1.

Ultrasonography image showing various structures and measurement of muscle thickness

Statistical analysis

Data were analyzed using statistical software SPSS Version 20 (IBM Corp., Armonk, NY) for Windows. Intra- and inter-observer reliability was assessed by intra-class correlation coefficient (ICC) and Bland–Altman approach. All values are reported as mean ± standard deviation (SD); P < 0.05 was considered statistically significant.

RESULTS

Baseline characteristics

The study included twenty patients with sepsis admitted to pulmonary medicine services. The mean (± SD) age of the participants was 52 (±16.23) years. There were 11 (55%) males. APACHE-II score of the study population was 18.65 (±4.44). The underlying diagnoses in these patients were chronic obstructive pulmonary disease (9 [45%]), bronchiectasis (4 [20%]), bronchial asthma (2 [10%]), pneumonia (2 [10%]), and interstitial lung disease, allergic bronchopulmonary aspergillosis, and urinary tract infection each in one patient.

Quadriceps muscle thickness measurement

Each observer (RK and AD) took three ultrasonographic measurements of quadriceps muscle in each patient, totally 120 observations. First, second, and third measurements (mean ± SD) taken by the first observer (RK) were 35.030 ± 3.546 mm, 35.055 ± 3.307 mm, and 35.245 ± 3.027 mm, respectively. The three values recorded by the second observer (AD) were 35.585 ± 3.746 mm, 35.1 ± 3.006 mm, and 34.89 ± 2.556 mm, respectively.

Intra- and inter-observer reliability of measurements

ICC with 95% confidence interval (CI) was calculated to assess intra- and inter-observer reliability of the measurements taken by the observers.

Intra-observer reliability

ICC was excellent when computed for intra-observer reliability for each observer separately. The results are summarized in Table 1. ICC for observer 1 and 2 was 0.925 (95% CI: 0.851–0.967) and 0.835 (95% CI: 0.689–0.925), respectively.

Table 1.

The intraclass correlation coefficient of the measurements for intraobserver reliability

Inter-observer reliability

ICC was also computed to check the results of inter-observer reliability for the measurement of quadriceps muscle thickness using intra-class variability coefficient [Table 2]. The mean ICC (95% CI) for inter-observer reliability was 0.992 (0.979–0.997); P < 0.001. We also used Bland–Altman method to depict inter-observer difference [Figure 2]. The mean difference of measurements between the two observers was –0.082 mm (95% CI: −1.194–1.031).

Table 2.

Interclass correlation coefficient of the measurements of muscle thickness

Figure 2.

Bland–Altman plot showing inter-observer differences. None was outside the limits of agreement. The mean difference was −0.082 (95% confidence interval: −1.194–1.031)

DISCUSSION

This study demonstrated USG as an excellent tool in the hands of critical care physicians for measurement of the quadriceps muscle thickness among patients hospitalized with sepsis. The high intra- and inter-observer reliability indicates that USG has a high potential for use in monitoring of muscle thickness over time.

Sepsis is one of the important risk factors for muscle dysfunction among patients admitted in ICU. Assessment of muscle function in these patients is important as muscle dysfunction leads to delayed weaning. However, evaluation of muscle function among these critically ill patients is challenging. Often, it is not feasible and reliable because of altered mentation, inability to understand the command, reduced effort, sedation, and altered hydration.[1,2,3] USG has the advantage that it is independent of these patient-related factors. Easy bedside availability and noninvasiveness of USG have made this tool useful for this purpose. In addition, USG is not associated with the risk of radiation exposure. However, the USG measurements are operator dependent; therefore, intra- and inter-observer reliability needs to be established before it can be applied for routine use in ICU. We assessed the same in twenty patients with sepsis using quadriceps muscle thickness as index measurement. We used ICC for demonstration of reliability of USG in this setting. ICC is a statistical tool to assess intra- and inter-observer reliability of the measurements when either more than one measurements are taken by single observer or more than one observers are taking measurements. It describes how strongly the multiple measurements resemble to each other. ICC can be graded as excellent (>0.75), good (0.60–0.75), fair (0.40-0.60), and poor (<0.40).[16] In our study, intra-observer ICC for both the observers was graded as excellent (0.925 and 0.835), indicating that the observations were closely related with minimal variations. The variation between two observers (inter-observer variation) was also very small. These results are similar to previously published studies.[17,18] These observations highlight that USG has an excellent intra- and inter-observer reliability in the assessment of quadriceps muscle thickness, hence it is a potentially useful method for objective and accurate measurement.

Previously, two more studies have been published which describe intra- and inter-observer reliability of quadriceps thickness measurement among critically ill patients.[6,18] The first study by Campbell et al. included only nine patients with multiorgan failure and the reliability of USG was not their primary objective.[6] The second study by Sabatino et al. published recently included 34 patients with acute kidney injury.[18] In this study, authors measured rectus femoris and vastus intermedius separately, not whole quadriceps muscle group. We used quadriceps measurement as a whole muscle as sometimes it may not be feasible to assess these two muscle parts separately. They demonstrated excellent ICC for both intra- and inter-observer variation. We included patients with sepsis which is the most important risk factor for muscle wasting. Our study strengthens the evidence regarding the reliability of USG measurement of quadriceps muscle group in this setting. Second, in our study, the measurements of muscle thickness with USG were performed by critical care physicians, not by the radiologist. This observation is important because ICUs are managed primarily by critical care physicians and it is desirable that any proposed diagnostic tool should be useful for them with acceptable reliability. Our study has shown that critical care physicians can use USG for the purpose of serial quadriceps muscle thickness measurement with excellent reliability. Third, in this study, all measurements by both observers were taken independently without being aware of the results of the fellow observer which has led to elimination of bias. High level of ICC despite blinding strengthens the reliability of the study results. Finally, all patients were assessed by both observers, hence all paired measurements were available for comparison.

Our study has few limitations as well. First, there was no “gold standard” used to check the accuracy of the thickness measurement. Measurements using CT scan, MRI, or DEXA scan have been described as “gold standard” for the assessment of muscle thickness; however, many patients admitted in ICU are too sick to be shifted for these investigations.[6,19,20,21] There are studies which have shown that measurements using USG have good correlation with muscle thickness measured by CT, MRI, or DEXA scan.[1,3,19,21] Importantly, USG can be done bedside, hence our study results are relevant in this setting. Second, in our study, there was no radiologist as an observer. Radiologists are considered as expert for USG and measurements taken by them are likely to be more reliable. Hence, it is desirable to compare the results obtained by a nonradiologist with that obtained by a radiologist. Having a dedicated radiologist round the clock is not possible in most settings out of radiology department. In our ICU also, we do not have a dedicated radiologist round the clock and it was not considered feasible to include a radiologist as an independent observer. As in our study, the intra- and inter-observer variations for the measurements were very small, it is unlikely that addition of radiologist as an observer would alter the conclusions significantly. Finally, we did not assess whether USG measurements can be performed by other ICU staff such as nurses, dieticians, or physiotherapists. These team members contribute significantly for day-to-day monitoring of the patients in ICU, and therefore it is desirable (in ideal situation) to assess reliability of their observations. Indeed, there are data which have shown intra- and inter-observer ICC for the quadriceps muscle thickness measurements among various ICU staff (nurses, physiotherapists, research assistants, and dieticians) and physician.[17] We could not do this because of practical reasons.

CONCLUSIONS

This study has shown that USG can be performed by critical care physicians for the measurement of quadriceps muscle thickness with excellent intra- and inter-observer reliability among patients admitted with sepsis.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.