Feasibility and Reliability of Muscle Strength Testing in Critically Ill Children

Abstract

Diagnosing pediatric intensive care unit–acquired weakness (PICU-AW) is challenging. The Medical Research Council (MRC) score is a widely used screening method for muscle weakness in critically ill adults; however, its utility in critically ill children has not been established. Our objective was to determine the feasibility and interobserver reliability of muscle strength testing using MRC score in critically ill children. A prospective observational substudy of critically ill children aged 1 to 17 years and limited to bed rest during the first 48 hours of PICU admission was evaluated with weekly MRC exams independently performed by two clinical raters. MRC exams were attempted on all 33 participants, but could be completed in only 21 (64%), 9 of who (43%) received at least one exam while in the PICU, and in the remaining 12 (57%), MRC exams could only be completed after PICU discharge. Of the 95 attempted MRC exams, 55 (57%) could not be conducted or completed, most commonly due to patient sedation, and inability to comply due to cognitive ability, pain, or noncooperation. The inter-rater reliability for MRC sum score was excellent (intraclass correlation coefficient: 0.87). However, the inter-rater reliability was only moderate when used to determine PICU-AW (Cohen kappa: 0.48). MRC testing in the PICU was not feasible as an early screening tool for muscle weakness in the majority of critically ill children in this study. Further research is needed to find an appropriate screening tool that is both feasible and predicts clinically relevant outcomes in children, such as function and recovery following critical illness.

Introduction

Intensive care unit–acquired weakness (ICU-AW) is an acquired polyneuromyopathy commonly described in critically ill adults.^{1 2} The incidence in this population ranges from 25 to 100% and is associated with increased morbidity as well as mortality, and negatively impacts long-term functional outcomes and quality of life in affected individuals.^{3 4 5 6 7 8} In contrast to the adult literature, there is a paucity of prospective research on pediatric intensive care unit–acquired weakness (PICU-AW) and its functional sequelae in children. The reported incidence of PICU-AW ranges from 2 to 30%.^{9 10} This variation in incidence is due in part to a lack of awareness of this condition and challenges in detecting muscle weakness in critically ill children.¹¹

The Medical Research Council (MRC) score is a reliable and simple clinical assessment tool for grading muscle strength that is widely used in both adult and pediatric populations, including Duchenne Muscular Dystrophy and Guillain-Barre Syndrome.^{11 12 13} Unlike specialized neurophysiologic testing, MRC grading can be easily performed at the bedside by trained clinicians. Muscle strength testing based on MRC score has been significantly correlated with hospital length of stay, mortality, and ventilator days in critically ill adults and is widely recommended as a screening method for ICU-AW.^{1 14 15} While MRC grading has also been used in certain pediatric populations, its utility as a reliable screening tool for PICU-AW has not been established.^{16 17} The primary objective of this study was to determine the feasibility of using the MRC score for muscle strength testing in critically ill children. Feasibility was defined as the ability to conduct MRC exams on PICU patients. The secondary objective was to explore the interobserver reliability of using the MRC score for muscle strength testing in critically ill children.

Materials and Methods

Study Design

This prospective observational substudy of the Functional Recovery in Critically ill Children (“Wee-Cover”) pilot study (ClinicalTrials.gov Identifier: NCT01724593) was approved by the Institutional Research Ethics Board and conducted at the McMaster Children's Hospital PICU in Hamilton, Ontario, Canada.¹⁰

Participants

Eligibility criteria were reflective of children who may be at risk of PICU-AW. Children older than 12 months to 17 years old with at least one organ dysfunction as measured by the Pediatric Logistic Organ Dysfunction score, with a greater than 48-hour PICU stay and whose activity was limited to bed rest during that time, were eligible for inclusion. Patients with a known underlying neuromuscular disorder, who were not expected to survive their PICU stay, who were already mobilizing well, or who were at their baseline functional status at the time of screening were excluded. Written, informed consent was obtained for all participants.

Study Procedures

The MRC score is a bedside examination consisting of a series of voluntary muscle contractions evaluated by physicians or physical therapists to assess muscle strength. Given that muscle strength is determined by movement against gravity and resistance, the patient's alertness and compliance is necessary to determine the full extent of one's power. The MRC was scored on a 5-point scale, where 0 represents complete paralysis and 5 represents full strength (Table 1). We included the optional subcategories put forth by the MRC of 4+, 4, and 4⁻ to represent resistance against slight, some, and moderate resistance. One-point equivalents were assigned to each of these subcategories, allowing for a maximum score of 7 points. Individual MRC scores from predefined muscle groups can be combined in a “sum score,” which yields a global estimate of motor function. In our study, the scores from four large muscle groups (elbow flexion, wrist flexion, knee extension, and ankle plantar flexion) were totaled to provide a maximum MRC sum score of 28, denoting normal muscle strength in all four muscle groups. PICU-AW was defined by a total MRC sum score of less than 24, while proximal or distal weakness was defined as a sum score totaling less than 10 in two proximal or two distal muscle groups, respectively. These cutoffs were selected to represent an average score of <7 for each muscle group, denoting “weakness” in each limb.¹⁸ The muscle groups included in our score were selected to include both proximal and distal muscles that have been shown to have good reproducibility and inter-rater reliability in ambulatory pediatric settings.¹³

Table 1. Medical Research Council sum score for muscle strength^a .

Score	Description
0	No contraction
1	Flicker or trace of contraction
2	Active movement with gravity eliminated
3	Active movement against gravity
4-	Active movement against slight resistance
4	Active movement against some resistance
4+	Active movement against moderate resistance
5	Normal power

^aA sum score is calculated using four muscle groups (elbow flexion, wrist flexion, knee extension, and ankle plantar flexion).

Two clinically trained raters independently assessed participants' muscle strength using the MRC scoring at enrollment, and thereafter weekly until hospital discharge. Before the commencement of this study, all three raters were trained on the MRC score, and established the standardized technique for assessment and scoring. There were three possible raters: two physicians and one physiotherapist. Raters were blinded to the results of the other rater's score. To minimize examination burden on the patient and to minimize variability due to time- and patient-related factors, both raters visited the patient together, but conducted the MRC examinations independently. Patients were assessed in a standardized manner midline in bed with the head of bed elevated at 30 degrees. For those patients uncomfortable or uncooperative in this position, a modified sitting position on the parent's lap was used. For consistency, the patient's right side was tested except in the case of invasive lines, orthopedic injuries, burns, or hemiparesis, where the patient's unaffected side was tested. Limitations, interruptions to or inability to conduct the MRC examinations, were recorded.

Feasibility Outcomes and Criteria for Success

The primary outcome of interest was feasibility, defined as the ability to conduct full MRC exams on critically ill children, while in the PICU, and the time taken to successfully complete the first MRC. Secondary outcomes were to describe the barriers to conducting these exams, and to assess the interobserver reliability of MRC scores in this population.

Sample Size

The sample size was determined based on the primary study endpoints.¹⁰ We aimed to recruit 33 patients for the study over 8 months.

Statistical Analysis

Demographic data and feasibility outcomes were presented using descriptive statistics reported as mean (standard deviation [SD]) or median (minimum [min], maximum [max]) for continuous variables and count (percent) for categorical variables. Inter-rater reliability was assessed using interclass correlation (ICC) for continuous variables, and Cohen kappa coefficient for binary variables, together with their associated 95% confidence intervals (CIs). We used the following scale for kappa values described by Landis and Koch to present the strength of the agreement: poor (<0.0), slight (0.00–0.20), fair (0.21–0.4), moderate (0.41–0.60), excellent (0.61–0.80), and almost perfect (0.81–1).¹⁹ We also calculated the percentage agreement between the two raters for individual muscle groups, selected muscle groupings, summed MRC scores, and the diagnosed PICU-AW (sum score <24). We considered the raters' MRC scores to agree that there was a less than 10% difference between sum scores.^{13 18} Data were analyzed using SAS 9.2 (SAS Institute Inc., Cary, North Carolina, United States).

Results

The study was conducted from October 2012 to April 2013. Baseline characteristics of the 33 participants are presented in Table 2. The median (min, max) age was 6.5 (1.1, 16.1) years and 16 (49%) were males. The most common diagnosis requiring PICU admission was respiratory failure (27%), followed by septic shock (15%), trauma (13%), and for post-operative surgical care (13%). Sixteen participants (49%) had a preexisting comorbid health condition.

Table 2. Patient demographics.

Demographic variable	Number of patients (n = 33)
Age, y; median (min, max)	6.5 (1.1, 16.1)
Gender, male; n (%)	16 (48.48)
Primary reason for admission: n (%)
Respiratory failure (including respiratory tract infections)	8 (26.67)
Septic shock	5 (15.15)
Trauma	4 (13.33)
Surgery	4 (13.3)
Neurologic	3 (10.00)
Hypovolemic shock	1 (3.03)
Other	5 (16.67)
Preexisting comorbidity/chronic condition: n (%)	16 (53.33)
Mean intensive care unit stay, days (min, max)	9 (3, 127)

The participant flowchart is presented in Fig. 1. A total of 255 patients were screened for eligibility, 39 of whom met eligibility criteria and were approached for consent. Thirty-three participants (85%) consented to the study. MRC exams were attempted on all 33 participants, but could be completed in only 21 (64%), 9 of who (43%) received at least one exam while in the PICU, and in the remaining 12 (57%), MRC exams could only be conducted after PICU discharge. The median (min, max) time to the first successful MRC exam was 7 days (0, 51) from enrollment.

Fig. 1.

Flowchart of participant enrollment.

There were a total of 95 attempted MRC exams on the 33 participants. MRC exams could not be conducted or completed on 55 (57%) of the 95 attempts. Table 3 outlines the barriers to MRC exams. The most common reasons for failure to conduct or complete an MRC examination were patient sedation, poor cognition or developmental delay, patient unavailability, and unwillingness to cooperate. Of the 45 successfully completed MRC examinations on 21 participants, challenges were reported on 25 (56%) of those exams (Table 3). Most commonly reported challenges included an upset/uncooperative patient (22%), lack of cognitive ability (22%), and baseline neurological deficits (18%). In this study, which enrolled children aged 1 year or older, none of the children younger than 2.6 years were able to complete an MRC. The median age for those with completed MRC exams was 7.6 years (compared with 3.5 years in those in whom an MRC exam could not be completed).

Table 3. Feasibility of MRC scoring in pediatric intensive care unit.

Characteristic	Value
Patients with at least one completed MRC in pediatric intensive care unit, n (%)	9 (27)
Median age in whom MRCs could be conducted, y (min, max)	7.6 (2.6, 16.1)
Median age in whom MRCs could not be conducted, y (min, max)	3.5 (1.1, 16)
Median days to first successful exam (min, max)	7 d (0, 51)
Reasons MRC could not be conducted/completed, n (%); (total number of incomplete attempts = 55, more than one reason possible)
Patient sedated or comatose	27 (49)
Inability to follow commands due to cognitive status (age, developmental delay)	10 (18)
Patient not available (i.e., at test, or receiving procedure)	9 (16)
Patient noncooperation/refused exam	8 (15)
Patient in pain	7 (13)
Hemodynamic instability	5 (10)
Investigator not available	1 (2)
Reported challenges encountered during completed MRC exams, n (%); (total of 45 MRC exams completed on 21 patients, more than one reason possible)
Difficulty following commands due to cognitive status (i.e., young age, developmental delay)	10 (22)
Patient upset during exam	10 (22)
Patient has baseline neurological deficits	8 (18)
Patient reported pain during testing	5 (11)
Patient sedated	2 (4)

Abbreviation: MRC, medical research council.

Tables 4 and 5 present the inter-rater reliability results for MRC sum scores, individual muscle groups, and selected muscle groupings. Inter-rater reliability for the MRC scores was excellent to almost perfect for most of the muscle groups, with the exception of knee extension, on which inter-rater reliability was moderate. When we used MRC scores to determine the clinical diagnosis of weakness, defined as a sum score of less than 24, inter-rater reliability was moderate for generalized weakness (kappa coefficient: 0.48, 95% CI: 0.1, 0.87), and moderate to excellent for the diagnosis of proximal, distal, upper, or lower limb weakness. However, the CIs for each of these kappa calculations were wide (Table 5).

Table 4. Interobserver reliability regarding Medical Research Council score (n = 21 patients).

	Intraclass correlation (95% confidence interval)	Agreement, % (95% confidence interval)a
Sum score	0.87 (0.71, 0.94)	71.4 (52.1, 91.7)
Elbow flexion	0.77 (0.52, 0.90)	52.4 (31.0, 73.7)
Wrist flexion	0.76 (0.51, 0.90)	47.6 (26.3, 69.0)
Knee extension	0.68 (0.36, 0.85)	52.4 (31.0, 73.7)
Ankle plantar flexion	0.91 (0.78, 0.96)	52.4 (31.0, 73.7)
Proximal muscles	0.74 (0.46, 0.88)	61.9 (41.1, 82.7)
Distal muscles	0.88 (0.73, 0.95)	61.9 (41.1, 82.7)
Upper limb	0.78 (0.54, 0.91)	47.6 (26.3, 69.0)
Lower limb	0.90 (0.77, 0.96)	61.9 (41.1, 82.7)

^aPercentage agreement was measured allowing for up to a 10% difference in sum scores to reflect rater agreement.

Table 5. Interobserver agreement on muscle weakness (n = 21 patients).

	Rater 1a count n (%)	Rater 2a count n (%)	Kappa (95% confidence interval)
Generalized weakness (MRC sum score <24)	13 (61.9)	14 (66.7)	0.48 (0.10, 0.87)
Proximal weakness (MRC sum score <10)	11 (52.4)	11 (52.4)	0.62 (0.28, 0.96)
Distal weakness (MRC sum score <10)	11 (52.4)	8 (38.1)	0.52 (0.16, 0.87)
Upper limb weakness (MRC sum score <10)	11 (52.4)	10 (47.6)	0.72 (0.42, 1)
Lower limb weakness (MRC sum score <10)	10 (47.6)	12 (57.1)	0.62 (0.29, 0.95)

Abbreviation: MRC, medical research council.

^aRater 1 and 2 counts reflect the number of identified cases of weakness identified by each rater.

Discussion

ICU-AW is a common and important sequelae of critical illness in adults. As immobility is one of the key risk factors for ICU-AW, early mobilization interventions have been shown to prevent ICU-AW, and are now recommended as a standard of care in adults.²⁰ It is therefore important to identify a reliable and feasible method of screening for muscle weakness in children, to determine its clinical significance and impact on short- and long-term outcomes in this population. Although MRC scoring has been widely used as a screening tool for the detection of ICU-AW in adults, its use in the pediatric ICU population has been limited.¹ In contrast to adults, pediatric-specific factors such as developmental and physical age, temperament, cognitive ability, and parental stress add to the challenge of assessing muscle strength in this population. In the only prospective study to date, Banwell et al⁹ used MRC testing as a screening tool to detect muscle weakness in critically ill children. They reported a very low (1.7%) incidence of PICU-AW. However, this may be attributed to the limitations in the ascertainment and determination of PICU-AW. Subsequently, the true incidence of PICU-AW remains unclear.

Overall, we found that MRC exams were not feasible in the majority of critically ill children in this study. This is the major limitation in the use of the MRC as an early bedside screening tool for those with or at risk of PICU-AW, during the acute phase of illness, when preventative measures may be implemented. MRC exams could only be conducted in 27% of our patient cohort while in the PICU; MRC exams could not be executed, completed, or were challenging during the majority of attempts, most commonly because of depressed level of consciousness, limited cognition, patient refusal, or anxiety. MRC exams were therefore possible only in a specific subset of awake, cooperative, cognitively able, and older children. MRC grading of muscle strength requires a patient to be willing and able to perform muscle contractions on command. This is often difficult for young or developmentally delayed children who lack the cognitive ability to comply. This explains why we found that the median age for those in whom MRC exams were feasible was significantly older (7.6 years), compared with those in whom they were not (3.5 years). Young age is therefore a major limitation to the use of the MRC scoring in PICU, where over half of the population may be younger than 2 years.²¹ The median time to successfully executing the first MRC was 1 week after PICU admission, and the majority of MRCs could only be conducted after PICU discharge.

Although the MRC score has been widely used in critically ill adults, the feasibility of this test has only recently been examined.¹² Hough et al¹⁸ found that MRC testing was not possible for the majority of their 135 ICU patients because of coma, delirium, and/or injury. Hermans et al²² used an attention screen to select patients for MRC testing, resulting in a high exclusion of patients who were unconscious or cognitively impaired. As with our study, patient sedation was noted to be a significant barrier to performing MRC exams in both of these studies. This is concerning as the adult literature suggests that sedated and mechanically ventilated patients have the highest risk of developing ICU-AW.¹

Although we encountered many challenges, our results suggest that the MRC is reliable in a select group of awake, cooperative, and cognitively able critically ill children. In this subset of children in whom we were able to perform MRC examinations, the inter-rater reliability of the MRC scores and sum scores was excellent. This is consistent with findings observed in adult studies.^{3 4 18 22} However, when using a cutoff MRC score to clinically define, if a patient has PICU-AW, the inter-rater reliability was moderate. This discrepancy in inter-rater reliability between the absolute score and diagnosis of weakness may be explained by the use of a strict cutoff of MRC score to denote weakness in a particular muscle. Hence, any marginal decrease in muscle power less than normal qualifies as “weakness.” Similar methods for selecting a cutoff have been used in adult studies; however, because the score is simply a summation of individual muscle scores, it is possible that focal weakness could account for some cases of meeting the threshold.¹⁸ Furthermore, this cutoff has not been correlated with any clinically relevant or functional outcomes in the critical care setting. We interpret our results with caution, given the small sample size and wide confidence intervals for ICC and kappa coefficients.

While this is the first prospective study to evaluate the use of MRC scoring in critically ill children, we acknowledge the following limitations. Our cohort is derived from a single tertiary care center and we therefore cannot comment on the generalizability across other PICU populations. While we standardized our approach between three clinically trained raters, MRC exams were not conducted by the same two raters on every occasion. We did not use a control group of standardized patients, and therefore could only determine patient-related factors and not rater-dependent factors in our inter-rater reliability evaluation. Although MRC exams were repeated at weekly intervals, we were not able to assess inter-rater reliability over time, as very few patients had more than one successful exam. While reporting percentage as an index of rater agreement is widely used, this method may be misleading as it does not take into consideration agreements that would be expected by change, and may overestimate agreement. Our use of ICC and kappa coefficient in the analysis was therefore more appropriate.

Screening for PICU-AW remains a significant challenge. Most measures of muscle strength require volitional muscle contraction, and yet those at highest risk of PICU-AW are likely unable to comply with these exams. Hand dynamometry measurement of grip strength, though simple and easy to use, is totally effort dependent, and less suited for assessing very weak muscles.^{14 23} Furthermore, handgrip strength is suggested to be less predictive of mortality and length of stay than measures of global muscle strength such as the MRC score.²⁴ Electrophysiological studies, such as electromyogram and nerve conduction studies, along with muscle biopsies have also been used to assess neuromuscular dysfunction in the ICU.^{9 25 26 27} However, a high frequency of nonspecific electrophysiologic and histologic abnormalities has been reported in a high proportion of all adult ICU patients, making it difficult to screen for clinically relevant outcomes.²⁸

Multiple barriers to success, including young age, low cognitive status, deep sedation, and low cooperation make MRC testing difficult in the PICU. Although this study is small, we believe that our findings have broader implications for the use of MRC testing in any critically ill child, not just those at risk of PICU-AW. It is clear from this study that MRC is not a feasible method for the early screening of patients at risk of PICU-AW. Furthermore, in contrast to the adult ICU-AW literature, it remains unclear whether PICU-AW in itself predicts poor functional recovery beyond the PICU.^{6 21} Future research efforts should therefore focus on evaluating patient important outcomes such as function and recovery following a critical illness, and the predictors of such outcomes. Only then can we evaluate the efficacy of acute mobilization and rehabilitation interventions in these children.

In conclusion, although inter-rater reliability of MRC grading is excellent, this test of muscle strength is not feasible in the majority of critically ill patients as an early screening tool for PICU-AW in children. These findings underscore the challenges in evaluating the true incidence of PICU-AW. Given the paucity of data linking muscle strength to objective measurement of physical function in the critically ill, future research evaluating the efficacy of exercise-based rehabilitation should focus on more meaningful patient endpoints such as functional outcomes and recovery.