Abstract
Background
Cerebral palsy (CP) is a complex neuromuscular condition that may negatively influence gross motor function. Children diagnosed with CP often exhibit spasticity, weakness, reduced motor control, contracture, and bony malalignment. Despite many previous association studies, the causal impact of these impairments on motor function is unknown.
Aim
In this study, we proposed a causal model which estimated the effects of common impairments on motor function in children with spastic CP as measured by the 66-item Gross Motor Function Measure (GMFM-66). We estimated both direct and total effect sizes of all included variables using linear regression based on covariate adjustment sets implied by the minimally sufficient adjustment sets. In addition, we estimated bivariate effect sizes of all measures for comparison.
Method
We retrospectively evaluated 300 consecutive subjects with spastic cerebral palsy who underwent routine clinical gait analysis. Model data included standard information collected during this analysis.
Results
The largest causal effect sizes, as measured by standardized regression coefficients, were found for selective voluntary motor control and dynamic motor control, followed by strength, then gait deviations. In contrast, common treatment targets, such as spasticity and orthopedic deformity, had relatively small effects. Effect sizes estimated from bivariate models, which cannot appropriately adjust for other causal factors, substantially overestimated the total effect of spasticity, strength, and orthopedic deformity.
Interpretation
Understanding the effects of impairments on gross motor function will allow clinicians to direct treatments at those impairments with the greatest potential to influence gross motor function and provide realistic expectations of the anticipated changes.
Introduction
Gross motor function in cerebral palsy
Gross motor function in children directly influences quality of life indicators such as activity and participation [1–3]. Children diagnosed with cerebral palsy (CP) often have limitations in their gross motor abilities [4]. An important goal of treatment for these children is to optimize functional abilities. However, knowing what to treat is not obvious, since the causal link from neurological and musculoskeletal factors to gross motor skills is poorly understood.
Cerebral palsy is a complex neurological condition and the most common childhood motor disability with incidence ranging from 2–3.5 per 1000 children [5]. CP describes a group of permanent movement and posture disorders in which damage to the developing brain impairs motor control and may induce spasticity, leading to reduced strength and abnormal musculoskeletal loading, in turn leading to joint contractures and bony deformities. Although the injury is non-progressive, secondary effects can continue to worsen with maturation. Function in this population is commonly measured using the Gross Motor Function Measure (GMFM), a task-based standardized test consisting of five domains that rate a patient’s level of function (A. Lying and Rolling, B. Sitting, C. Crawling and Kneeling, D. Standing, and E. Walking, Running and Jumping) [6]. The GMFM may be reported using the full 88-item score (GMFM-88), a 66-item score (GMFM-66) [7], or individual domain scores. The GMFM has been shown to be related to measures of mobility, self-care, and social function [8, 9]. For ambulatory children, the standing (GMFM-D) and walking, running, jumping (GMFM-E) domain scores are highly relevant to mobility.
Treatment options for children with CP span many disciplines including physical therapy, orthopaedic surgery, neurology, and physiatry. Physical therapy treatment focuses on goal oriented task to improve function and participation with interventions typically focused on strengthening, goal directed functional training, treadmill training, casting and bracing. Surgical treatment to address structural changes secondary to motor disorders commonly include lengthening of musculotendon units to improve range of motion, and osteotomy or guided growth to improve alignments. Multiple procedures are commonly performed at one surgical event, deemed single event multilevel surgery or SEMLS. Spasticity management may be attempted with either focal (chemodenervation) or global (baclofen, selective dorsal rhizotomy) approaches [10].
There is little evidence that any of these common treatments for the CP child improve function. Significant improvements in gross motor function have not been found with SEMLS [11–13]. Selective dorsal rhizotomy may result in small functional increases at short- and mid-term, but these are not maintained at long term follow-up [14–19]. Likewise, evidence of functional gains from physical therapy treatments are limited to short-term changes [20–22]. Knowing the relative influences of impairments on gross motor function can allow clinicians to target treatments at those impairments with the greatest impact on this important outcome domain.
Association studies of function
Much of the literature investigating the relationships between impairments and function consists of studies that consider the associations of a few chosen variables (e.g., strength of a single joint). Several such studies investigated relationships between impairments and gross motor function using bivariate correlation or regression; analysis of one independent variable of cause (e.g., spasticity, strength) to one dependent variable of effect (e.g., gait, GMFM) [23–26]. This approach is insufficient when complex relationships between variables exist, since it ignores the dependent effects of other influential variables. These studies focus on associations, with little or no formal discussion of causal mechanisms. For example, Shin et al. evaluated bivariate correlations between isometric hip and knee strength in 24 pediatric subjects with CP and GMFM-D, and GMFM-E. Correlations with GMFM domain scores ranged from r = 0.06–0.30, and none were significant. Other primary (e.g., motor control) and secondary (e.g., contracture) impairments were not considered, and each strength measure was considered in isolation [25].
When multivariate relationships have been explored, it has been done with limited variable sets, small numbers of participants, and without explicitly proposing and testing causal relationships. As an example, Eek and Beckung examined the relationships between hip, knee, and ankle strength and function in 55 children with CP. No other variables were considered. In contrast to Shin, significant bivariate correlations were found ranging from r = 0.59–0.80 between strength and GMFM-66. The multivariate combination most strongly correlated to GMFM-66 was ankle plantarflexor and hip flexor strength, yielding an r2 = 0.73 [27].
While previous studies identified many factors associated with gross motor function, the utility of these models for guiding intervention is limited since pernicious effects like confounding, mediation, and opening of non-causal pathways is possible [28]. By analyzing strength in isolation, without consideration for the causal influences of other variables, Eek and Beckung’s results suggest that just two muscle groups account for 73% of the variance in GMFM-66. On the surface, these results imply strengthening these two muscles would be a highly effective therapy. However, neurological impairments do not occur in isolation. If weakness is influenced by poor motor control, age, or overall severity of injury, the effects of strengthening might be smaller than expected. In what follows we show that proper accounting for covariates reduces the apparent importance implied by non-causal analysis of many individual clinical factors.
Explicit causal approach
Behind most, if not all, association studies lurks an implicit causal hypothesis. It is unlikely that Eek and Beckung were simply trying to demonstrate an association between strength and GMFM-66. Rather, their study almost certainly reflects an assumption that strength is an important cause of function. We demonstrate below that strength is an important causal factor, though with a substantially smaller impact than bivariate estimates would suggest. However, if causal claims are going to be made, they should be explicit and they should be tested.
An explicit causal model provides a set of conditions that assess the plausibility of the model. Once plausibility is established, a casual model can be used to derive covariate adjustments necessary for statistical modeling. Both the direct effect sizes (dependent variable changes occurring from isolated independent variable changes) and total effect sizes (dependent variable changes occurring when the independent variable is transmitted through mediator variables) can be evaluated to assess the relative influences of model variables [29]. There has been limited use of causal modeling in the study of CP. Kim and Park proposed a simple causal model to examine the effects of spasticity and strength on GMFM-88 and functional outcomes as measured by the Pediatric Evaluation of Disability Inventory (PEDI) [30]. Only GMFM-88 had significant direct effects on PEDI, whereas spasticity and strength had indirect effects, mediated by GMFM-88. The authors noted that the model lacked other potentially important variables such as orthopedic deformities.
Instrumented three-dimensional gait analysis is a common diagnostic and treatment-planning tool used in the assessment of children with CP. Standard gait analysis protocols include direct and indirect measures of neurological and orthopedic impairments that may influence function. The aim of this study is to hypothesize and test a causal model for GMFM-66 that examines the influence of key neurological and orthopedic impairments commonly measured during a gait analysis visit, which includes a comprehensive physical exam, standard functional measures, and 3D kinematics and electromyography collected during walking. Compared to Kim and Park’s study, our proposed model considers measures encompassing all International Classification of Functioning, Disability and Health (ICF, www.asha.org/slp/icf) domains, including measures of orthopedic deformity, spasticity, gait quality, and both dynamic and selective motor control. Understanding the causal factors influencing function can help to better educate patients, families, and providers about realistic goals for treatments.
Methods
Approval with a waiver of informed consent was obtained for this retrospective analysis through the Western IRB (study number 1249365).
Causal model
A graphical causal model [31, 32] is proposed to explain how a set of commonly measured and potentially treatable neurological and orthopedic impairments affect function as measured by GMFM-66 (Fig 1 and Table 1). Model inputs are limited to those variables routinely collected during a motion analysis study for children with spastic CP at one center. The hypothesized relationships in the model, along with linearity assumptions, lead to conditional independence tests that assess the plausibility of the model and covariate adjustment sets for statistical models used to estimate the relative causal contributions of various factors [28].
Table 1. Causal model.
Clinical Feature | Measure | Causes |
---|---|---|
Injury | ||
Age | Age | |
Motor Control | MC, Latent | Injury |
Selective Motor Control | SCALE | Motor Control, Age |
Dynamic Motor Control | DMC | Motor Control, Age |
Strengtha | Strength | Injury, Motor Control, Age |
Spasticity | Spasticity | Injury, Age |
Contracture | Latent | |
Ankle Dorsiflexion Contractureb | AnkleDF | Strength, Spasticity, Dynamic & Selective voluntary motor control, Age |
Knee Extension Contractureb | KneeExt | Strength, Spasticity, Dynamic & Selective voluntary motor control, Age |
Hamstrings Contractureb | Popliteal | Strength, Spasticity, Dynamic & Selective voluntary motor control, Age |
Hip Extension Contractureb | HipExt | Strength, Spasticity, Dynamic & Selective voluntary motor control, Age |
Torsion | Latent | |
Tibial Torsionc | TibRot | Strength, Spasticity, Dynamic & Selective voluntary motor control, Age |
Femoral Anteversionc | FemRot | Strength, Spasticity, Dynamic & Selective voluntary motor control, Age |
Gait Pattern | GDI | Strength, Spasticity, Dynamic & Selective voluntary motor control, Age, Ankle ROM, Knee ROM, Popliteal ROM, Hip ROM, Tibial Torsion, Femoral Anteversion |
Gross Motor Function | GMFM-66 | Strength, Spasticity, Dynamic & Selective voluntary motor control, Age, Ankle ROM, Knee ROM, Popliteal ROM, Hip ROM, Gait Pattern |
Features included in the causal model illustrated in Fig 1. Measures here match the Fig 1 variable names. The hypothesized direct causes of each feature are listed.
a. Because we measure strength in an age-adjusted manner, we do not include an explicit causal link in our graphical model used for testing plausibility and determining adjustment sets.
b. Ankle, Knee, Hamstrings, and Hip contractures are influenced by an unmeasured latent variable Contracture so that Contracture → Ankle Dorsiflexion Contracture, Contracture → Knee Extension Contracture, Contracture → Hamstrings Contracture, Contracture → Hip Extension Contracture.
c. Tibial Torsion and Femoral Anteversion are influenced by an unmeasured latent variable Torsion so that T → Tibial Torsion, T → Femoral Anteversion.
The model is developed by considering the included clinically relevant variables and forming a hypothesis of how each variable influences every other variable in a causal relationship. If there is reasonable evidence or if a logical inference of direct causality exists, then graphically, a linkage is added from the causal variable to the effected variable. If a causal variable’s effect is fully mediated by other included variables, then a direct effect is not included. The causal model proposed here starts with the neurologic injury that has led to the diagnosis of CP. The primary impairments resulting from this injury commonly manifest as spasticity and reductions in selective motor control, dynamic motor control, and strength. Each of these neurological factors is hypothesized to have a direct effect on gross motor function. Additionally, these primary impairments cause gait deviations through improper muscle activation patterns, which in turn impairs GMFM-66 by making intended movements more difficult. Selective voluntary motor control deficits also reduce strength by impeding the ability to isolate and volitionally activate muscles, resulting in functional weakness. Strength in turn affects both gait and gross motor function by altering and limiting movement strategies. The effect of age on gross motor function is accounted for both directly, reflecting motor learning associated with practice, and indirectly, reflecting neuromaturation and growth. Orthopedic deformities, in the form of contractures (Ankle, Knee, Hamstrings, and Hip) are caused, in part, by the improper muscle forces brought about by spasticity, poor strength, and poor motor control. These improper forces also influence the remodeling of bony torsions (femoral anteversion and tibial torsion). Contractures and long-bone torsions both affect gait, and the contractures directly impede gross motor function by restricting movement. Note that we assume no direct effect of long bone torsion on gross motor function. There are undoubtedly other impairments associated with the injury that are not represented. For the present model, these other factors are ignored. This meaningful limitation and its impact on our results will be discussed later.
Study data
We evaluated the model using 300 consecutive unique individuals with a diagnosis of CP who were seen at a single center for a quantitative 3D gait assessment including electromyography (EMG) of the lower extremity and had complete datasets. All studies were conducted by an experienced physical therapist. The center collecting the data was fully accredited during the times of data collection (Center for Motion Laboratory Accreditation, cmlainc.org). For all bilateral measures, averages between sides were used, including for unilaterally involved individuals. The following metrics categorized by ICF domains were used for the model.
Body functions and structure
Spasticity was computed on a 0–5 scale derived from multiple measures collected during physical examination [33]. The spasticity scale included modified Ashworth scales [34] recorded for between 3 to 5 muscles, Duncan-Ely tests of rectus femoris spasticity, beats of clonus, and ankle plantarflexor differences between initial catch of quick stretch and maximal values [33].
Strength was measured by both manual muscle testing grade and hand-held dynamometry of lower extremity muscle groups. Dynamometry values were expressed as z-scores relative to typically developing individuals after controlling for gender, age, and height [35, 36]. These were then combined with manual grades to yield a composite 0–5 strength scale.
Selective motor control was measured using the Selective Control Assessment of the Lower Extremity (SCALE). Total limb SCALE values (0–10 scale) were used as an overall representation of selective motor control [37].
Limb rotational deformities were measured with an inclinometer. Femoral anteversion was estimated from the midpoint between maximum inward and outward rotation of the hip with the subject prone and knee flexed to 90° [38]. Tibial torsion was estimated using bimalleolar and epicondyle landmarks with the subject supine and the knee extended [39, 40]. Typically developing values of 26.9° for femoral anteversion and 16.0° for tibial torsion were subtracted from raw data to reflect deviations from typical developing norms, with positive values internally directed [41].
Sagittal plane joint end ranges of motion were measured with handheld goniometry. Maximal hip extension was measured with the opposite hip flexed, knee extension in a supine position, popliteal angle with the opposite leg extended, and ankle dorsiflexion in a prone position with the knee at maximal extension [39, 40]. Joint contractures were expressed as the measured raw end range values offset by reported typically developing values: 0° for hip extension contracture, 4° of hyperextension (recurvatum) for knee extension contracture, 25.6° popliteal angle for hamstrings contracture, and 21.3° for ankle dorsiflexion contracture [41].
Activity and participation
Gait quality was measured by the gait deviation index (GDI), computed using published techniques [42]. The GDI is a validated measure of overall gait deviations, scaled so that 100 (10) is the mean (sd) for typically developing controls.
Dynamic motor control was expressed by the Walking Dynamic Motor Control Index (Walk-DMC), calculated from electromyography collected during walking at self-selected speed. Walk-DMC is a validated measure of motor control that is scaled so that 100 (10) is the mean (standard deviation) of typically developing peers. Walk-DMC values were computed based on the methods described by Steele et al. [43], using an aggregate analysis of 5–7 self-selected speed trials obtained as part of the clinical gait analysis protocol.
Gross motor function classification level (GMFCS) and the 66 item gross motor function measure (GMFM-66) were recorded by the physical therapist conducting the examination.
Personal factors
Age at assessment was considered to reflect maturation of strength and control. Height, weight, and gender were recorded and used to normalize dynamometry strength measures, but otherwise were not hypothesized to be explicit causal factors in this analysis.
Health condition
CP subtype (hemiplegia, diplegia, quadriplegia) was determined from the medical record.
Variables were transformed as follows. First, each variable was standardized using the sample mean and standard deviation. Next, the sign of the variable was adjusted so that larger values indicate less impairment and smaller values indicate more impairment. This was done so that positive effect sizes imply a higher score (better function) on the GMFM-66 scale. For age, larger values retain the conventional meaning of older.
Analysis
Implied conditional independencies were identified to test the plausibility of the proposed model [31]. These conditional independencies are a list of partial correlations that must be zero or insignificant for the observed data to be consistent with the structure of the hypothesized causal model [44]. Meeting these conditions is necessary, but not sufficient, for a causal model to be correct, since there can be multiple models that meet the same implied independencies.
We chose to use linear models to estimate the magnitude and sign of total and direct causal effects for each factor of interest. A linear model was built for each factor whose effect size was being investigated using the adjustment set identified from the causal model. Adjustment sets are lists of covariates that, when included in the linear model, allow the causal effect of a particular factor to be estimated by controlling appropriately for other factors. Only main effects were allowed in the linear models (no interactions). This decision was driven by both causal and practical considerations. From a causal perspective, we did not see clear rationale for interaction effects, and, if they did exist, we assumed they would likely be substantially smaller than the main effects. From a practical perspective, we know that spurious significant interactions are likely, given the number of factors under consideration, and that approximately 16x more samples are needed to detect significant interactions compared to main effects [45]. Thus, in light of our relatively coarse measurements and modest sample size, we settled on a main-effects-only model. In addition to deriving causal effect sizes for each individual factor, we also built a predictive model to examine the total causal effect of the factors taken together, enabling estimates of GMFM-66 as a function of the key clinical measures of neurological and orthopedic impairment.
We also derived bivariate linear models to assess the apparent, non-causal, effect sizes for each of the factors considered in the causal model. For example, we regressed GMFM-66 against Strength as a single predictor, then against Spasticity as a single predictor, and so on. The effect sizes from the bivariate models typify the estimates that result from the non-causal association analyses that pervade the literature. Thus, comparing the causal and bivariate results shows the interpretive errors that can arise from using a non-causal method.
A 10-fold cross-validation was performed to estimate the model’s performance on out-of-sample observations, thereby quantifying the ability of model to predict GMFM-66. All modeling and statistics were performed with R (version 4.02, R Core Team, 2020). Causal model building and testing was done using dagitty [31].
Results
Participants were typical for the population of CP patients referred for clinical gait analysis at our center (Table 2). We screened 314 individuals to reach the final cohort of 300 participants. Three subjects excluded had missing data due to visit time constraints while the other 11 were unable to complete testing due to an inability to follow directions. Of these 11, five were age six years or younger.
Table 2. Participant characteristics.
Variable | Units | TD Value | GMFCS I | GMFCS II | GMFCS III |
---|---|---|---|---|---|
mean (SD) unless noted | |||||
Number of Participants | 115 | 132 | 53 | ||
CP Subtype (%) | |||||
Spastic diparesis | 30 (26) | 95 (72) | 47 (89) | ||
Spastic hemiparesis | 86 (74) | 32 (24) | 0 (0) | ||
Spastic quadriparesis | 0 (0) | 5 (4) | 6 (11) | ||
Gender = Male (%) | 52 (45) | 75 (57) | 34 (64) | ||
Age | Yr. | 10.5 (3.5) | 11.6 (3.5) | 10.9 (3.3) | |
Walk-DMC | None | 100 | 91 (10.0) | 80 (12.2) | 66 (9.8) |
GDI | None | 100 | 82 (10.7) | 71 (9.4) | 63 (8.5) |
SCALE | None | 10 | 6.8 (1.4) | 5.2 (1.4) | 3.0 (1.4) |
Spasticity | None | 1 | 1.6 (0.9) | 2.1 (1.1) | 2.9 (1.1) |
Strength | None | 5 | 4.6 (0.3) | 4.2 (0.5) | 3.5 (0.6) |
GMFM-66 | None | 100 | 87 (6.2) | 73 (7.0) | 57 (4.6) |
Ankle Dorsiflexion Contracture | Deg. | 0 | -17 (6.1) | -20 (8.1) | -20 (8.2) |
Knee Extension Contracture | Deg. | 0 | 1 (4.9) | -2 (7.2) | -7 (8.8) |
Hamstrings Contracture | Deg. | 0 | -13 (17.4) | -24 (18.7) | -31 (18.3) |
Hip Extension Contracture | Deg. | 0 | -1 (2.7) | -4 (5.1) | -8 (7.6 |
Femoral Anteversion Deviation | Deg. | 0 | -1 (24.1) | -4 (25.7) | -11 (24.9) |
Tibial Torsion Deviation | Deg. | 0 | -1 (5.8) | -2 (6.7) | -3 (7.0) |
Note: Raw values reported. Contractures are negative values. For femoral anteversion and tibial torsion, positive values are inwardly directed. For modeling purposes, variables were transformed so that more positive values indicated less impairment and more negative values indicated more impairment.
Standardized coefficients were computed to assess effect sizes for the various neurological and orthopedic factors (Fig 2). Comparing effect sizes between the causal model and the bivariate regression models shows the important impact that causal modeling can have on effect sizes. Bivariate (non-causal) effect sizes overestimated the importance of factors by 0.18–0.48 compared to their causal counterparts, which corresponded to bivariate overestimations of more than double at a minimum to several fold.
All but two of the 11 independencies implied by the structure of our proposed causal model were statistically insignificant (no adjustment for multiple comparisons). There were significant correlations between hip extension contracture and tibial torsion (r = .18) and knee extension contracture and tibial torsion (r = .28), both conditional on age, DMC, SCALE, spasticity, and strength. This result demonstrates that the structure of the proposed causal model is largely consistent with the observed data, suggesting that the proposed model is plausible.
Coefficients for the direct linear model were estimated for all causal factors (Table 3). The cross validation analysis resulted in an out-of-sample r2 = 0.75 and a mean absolute error of 4.9 points (Fig 3).
Table 3. Predictive model coefficients.
Variable | β | SE | p | Units | Range | TD | Description |
---|---|---|---|---|---|---|---|
Intercept | 14.7 | 4.3 | < .01 | S | NA | NA | GMFM-66 linear intercept [7] |
Age | 0.08 | 0.12 | .50 | Yr. | NA | NA | Child’s age in decimal years (e.g. 10.4 yrs) |
SCALE | 2.3 | .27 | < .01 | S | 0–10 | 10 | SCALE total limb score bilateral average (0–10) [37] |
Walk-DMC | 0.24 | 0.04 | < .01 | S | 22–100+* | 100 (10) | Dynamic motor control scaled from normal [43] |
Strength | 4.5 | 0.85 | < .01 | S | 0–5 | 5 | Multivariate strength scale [35] |
Spasticity | -0.57 | 0.38 | .13 | S | 0–5 | 0 | Multivariate spasticity scale [33] |
GDI | 0.17 | 0.04 | < .01 | S | 0–100+* | 100 (10) | Gait deviation index [42] |
Ankle Dors. Contracture | -0.13 | 0.05 | < .01 | Deg. | > = 0 | 0 | Limitation in achieving > = 21.3° dorsiflexion with knee extended [39–41] |
Hip Ext. Contracture | -0.17 | 0.08 | .02 | Deg. | > = 0 | 0 | Limitation in achieving > = neutral hip extension [39, 40] |
Knee Ext. Contracture | -0.11 | 0.07 | .10 | Deg. | > = 0 | 0 | Limitation in achieving > = 4.0° knee extension with hip at neutral [39–41] |
Hamstrings Contracture | -0.01 | 0.02 | .54 | Deg. | > = 0 | 0 | Limitation in achieving > = 25.6° knee extension with hip flexed to 90° (Popliteal Angle) [39–41] |
Femoral Anteversion | -0.01 | 0.02 | .44 | Deg. | > = 0 | 0 | Difference from typical anteversion (26.9°) [38, 41] |
Tibial Torsion | -0.08 | 0.08 | .21 | Deg. | > = 0 | 0 | Non-weightbearing bimalleolar axis difference from typical (16.0°) [39–41] |
S = unitless scale variable defined in description.
NA = not applicable, TD = typically developing value, SE = standard error, p = p-value.
*Both Walk-DMC and GDI are scaled Z-scores that can exceed the mean typical value of 100, additionally Walk-DMC has a theoretical minimum value dependent upon the scaling values.
To obtain subject GMFM66 use equation: GMFM66 = Intercept + ∑iβi ∙ Variablei.
Discussion
A causal model of function in children with CP provided estimates of direct and total effect sizes for key clinical impairments, and explained 75% of the variance in the GMFM-66 scores in an out-of-sample test set (10-fold cross-validation). The model is plausible and can be evaluated using measurements collected during routine gait analysis evaluations. Selective voluntary motor control, dynamic motor control, and strength had the largest total effects on GMFM-66. Gait impairments (GDI) had the next largest impact on GMFM-66. Note that GDI is not directly manipulable by treatment, but rather it changes in response to modification of orthopedic or neurological factors. Ankle and hip contractures had small effects on GMFM-66, while other orthopedic deformities had statistically insignificant effects. The causal effects of most factors were found to be substantially smaller than those obtained from bivariate analysis.
Our findings have important clinical implications. First, the effects of these measures on gross motor function can guide targets for treatment and inform expectations regarding possible improvements. Second, the results indicate that 75% of gross motor function as measured by the GMFM-66 can be accounted for by measures commonly recorded during clinical gait analysis. Finally, the results clearly show the errors that can arise from computing non-causal, bivariate relationships among clinical factors in children diagnosed with CP. The interconnectedness of neurological and musculoskeletal impairments, gait, and function in this population places a premium on employing analyses that respect the complexity of the clinical condition.
Selective voluntary motor control, as measured by the SCALE composite limb score, had the highest effect in the total effects model. The SCALE score has previously been associated with GMFM-66, and our analysis confirms that this association appears to be causal [46]. Dynamic motor control, as measured by Walk-DMC [43], also had a large effect size, even after controlling for selective voluntary motor control. This suggests that the neurological pathways employed during quasi-static tasks and those used during gait are not the same, and both are important for function. In our proposed model, selective and dynamic motor control deficits share a common cause and are hence correlated (Walk-DMC ← Injury → SCALE). Walk-DMC has previously been associated with changes in gait measures after treatment, including overall gait deviation (GDI), consistent with the proposed causal diagram (Fig 1) in this study [47]. The same study demonstrated that Walk-DMC was associated with changes in the sports and physical function sub-score of the pediatric outcomes data collection instrument (PODCI). While the PODCI sports and physical function score is not identical to the GMFM-66, it is reasonable to think that there is a significant overlap in the underlying construct of the two measures. Selective and dynamic motor control and are difficult to alter with treatments or therapies. Nevertheless, our results suggest that we should continue efforts towards finding control-enhancing therapies, as these hold the greatest prospect for impacting overall function.
Spasticity is another common primary neurological impairment observed in children diagnosed with CP. Both the direct and total effects of spasticity were shown to be relatively small. Thus, intrathecal baclofen pump implantation, selective dorsal rhizotomy, or chemodenervation therapies aimed at spasticity reduction should not be expected to appreciably improve gross motor function. This prediction is consistent with what has been observed in outcome analyses of these treatments [16, 48, 49].
Age did not have a significant effect. Thus, independent of neurological and orthopedic impairments, age alone is not expected to alter gross motor function substantially. Contradicting this finding is previous work showing that GMFM-66 improves from an early age and begins to plateau around age 8 [50]. Approximately 85% of the individuals in this study were older than 7.5, and this may explain our finding that age was not an important cause of GMFM-66. In addition, the largest improvements due to ageing are observed in Gross Motor Function Classification (GMFCS) I individuals [51], who make up only 38% of our study population. Furthermore, we made the simplifying assumption of linearity, which cannot accurately model the growth and plateauing nature of previously observed GMFM-66 vs. age response. Lastly, the well-known GMFM-66 vs. age curves are stratified by GMFCS. While GMFCS may constitute a reasonable adjustment set, our causal model would suggest the nature of GMFM-66 changes are more complex. In fact, GMFCS level can be thought of as a causal consequence of many factors in our model. Put another way, children with poorer strength and motor control and worse spasticity are likely to be classified with a GMFCS level indicating greater severity of the disease. Thus, adjusting for GMFCS alone may not fully capture the effects that motor control, strength, orthopedic impairments, and gait impairments have on function. The lack of a direct age effect in our model suggests that changes in other factors with age (e.g. contractures, strength, gait quality) may explain some of the marginal age effect observed in the GMFM-66.
The effect sizes of strength and gait quality (GDI) were both significant, suggesting that they are important causes of function. Both variables can be improved by treatment (strengthening, physical therapy, orthopedic surgery). While our model suggests that improvement in these domains can raise gross motor function, the evidence for this in practice is not clear. Outcomes of these therapies have not consistently demonstrated improved function in patients [52]. One possible explanation could be the magnitude of strength or GDI improvement achieved with treatment. The standardized effect size for strength and GDI are 0.26 and 0.16, respectively. This means that for one standard deviation of strength improvement, we should expect 0.26 standard deviations of GMFM-66 improvement (3.2 points for the data in this study), and for one standard deviation of GDI improvement we should expect 0.16 standard deviations in GMFM-66 improvement (2.0 points). Typical GDI improvements following surgery are on the order of 7 points, which would correspond to ~1.3 point improvement on GMFM-66 scores, well below the reported minimally clinical important difference [53].
Similar to Kim and Park’s less comprehensive causal model of function [30], we found that both spasticity and strength had significant effects on GMFM-66. However, the inclusion of additional causal factors significantly changed the absolute and relative magnitude of the effect sizes. Kim and Park estimated the effect sizes for spasticity and strength to be 0.34 and 0.45, respectively, while we found them to be 0.08 and 0.26. Differences in testing methods for these subjective measures may impact absolute effect sizes, but should have less impact on relative effect sizes, which were also substantially different in our model compared to Kim and Park’s (4.82 and 3.25, respectively).
One limitation to this study was the exclusion of 14 individuals (4%) due to missing data. In some cases, data was not collected due to the patient’s inability to follow instructions. These missing data have potential to bias the results since some of the lowest functioning individuals may have been excluded. However, only six subjects (2%) were excluded for this reason. In addition, both sides of bilateral measures for unilateral individuals were included and averaged. While this may diminish the influence of the condition on the hemiparetic side, GMFM-66 is a score for an individual, allowing compensatory contributions of the non-hemiparetic side, and thus we reasoned that averaging sides was an appropriate approach. Finally, this is a retrospective analysis of cross sectional data and all causal relationships are hypothesized based on past observations or logistically based assumptions.
Summary
A causal diagram is a hypothesis. Since counterfactual observations (e.g., a clone of a specific child with one clinical parameter altered) do not exist, causal diagrams can never be fully proven or falsified. They can, however, be tested for plausibility from both a domain knowledge and statistical perspective. We showed that the proposed diagram is plausible and explained 75% of the variance in GMFM-66. Nevertheless, unobserved and unmeasured variables may play a significant role. As data were limited to variables retrospectively collected from standard gait analysis studies at a single center, some potentially important factors were omitted. Cognition may be one such omitted but important variable [54]. It is likely that cognition plays a meaningful role in function and including this variable could change the effect sizes of the factors we have examined. Socio-economic factors, access to resources, participation in the community, availability of recreational sports opportunities, emotional health, and myriad other complex factors could also be considered in future extensions of this model.
This study should be viewed as a starting point, and not as a definitive model of causality and function in children with CP. We proposed a model limited to data available from existing motion analysis studies, demonstrated its plausibility, and estimated the causal effect sizes of key clinical factors. We also showed that important differences in interpretation arise when causality, rather than association, is considered. The strength of our approach is the explicit proposal of a causal diagram, leading to a priori statistical models. This contrasts with much of the previous work in this area, which commonly examines ad hoc bivariate relationships in an unstructured methodology. In many cases, the bivariate approach leads to significant and strong correlations, but these findings may be misleading since they lack the proper causal context.
As noted earlier, lurking behind virtually every study of association is a hidden belief in a causal relationship. Except for purely predictive models, we suspect no researchers examine associations without some underlying hypothesis about causality. While our proposed model is imperfect, it is transparent and explicit. Additional work is needed to discover more complete causal models and to devise experiments that confirm or refute the results of these models. The complexity of CP and the relationships among commonly measured variables demand that causality be considered when interpreting data.
Conclusions
Our model results highlight the challenges faced for improving function in children with CP. The largest effect sizes are found in the selective and dynamic motor control measures; impairments for which there are currently no highly effective treatments. In contrast, spasticity, which we can treat via rhizotomy or baclofen pump, has a very small impact on gross motor function. Strength and gait quality both have significant, but modest, effect sizes. However, improvements in these domains following treatment tend to be small, leading to minimal functional gains. If we hope to improve function through these domains, we need more effective treatments for strength and gait.
Supporting information
Data Availability
All relevant data are within the manuscript and its Supporting Information files.
Funding Statement
The author(s) received no specific funding for this work.
References
- 1.Kerr C, McDowell B, McDonough S. The relationship between gross motor function and participation restriction in children with cerebral palsy: an exploratory analysis. Child: Care, Health and Development. 2007;33: 22–27. doi: 10.1111/j.1365-2214.2006.00634.x [DOI] [PubMed] [Google Scholar]
- 2.Lee B-H. Relationship between gross motor function and the function, activity and participation components of the International Classification of Functioning in children with spastic cerebral palsy. Journal of Physical Therapy Science. 2017;29: 1732–1736. doi: 10.1589/jpts.29.1732 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Palisano RJ, Kang L-J, Chiarello LA, Orlin M, Oeffinger D, Maggs J. Social and community participation of children and youth with cerebral palsy is associated with age and gross motor function classification. Physical Therapy. 2009;89: 1304–1314. doi: 10.2522/ptj.20090162 [DOI] [PubMed] [Google Scholar]
- 4.Rosenbaum PL, Walter SD, Hanna SE, Palisano RJ, Russell DJ, Raina P, et al. Prognosis for gross motor function in cerebral palsy: creation of motor development curves. JAMA. 2002;288: 1357–1363. doi: 10.1001/jama.288.11.1357 [DOI] [PubMed] [Google Scholar]
- 5.Colver A, Fairhurst C, Pharoah POD. Cerebral palsy. Lancet. 2013. doi: 10.1016/S0140-6736(13)61835-8 [DOI] [PubMed] [Google Scholar]
- 6.Russell DJ, Rosenbaum PL, Cadman DT, Gowland C, Hardy S, Jarvis S. The gross motor function measure: a means to evaluate the effects of physical therapy. Developmental Medicine and Child Neurology. 1989;31: 341–352. doi: 10.1111/j.1469-8749.1989.tb04003.x [DOI] [PubMed] [Google Scholar]
- 7.Russell DJ, Avery LM, Rosenbaum PL, Raina PS, Walter SD, Palisano RJ. Improved scaling of the gross motor function measure for children with cerebral palsy: evidence of reliability and validity. Phys Ther. 2000;80: 873–885. [PubMed] [Google Scholar]
- 8.Ostensjø S, Carlberg EB, Vøllestad NK. Motor impairments in young children with cerebral palsy: relationship to gross motor function and everyday activities. Dev Med Child Neurol. 2004;46: 580–589. doi: 10.1017/s0012162204000994 [DOI] [PubMed] [Google Scholar]
- 9.Smits D-W, Gorter JW, Ketelaar M, Van Schie PE, Dallmeijer AJ, Lindeman E, et al. Relationship between gross motor capacity and daily-life mobility in children with cerebral palsy. Dev Med Child Neurol. 2010;52: e60–66. doi: 10.1111/j.1469-8749.2009.03525.x [DOI] [PubMed] [Google Scholar]
- 10.Novak I, McIntyre S, Morgan C, Campbell L, Dark L, Morton N, et al. A systematic review of interventions for children with cerebral palsy: state of the evidence. Dev Med Child Neurol. 2013;55: 885–910. doi: 10.1111/dmcn.12246 [DOI] [PubMed] [Google Scholar]
- 11.Thomason P, Baker R, Dodd K, Taylor N, Selber P, Wolfe R, et al. Single-event multilevel surgery in children with spastic diplegia: a pilot randomized controlled trial. The Journal of Bone and Joint Surgery American Volume. 2011;93: 451–460. doi: 10.2106/JBJS.J.00410 [DOI] [PubMed] [Google Scholar]
- 12.McGinley JL, Dobson F, Ganeshalingam R, Shore BJ, Rutz E, Graham HK. Single-event multilevel surgery for children with cerebral palsy: a systematic review. Developmental Medicine and Child Neurology. 2012;54: 117–128. doi: 10.1111/j.1469-8749.2011.04143.x [DOI] [PubMed] [Google Scholar]
- 13.Chang C-H, Chen Y-Y, Yeh K-K, Chen C-L. Gross motor function change after multilevel soft tissue release in children with cerebral palsy. Biomedical Journal. 2017;40: 163–168. doi: 10.1016/j.bj.2016.12.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Bolster EAM, van Schie PEM, Becher JG, van Ouwerkerk WJR, Strijers RLM, Vermeulen RJ. Long-term effect of selective dorsal rhizotomy on gross motor function in ambulant children with spastic bilateral cerebral palsy, compared with reference centiles. Developmental Medicine and Child Neurology. 2013;55: 610–616. doi: 10.1111/dmcn.12148 [DOI] [PubMed] [Google Scholar]
- 15.van Schie PEM, Schothorst M, Dallmeijer AJ, Vermeulen RJ, van Ouwerkerk WJR, Strijers RLM, et al. Short- and long-term effects of selective dorsal rhizotomy on gross motor function in ambulatory children with spastic diplegia. Journal of Neurosurgery Pediatrics. 2011;7: 557–562. doi: 10.3171/2011.2.PEDS10452 [DOI] [PubMed] [Google Scholar]
- 16.Tedroff K, Hägglund G, Miller F. Long-term effects of selective dorsal rhizotomy in children with cerebral palsy: a systematic review. Developmental Medicine and Child Neurology. 2020;62: 554–562. doi: 10.1111/dmcn.14320 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Tedroff K, Löwing K, Åström E. A prospective cohort study investigating gross motor function, pain, and health-related quality of life 17 years after selective dorsal rhizotomy in cerebral palsy. Developmental Medicine and Child Neurology. 2015;57: 484–490. doi: 10.1111/dmcn.12665 [DOI] [PubMed] [Google Scholar]
- 18.McLaughlin J, Bjornson K, Temkin N, Steinbok P, Wright V, Reiner A, et al. Selective dorsal rhizotomy: meta-analysis of three randomized controlled trials. Developmental Medicine and Child Neurology. 2002;44: 17–25. doi: 10.1017/s0012162201001608 [DOI] [PubMed] [Google Scholar]
- 19.MacWilliams BA, McMulkin ML, Duffy EA, Munger ME, Chen BP-J, Novacheck TF, et al. Long-term effects of spasticity treatment, including selective dorsal rhizotomy, for individuals with cerebral palsy. Developmental Medicine & Child Neurology. 2021; n/a. doi: 10.1111/dmcn.15075 [DOI] [PubMed] [Google Scholar]
- 20.Tsorlakis N, Evaggelinou C, Grouios G, Tsorbatzoudis C. Effect of intensive neurodevelopmental treatment in gross motor function of children with cerebral palsy. Developmental Medicine and Child Neurology. 2004;46: 740–745. doi: 10.1017/s0012162204001276 [DOI] [PubMed] [Google Scholar]
- 21.İçağasıoğlu A, Mesci E, Yumusakhuylu Y, Turgut ST, Murat S. Rehabilitation outcomes in children with cerebral palsy during a 2 year period. Journal of Physical Therapy Science. 2015;27: 3211–3214. doi: 10.1589/jpts.27.3211 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Park E-Y. Effect of physical therapy frequency on gross motor function in children with cerebral palsy. Journal of Physical Therapy Science. 2016;28: 1888–1891. doi: 10.1589/jpts.28.1888 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Kramer JF, MacPhail AH. Relationships Among Measures of Walking Efficiency, Gross Motor Ability, and Isokinetic Strength In Adolescents With Cerebral Palsy. Pediatric Physical Therapy. 1994;6: 3–9. [Google Scholar]
- 24.Damiano DL, Martellotta TL, Sullivan DJ, Granata KP, Abel MF. Muscle force production and functional performance in spastic cerebral palsy: relationship of cocontraction. Arch Phys Med Rehabil. 2000;81: 895–900. doi: 10.1053/apmr.2000.5579 [DOI] [PubMed] [Google Scholar]
- 25.Shin HI, Sung KH, Chung CY, Lee KM, Lee SY, Lee IH, et al. Relationships between Isometric Muscle Strength, Gait Parameters, and Gross Motor Function Measure in Patients with Cerebral Palsy. Yonsei Med J. 2016;57: 217–224. doi: 10.3349/ymj.2016.57.1.217 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Ross SA, Engsberg JR. Relationships Between Spasticity, Strength, Gait, and the GMFM-66 in Persons With Spastic Diplegia Cerebral Palsy. Archives of Physical Medicine and Rehabilitation. 2007;88: 1114–1120. doi: 10.1016/j.apmr.2007.06.011 [DOI] [PubMed] [Google Scholar]
- 27.Eek MN, Beckung E. Walking ability is related to muscle strength in children with cerebral palsy. Gait Posture. 2008;28: 366–371. doi: 10.1016/j.gaitpost.2008.05.004 [DOI] [PubMed] [Google Scholar]
- 28.Pearl J. Causal diagrams for empirical research. Biometrika. 1995;82: 669–688. doi: 10.1093/biomet/82.4.669 [DOI] [Google Scholar]
- 29.Fairchild AJ, MacKinnon DP, Taborga MP, Taylor AB. R2 effect-size measures for mediation analysis. Behav Res Methods. 2009;41: 486–498. doi: 10.3758/BRM.41.2.486 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Kim WH, Park EY. Causal relation between spasticity, strength, gross motor function, and functional outcome in children with cerebral palsy: a path analysis. Developmental Medicine and Child Neurology. 2011;53: 68–73. doi: 10.1111/j.1469-8749.2010.03777.x [DOI] [PubMed] [Google Scholar]
- 31.Textor J, van der Zander B, Gilthorpe MS, Liśkiewicz M, Ellison GT. Robust causal inference using directed acyclic graphs: the R package ‘dagitty.’ Int J Epidemiol. 2016;45: 1887–1894. doi: 10.1093/ije/dyw341 [DOI] [PubMed] [Google Scholar]
- 32.Pearl J. Causality. 2nd ed. Cambridge: Cambridge University Press; 2009. doi: 10.1017/CBO9780511803161 [DOI] [Google Scholar]
- 33.Munger ME, Chen BP-J, MacWilliams BA, McMulkin ML, Schwartz MH. Comparing the effects of two spasticity management strategies on the long-term outcomes of individuals with bilateral spastic cerebral palsy: a multicentre cohort study protocol. BMJ open. 2019;9: e027486. doi: 10.1136/bmjopen-2018-027486 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Bohannon RW, Smith MB. Interrater reliability of a modified Ashworth scale of muscle spasticity. Physical Therapy. 1987;67: 206–207. doi: 10.1093/ptj/67.2.206 [DOI] [PubMed] [Google Scholar]
- 35.Kendall FP. Muscles: Testing and function with posture and pain. Baltimore, MD: Lippincott Williams & Wilkins; 2005. [Google Scholar]
- 36.van der Ploeg RJ, Oosterhuis HJ. The “make/break test” as a diagnostic tool in functional weakness. Journal of Neurology, Neurosurgery, and Psychiatry. 1991;54: 248–251. doi: 10.1136/jnnp.54.3.248 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Fowler EG, Staudt LA, Greenberg MB, Oppenheim WL. Selective Control Assessment of the Lower Extremity (SCALE): development, validation, and interrater reliability of a clinical tool for patients with cerebral palsy. Dev Med Child Neurol. 2009;51: 607–614. doi: 10.1111/j.1469-8749.2008.03186.x [DOI] [PubMed] [Google Scholar]
- 38.Kerr AM, Kirtley SJ, Hillman SJ, van der Linden ML, Hazlewood ME, Robb JE. The mid-point of passive hip rotation range is an indicator of hip rotation in gait in cerebral palsy. Gait Posture. 2003;17: 88–91. doi: 10.1016/s0966-6362(02)00056-5 [DOI] [PubMed] [Google Scholar]
- 39.Keenan WN, Rodda J, Wolfe R, Roberts S, Borton DC, Graham HK. The static examination of children and young adults with cerebral palsy in the gait analysis laboratory: technique and observer agreement. Journal of Pediatric Orthopedics Part B. 2004;13: 1–8. doi: 10.1097/00009957-200401000-00001 [DOI] [PubMed] [Google Scholar]
- 40.Novacheck TF, Trost JP, Sohrweide S. Examination of the Child with Cerebral Palsy. Orthopedic Clinics of North America. 2010;41: 469–488. doi: 10.1016/j.ocl.2010.07.001 [DOI] [PubMed] [Google Scholar]
- 41.Mudge AJ, Bau KV, Purcell LN, Wu JC, Axt MW, Selber P, et al. Normative reference values for lower limb joint range, bone torsion, and alignment in children aged 4–16 years. J Pediatr Orthop B. 2014;23: 15–25. doi: 10.1097/BPB.0b013e328364220a [DOI] [PubMed] [Google Scholar]
- 42.Schwartz MH, Rozumalski A. The Gait Deviation Index: a new comprehensive index of gait pathology. Gait Posture. 2008;28: 351–357. doi: 10.1016/j.gaitpost.2008.05.001 [DOI] [PubMed] [Google Scholar]
- 43.Steele KM, Rozumalski A, Schwartz MH. Muscle synergies and complexity of neuromuscular control during gait in cerebral palsy. Developmental Medicine & Child Neurology. 2015;57: 1176–1182. doi: 10.1111/dmcn.12826 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Baba K, Shibata R, Sibuya M. Partial Correlation and Conditional Correlation as Measures of Conditional Independence. Australian & New Zealand Journal of Statistics. 2004;46: 657–664. doi: 10.1111/j.1467-842X.2004.00360.x [DOI] [Google Scholar]
- 45.Gelman A, Hill J, Vehtari A. Section 16.4. Regression and Other Stories. Cambridge: Cambridge University Press; 2020. doi: 10.1017/9781139161879 [DOI] [Google Scholar]
- 46.Noble JJ, Gough M, Shortland AP. Selective motor control and gross motor function in bilateral spastic cerebral palsy. Developmental Medicine and Child Neurology. 2019;61: 57–61. doi: 10.1111/dmcn.14024 [DOI] [PubMed] [Google Scholar]
- 47.Schwartz MH, Rozumalski A, Steele KM. Dynamic motor control is associated with treatment outcomes for children with cerebral palsy. Developmental Medicine & Child Neurology. 2016. [cited 25 Apr 2016]. doi: 10.1111/dmcn.13126 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Choi JY, Kim SK, Park ES. The Effect of Botulinum Toxin Injections on Gross Motor Function for Lower Limb Spasticity in Children with Cerebral Palsy. Toxins. 2019;11. doi: 10.3390/toxins11110651 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Pin T, McCartney L, Lewis J, Waugh M. Use of intrathecal baclofen therapy in ambulant children and adolescents with spasticity and dystonia of cerebral origin: a systematic review. In: Developmental medicine and child neurology [Internet]. Dev Med Child Neurol; Oct 2011 [cited 14 Oct 2020]. doi: 10.1111/j.1469-8749.2011.03992.x [DOI] [PubMed]
- 50.Hanna SE, Bartlett DJ, Rivard LM, Russell DJ. Reference curves for the Gross Motor Function Measure: percentiles for clinical description and tracking over time among children with cerebral palsy. Phys Ther. 2008;88: 596–607. doi: 10.2522/ptj.20070314 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Palisano R, Rosenbaum P, Walter S, Russell D, Wood E, Galuppi B. Development and reliability of a system to classify gross motor function in children with cerebral palsy. Developmental Medicine and Child Neurology. 1997;39: 214–223. doi: 10.1111/j.1469-8749.1997.tb07414.x [DOI] [PubMed] [Google Scholar]
- 52.Rajagopal A, Kidziński Ł, McGlaughlin AS, Hicks JL, Delp SL, Schwartz MH. Estimating the effect size of surgery to improve walking in children with cerebral palsy from retrospective observational clinical data. Scientific Reports. 2018;8: 16344. doi: 10.1038/s41598-018-33962-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53.Wang H-Y, Yang YH. Evaluating the responsiveness of 2 versions of the gross motor function measure for children with cerebral palsy. Arch Phys Med Rehabil. 2006;87: 51–56. doi: 10.1016/j.apmr.2005.08.117 [DOI] [PubMed] [Google Scholar]
- 54.Luz CL, Moura MCDS de, Becker KK, Teixeira RAA, Voos MC, Hasue RH. The relationship between motor function, cognition, independence and quality of life in myelomeningocele patients. Arq Neuropsiquiatr. 2017;75: 509–514. doi: 10.1590/0004-282X20170088 [DOI] [PubMed] [Google Scholar]