Pediatric Caregiver Version of the Shared Decision Making Process Scale: Validity and Reliability for ADHD Treatment Decisions

Abstract

Objective:

Shared decision making (SDM) is recommended for common pediatric conditions; however, there are limited data on measures of SDM in pediatrics. This study adapted the SDM Process scale and examined validity and reliability of the scale for attention-deficit/hyperactivity disorder (ADHD) treatment decisions.

Methods:

Cross-sectional survey of caregivers (n = 498) of children (aged 5–13) diagnosed with ADHD, who had made a decision about ADHD medication in the last 2 years. Surveys included the adapted SDM Process scale (scores range 0–4, higher scores indicate more SDM), decisional conflict, decision regret, and decision involvement. Validity was assessed by testing hypothesized relationships between these constructs. A subset of participants was surveyed a week later to assess retest reliability.

Results:

Pediatric Caregiver version of the SDM Process scale (M = 2.8, SD = 1.05) showed no evidence of floor or ceiling effects. The scale was found to be acceptable (<1% missing data) and reliable (intraclass correlation coefficient = 0.74). Scores demonstrated convergent validity, as they were higher for those without decisional conflict than those with decisional conflict (2.93 vs 2.46, P < .001, d = 0.46), and higher for caregivers who stated they made the decision with the provider than those who made the decision themselves (3.0 vs 2.7; P = .003). Higher scores were related to less regret (r = −0.15, P < .001), though the magnitude of the relationship was small.

Conclusions:

The adapted Pediatric Caregiver version of the SDM Process scale demonstrated acceptability, validity and reliability in the context of ADHD medication decisions made by caregivers of children 5–13. Scores indicate pediatricians generally involve caregivers in decision making about ADHD medication.

Attention-deficit/hyperactivity disorder (ADHD) is the most common chronic behavioral condition of childhood and impacts the daily life of both the child with ADHD and their caregivers.^1–3 Commonly diagnosed in school-aged children, there are multiple reasonable treatment options for children, such as counseling, medication, family-directed behavioral modifications or some combination of these.⁴ Families may have goals and values related to ADHD itself or the treatment options that influence their preferences. Such preferences may be concordant with treatment guidelines (eg, a family wanting medication along with behavioral classroom interventions⁴), or discordant with treatment guidelines (eg, a family preferring to avoid stimulant medications, which are the first-line treatment for ADHD). Because of these multiple options that vary in ways that would be likely to matter to families, a shared decision making (SDM) approach should be used to ensure that caregivers and healthcare providers make an informed and preference-concordant decision for the child.^5,6

SDM is a process by which to engage patients or their family members in deciding between possible treatment options. By combining clinical evidence with patient’s symptoms, goals, and preferences, SDM allows the patient, family members and healthcare providers to cooperatively identify the best option for the patient. Although SDM interventions in pediatrics have been shown to improve knowledge and reduce decisional conflict, there is still a need for a metric to identify how much SDM is experienced by caregivers.⁷

However, measuring SDM in pediatric settings may be more challenging than in adult settings. Caregivers, often parents, are making choices for their children and surrogate decision making for children is different than making decisions for oneself.⁸ For example, when making medical decisions for themselves, individuals tend to be more risk-tolerant than when asked to make decisions for others.⁹ Further, caregivers can have unique SDM needs when making decisions for their children, such as the need to establish a decision time horizon (eg, accounting for factors such as age, but also scheduling constraints) or explicitly acknowledging the uncertainty in decisions and identifying viable alternatives if a problem arises.¹⁰ Although measures of SDM have been adapted for use in pediatric decisions, data on their psychometric properties in the pediatric setting are limited. Psychometric properties such as feasibility (the extent to which a measure can be completed), acceptability (the extent to which the target audience is willing to complete the measure), convergent validity (the extent to which the measure is related to another measure that measures a related construct), divergent validity (the extent to which the measure is not related to another measure that measures an unrelated construct), and retest reliability (the extent to which a measure will provide consistent results for the same individual) are important in evaluating measures. The 9-item Shared Decision Making Questionnaire¹¹ was found to be reliable and to have moderate divergent validity in a pediatric population, but showed poor convergent validity. The pediatric adaptation of CollaboRATE showed moderate convergent validity and divergent validity but poor reliability.¹² The OPTION scale for coding observed behaviors has been adapted for use in pediatric populations,^6,13,14 but has not yet been psychometrically evaluated in the pediatric context. There are currently no measures of SDM that have been shown to have both strong validity and reliability in pediatric populations. Without a reliable and validated measure of SDM it is difficult to understand the decision making experience of patients and families, the SDM skills of healthcare providers, or the impact of interventions designed to improve SDM.

The Shared Decision Making (SDM) Process Scale is a short, patient reported measure that has been used extensively to examine the amount of SDM in adult populations (aged 40 and older) for common medical conditions.¹⁵ The existing version contains items that assess the amount of discussion of options, pros, cons and preferences between patients and their health care providers. The generalizability of the scale is limited by the lack of any data in pediatric setting, as its use has been predominantly with adult respondents reporting on their own decision making processes.

The purpose of this study is to adapt the SDM Process Scale for caregiver use in a pediatric setting and to gather evidence of validity and reliability of the scores in a sample of caregivers who have made treatment decisions for children with ADHD.

Methods

Adapting the items to Pediatric setting

We conducted three rounds of interviews with caregivers (n = 5 Round 1; n = 6 Round 2; n = 5 Round 3) to adapt the instructions and wording of the existing SDM Process survey to a pediatric situation. The goal of the interviews was to understand if the questions are working as intended—whether the respondents understand what is being asked, whether they have the information needed to answer the question, whether they can figure out how to answer the question, and whether they are willing and able to provide an answer.

Caregivers were eligible for participation in the in-depth interview if they had a child aged 3–15 diagnosed with ADHD and made a treatment decision about their child’s ADHD in the last 2 years. Respondents for the interviews were recruited from the Center for Survey Research volunteer database, Craigslist, and word-of-mouth and the 3 rounds were conducted between August 2020 and January 2021.

The Round 1 interviews revealed that discussions involving decisions with the healthcare team most often involved medications. Thus, items were adapted to focus on the most recent decision made about ADHD medicines and explored the comprehension of instructions and wording of the Pediatric Caregiver version of the SDM Process items as well as best wording to describe the various non-medication treatment options (counseling vs behavioral treatment vs therapy). Generally, caregiver interviewees were able to understand and answer the Pediatric Caregiver version of the SDM Process Scale items as intended. Some respondents had multiple decision points within the time frame (“in the last 2 years”) and this led to some difficulty focusing on one decision to answer about. To address this challenge, we added a question that allowed respondents to specify the main decision they made about ADHD medicine in the last 2 years. The supplemental material contains the original and final versions of the Pediatric Caregiver version of the SDM Process Scale.

Cross-sectional Survey

Sample and Recruitment

We worked with a national sampling firm to obtain a target of 500 responses. Eligible participants were caregivers who self-identified as having at least one child between the ages of 5 and 13 years old with ADHD and who had talked with a health care provider about starting, stopping, or changing an ADHD medicine in the last 2 years. The sample was not a national probability sample; however, the recruitment efforts were managed to be geographically diverse and to have at least 25% non-White respondents.

Design

The online survey was fielded between February 2021 and March 2021. Participants completed screener questions to determine eligibility. Then, participants were asked to identify the main decision they made about ADHD medicine over the past two years (“In the last 2 years, what was the main decision you made about using medicine to treat your child’s ADHD?”; see Table 1) and were instructed to respond to the rest of the items thinking about that main decision. These items are listed below in the order they were completed. Starting 7 days after they completed the survey, respondents were invited to complete a brief follow-up survey to examine short term test-retest reliability. The goal of the retest was to obtain 200 completed surveys.

Table 1.

Main Decision Regarding Treatment for ADHD in the Past 2 Years

	Frequency % (n)	Pediatric Caregiver SDM Process Score
1. Talked about starting an ADHD medicine but decided not to start at this time	33.0% (164)	2.8 (1.0)
2. Started taking an ADHD medicine for the first time	28.5% (142)	2.9 (1.1)
3. Changed the type or dose of current ADHD medicine	22.1% (110)	2.9 (1.0)
4. Stopped taking ADHD medicine	8.4% (42)	2.5 (1.1)
5. Talked about stopping or changing an ADHD medicine but decided to continue for now	8.0% (40)	2.7 (1.1)

Measures

Pediatric Caregiver version of the SDM Process scale

Four items cover discussion of options, pros, cons, and preferences.¹⁵ The items measured whether or not a specific behavior had occurred during an interaction with a healthcare provider about the treatment decision (see Table 2 for all items). Items were scored and summed in accordance with previously published work and a total score between 0 and 4 was calculated.¹⁵ Higher scores indicate more SDM occurred. Participants with missing answers were not given a Pediatric Caregiver SDM Process score.

Table 2.

Pediatric Caregiver Version of the SDM Process Scale Items and Responses

Pediatric Caregiver version of the SDM Process Scale Item	A Lot N (%)	Some N (%)	A Little N (%)	Not At All N (%)
How much did you and your child’s healthcare providers talk about the reasons you might want your child to take medicine to treat their ADHD?	250 (50.2)	181 (36.3)	53 (10.6)	14 (2.8)
How much did you and your child’s healthcare providers talk about the reasons you might not want your child to take medicine to treat their ADHD?	165 (33.1)	204 (41.0)	97 (19.5)	32 (6.4)
	Yes	No
	N (%)	N (%)
Did any of your child’s healthcare providers talk about ways other than medicine to treat their ADHD?	365 (73.3)	133 (26.7)
Did any of your child’s healthcare providers ask you whether or not you wanted your child to take medicine to treat their ADHD?	421 (84.5)	77 (15.5)

Decisional Conflict

The SURE scale is the 4-item short form of the Decisional Conflict Scale that measures the level of uncertainty about which treatment to choose. Each item is scored as “Yes” (1 point) or “No” (0 points) with total scores ranging from 0 to 4. A total score of 4 indicates no decisional conflict.^16,17 SURE has been shown to correlate positively with the SDM Process score in a range of surgical decisions,¹⁵ as well as for decisions surrounding medication to manage depression.¹⁸ The longer Decisional Conflict scale has been used to measure decisional conflict both in surrogate decision makers,¹⁹ as well as in pediatric populations.²⁰

Decision Regret

One item assessed whether or not respondents would make the same decision again. Response options were on a scale from 1 (definitely make same decision again) to 4 (definitely NOT make same decision again). This item has not been used in pediatric studies, but has been used in previous studies by the authors and has been shown to be inversely related to the SDM Process scores for surgical decisions.¹⁵

Modified Control Preferences Scale

One-item, modified by the researchers and used in prior work,²¹ asking who made the decision, with response options: “Mainly you,” “Mainly your child’s healthcare provider,” “You and the healthcare provider made the decision together,” and “Mainly someone else.”²² This version has not yet been used in a pediatric context, however the original Control Preference Scale been used in pediatric contexts.^23,24

ADHD severity

The Parent Informant Performance subscale of the National Institute for Children’s Health Quality (NICHQ) Vanderbilt Assessment Scales assessed ADHD severity.²⁵ Score range from 8 to 40 with higher scores indicating more problematic behavior.

Demographic and other information

Participants were also asked to report on demographic information, current ADHD treatment(s), prior AHDH treatment(s), and years with ADHD.

Consent was implied by completion of the survey. The study was approved by the Institutional Review Board at Mass General Brigham (protocol# 2019P001434) and was registered as an observational study at ClinicalTrials.gov (ID# NCT05048186).

Analysis

First, we examined descriptive results for the main decision item and the Pediatric Caregiver SDM Process score. For the Pediatric Caregiver SDM Process score we looked to see whether the scores spanned the range of total possible scores, were normally distributed, and whether there was evidence of floor or ceiling effects. Using a One-way ANOVA we tested for differences in Pediatric Caregiver SDM Process scores by main decisions.

Then we examined the convergent validity and reliability of the scale. As there is no gold-standard for measuring SDM, we tested the following hypotheses to examine validity of the scale by looking at relationships between known constructs such as decision conflict and decision regret.

Validity Hypotheses

1. Patients with higher Pediatric Caregiver SDM Process scores (≥0.33SD) were more likely to receive a top SURE score.

2. Patients with higher Pediatric Caregiver SDM Process scores had less regret (r≥–0.50).

3. Respondents who reported that their decision making process was mainly driven by the physician had lower Pediatric Caregiver SDM Process scores (≥0.33SD) than respondents who reported that they were involved and engaged in selecting treatment.

For hypothesis 1, we compared mean Pediatric Caregiver SDM Process scores for those who were SURE (score = 4) and those who were not (score <4) with a Welch’s 2-sample t-test since Pediatric Caregiver SDM Process scores did not meet the assumption of homogeneity between the two SURE groups needed to complete an independent t-test. For hypothesis 2, we used a Pearson’s correlation to determine the relationship between Pediatric Caregiver SDM Process and regret. ANOVA determined if there were differences in Pediatric Caregiver SDM Process scores by who made the decision.

We assessed retest reliability using Intraclass correlation coefficient (ICC; ICC >0.7 indicating sufficient reliability).²⁶

Results

Five-hundred and twelve participants completed the survey. Nine respondents were removed for inconsistent answers (e.g. indicating their main decision was to start taking medication but they reported their child was not on medication currently and had not been on medication in the past) and 5 were removed for missing responses to the Pediatric Caregiver version of the SDM Process scale for a total of 498 in the analytic sample. Two hundred fifteen participants completed the retest. Six retest participants were removed for inconsistent responses and one for missing responses to the Pediatric Caregiver version of the SDM Process scale for a total of 208 in the analytic retest sample. Participant characteristics are included in Table 3. No significant differences were found between characteristics or participants who participated at baseline and retest (not shown).

Table 3.

Participant Characteristics

	Main survey n = 498	Retest n = 208
Caregiver Age Mean (SD)	37.6 (5.5)	38 (5.5)
Child Age Mean (SD)	9.5 (2.6)	9.5 (2.5)
Caregiver Female N (%)*	338 (68.0)	149 (71.6)
Child Female N (%)	169 (33.9)	67 (32.2)
Caregiver Race and Ethnicity: N (%)
Hispanic Only	39 (7.8)	13 (6.2)
White, non Hispanic	337 (67.7)	137 (65.9)
Black	64 (12.9)	32 (15.4)
Asian	20 (4.0)	11 (5.3)
Pacific Islander	7 (1.4)	0 (0)
American Indian	4 (0.8)	3 (1.4)
Other	9 (1.8)	3 (1.4)
Multiple Races	18 (3.7)	9 (4.3)
Caregiver Education N (%)*
Less than high school	16 (3.3)	5 (2.4)
High school graduate or GED	102 (20.8)	41 (19.7)
Some college or 2-year degree	176 (35.9)	72 (34.6)
4-year college graduate	125 (25.5)	56 (26.9)
More than a 4-year college degree	71 (14.5)	34 (16.3)
Income N (%)*
<25 k	69 (14.3)	31 (15)
25–50 k	104 (21.5)	43 (20.9)
50–75 k	132 (27.3)	60 (29.1)
75–100k	89 (18.4)	37 (18)
>100 k	90 (18.6)	35 (17)
Child currently taking ADHD medication N (%)	297 (59.6)	134 (64.4)
Child ever taken ADHD medication N (%)*	343 (69.7)	155 (74.5)
Years since first diagnosed with ADHD Mean (SD)	3.9 (2.2)	3.9 (2.1)
ADHD Severity Mean (SD)	21.7 (6.3)	21.5 (6.8)

SD indicates standard deviation.

^*One caregiver did not report on their gender in the main survey; 8 caregivers did not report their education in the main survey; 5 caregivers did not report their income in the main survey; 2 caregivers did not report their income at retest; 8 caregivers did not report on their child ever taking medication in the main survey; 8 caregivers in the main survey and 1 at retest did not report on the years since their child’s diagnosis; 15 caregivers in the main survey and 5 at retest did not complete the ADHD severity measure.

Most caregivers stated the main decision they made in the last two years was deciding not to start an ADHD medication at that time (33.0%), followed by starting to take an ADHD medicine for the first time (28.5%), or changing the type or dose of ADHD medicine (22.1%; see Table 1).

Pediatric Caregiver SDM Process scores covered the entire scale, ranging from 0 to 4 (M = 2.8, SD = 1.05). The scores were not overly skewed nor was their evidence of ceiling or floor effects (skew = −0.84). Most patients stated they were told ‘a lot’ about the reasons to use medication to manage their child’s ADHD (50.2%). Fewer stated they were told ‘a lot’ about the reasons not to use medication to manage their child’s ADHD (33.1%). The majority (73.3%) were told they had options to manage their child’s ADHD and almost all were asked what they wanted to do to manage their child’s ADHD; see Table 2. There were no significant differences in Pediatric Caregiver SDM Process scores by the main decision selected by the caregiver (see Table 1; F(4,493) = 2.03, P = .089, η² = 0.02).

Tests of Validity

Most of the sample (71.8%, 348/485) had SURE top scores indicating no decisional conflict. Those who had SURE top scores had higher Pediatric Caregiver SDM Process scores (M = 2.93, SD = 0.94) than those who did not have SURE top scores (M = 2.46, SD = 1.22, t(203) = 4.04, P< .001,d = 0.46).

Caregiver regret was not overly skewed (skew = 0.86); average regret was 1.73 (SD = 0.73). There was a significant correlation between Pediatric Caregiver SDM Process and regret indicating that as Pediatric Caregiver SDM Process increased regret decreased (n = 497, r = −0.15, P< .001).

Forty-four percent of caregiver respondents indicated the decision was made jointly with the healthcare provider, 46.1% indicated the decision was made by themselves, 9.9% of caregiver respondents indicated that the decision was made by the health care provider, and <1% of caregiver respondents indicated the decision was made by someone else. Given the low number of participants indicating the decision was made by someone else (n = 2), this group was removed from consideration in the following analysis. There was a significant difference in Pediatric Caregiver SDM Process scores between decision groups, F(2,492) = 6.29, P = .002, η² = 0.02. Those who stated they made the decision with the provider had higher scores (Mean = 3.0, SD = 0.9) compared to those who stated they mainly made the decision themselves (Mean = 2.7, SD = 1.1; P = .003). No other group differences were significant (ps > 0.10).

Reliability

Test-retest reliability was found to be acceptable; ICC = 0.74, 95% CI (0.67, 0.79).

Discussion

This study provides the first evidence supporting the reliability and validity of the Pediatric Caregiver version of the SDM Process scale in pediatric ADHD populations. Only minor modifications to the scale were needed to adapt for caregiver respondents. The scale was highly acceptable to caregivers (as seen in the low number of patients who did not complete the scale), demonstrated adequate retest reliability and met two of the three tests of validity. Validity was demonstrated with higher Pediatric Caregiver SDM Process scores associated with less decisional conflict and with higher self-reported decision involvement, but while the scores were also associated with less regret, the magnitude of the relationship was smaller than hypothesized. Further, the Pediatric Caregiver SDM Process scores were generally high, suggesting that the pediatricians or the nature of this decision context lend themselves to meaningful involvement of caregivers in the selection of treatment.

The Pediatric Caregiver SDM Process scores in this sample are higher than those found for chronic disease medication decisions in adults. For example, in a meta-analysis of medication decisions, the average SDM Process score ranged from 1.4 for one high blood pressure medication study to 2.7 for another high blood pressure medication study, compared to the average of 2.8 found here for ADHD (H. Vo, unpublished data). The higher scores here suggest that pediatricians may be better at engaging caregivers in SDM than adult primary care physicians. This is consistent with earlier work indicating that parents of children with ADHD and healthcare providers were in favor of SDM, and that more than half of children’s caregivers reported high participation in SDM.^5,27 Nevertheless, there are opportunities for improvement. For instance, about a quarter of caregivers indicated that their child’s healthcare provider did not talk about ways other than medicine to treat their ADHD and about a quarter indicated that healthcare providers spoke little or not at all about the reasons they might not want their child to take medicine to treat their ADHD. This may be due to providers’ assumptions that families want medication or the emphasis on medication treatment in current treatment guidelines.⁴ Nevertheless, ensuring caregivers know their options and both reasons for and against each option is necessary to ensuring caregivers are engaged and informed.

Several SDM measures exist, all of which have been developed and evaluated primarily in the adult setting. Adapting to the pediatric context required reframing items to make sense for a surrogate decision maker, but overall, the changes to the existing survey were minor. In this sample, higher Pediatric Caregiver SDM Process scores were associated with less decisional conflict and with higher self-reported decision involvement. Although the scores were also associated with less regret, the magnitude of the relationship was smaller than expected. This runs counter to meta-analyses that suggest interventions to improve SDM do reduce decisional regret in this population,⁷ and prior studies of the SDM Process scale that found the scale to be inversely related to regret when used in the context of adult medication decisions (H. Vo, unpublished data). Although the results overall suggest the Pediatric Caregiver version of the SDM Process scale is a valid measure of SDM, it is necessary to further test and refine this version of the scale as features (eg, that these are decisions made for another, and not for oneself, the time between the decision and the report may be quite long) may be impacting our ability to find the expected relationship.

This study has some limitations. First, the online sample may not be generalizable to the larger population of caregivers. We did not receive clinical confirmation of the diagnosis of ADHD, did not identify subtypes of ADHD, and did not measure potential comorbidities present in our sample which may therefore not be generalizable. Second, given the retrospective nature of the survey, the responses may be subject to recall bias. This also may play a role in the lack of relationship found between Pediatric Caregiver SDM Process scores and regret given the distance between the event and the current emotional state. Third, the framing of the survey required at least some discussion of medication, but there are several possible treatment options that do not involve medication for ADHD that this may miss. Fourth, as this survey was conducted during the COVID-19 pandemic many children may not have been in school at the time of the study which may limit generalizability of the results. Finally, this scale focused only on one caregiver’s perspective (for their child aged 5–13) and did not incorporate multiple caregivers’, the child’s, or the pediatrician’s perspective, nor did it focus on caregivers of older-aged pediatric patients. Future work is needed to explore how the Pediatric Caregiver version of the SDM Process scale performs in clinical settings, how responses compare when assessed closer to the actual decision point, whether the survey is valid for children or adolescents to complete directly, and the relationship between the Pediatric Caregiver version of the SDM Process scale and decision concordance and medication adherence.

Conclusion

The Pediatric Caregiver version of the SDM Process scale is shown to be a valid and reliable caregiver-reported measure of SDM in the context of ADHD medication decisions for children 5–13. Armed with this measure of SDM, researchers can better understand the current state of SDM in pediatric care, provide individualized feedback to healthcare providers who care for these populations regarding their strengths and areas for improvement, and measure effectiveness of interventions designed to promote SDM.

What’s New.

This work demonstrates that in the context of attention-deficit/hyperactivity disorder, shared decision making can be feasibly, reliably, and validly measured with the Pediatric Caregiver version of the Shared Decision Making Process scale.