A Best-Worst Scaling Experiment to Prioritize Caregiver Concerns About Attention-Deficit/Hyperactivity Disorder (ADHD) Medication for Children

Abstract

Objective

The objective of this feasibility study was to develop and pilot an instrument to elicit caregivers’ priorities when initiating attention-deficit/hyperactivity disorder (ADHD) medication for their child.

Methods

A best-worst scaling experiment was conducted to assess the feasibility of measuring trade-offs among competing priorities when initiating ADHD medicine. Forty-six participants were recruited for this two-phase study: 21 in the survey development and 25 in the survey pilot. Data were analyzed as best-worst scores, 95% confidence intervals and t-tests for significance testing.

Results

The significant best-worst scores indicated choices were purposeful. The best-worst score ranking identified priority concerns. Most important was their child becoming a successful adult, a doctor addressing their concerns, and their child’s school behavior improving.

Conclusions

The best-worst scaling survey was acceptable to participants and generated proof that this method can elicit priorities for children’s mental health treatment. Future work using this method will guide family-centered care.

INTRODUCTION

Attention-deficit/hyperactivity disorder (ADHD) now affects 11% of US children aged 17 or younger (1, 2) and 3.5 million are prescribed a stimulant medication.(2) Children often need medication, yet among caregivers there is both low acceptability of and much uncertainty about using medication for their child.(3–7) Even when medication is initiated, many caregivers discontinue use within two years.(3, 4, 8)

Several studies have focused on caregivers’ perceptions of treatment for ADHD, and most have focused on low-income, minority families. Caregivers do not initially use medication, reluctantly turn to medication only after exhausting all other options, and do not always view ADHD medication as appropriate for children.(3–5) However, prior research has not elicited caregivers’ priorities that may influence decisions to initiate medication for their child.(7, 9) Therefore, this feasibility study aimed to develop and pilot a best-worst scaling instrument that would assess caregivers’ priorities when initiating ADHD medicine for their child. The _________ Institutional Review Board (IRB) approved the study and granted a waiver of informed consent.

METHODS

Mixed methods were used to develop and test a best-worst scaling instrument to elicit caregivers’ priority concerns when deciding whether to use ADHD medication for their child. Best-worst scaling was preferred to a conjoint discrete choice experiment, often used in the healthcare research, (10, 11) for several reasons. Grounded in random utility theory, best-worst scaling evokes tradeoffs by asking individuals to select one best and one worst attribute among competing alternatives within a profile. By comparison, conjoint experiments force selections among two or more different profiles. With best-worst scaling, individuals prioritize the options and select attributes that are of greatest value to them relative to other shown attributes; as a result more information is gained about an individual’s utility function. (12–14) This provides more enriched information on heterogeneity surrounding specific priority concerns than can be attained from selecting one profile in a conjoint, which contains multiple priorities in a profile.(12–14) In addition, a best-worst scaling allows estimation of attribute average utility, which can be compared across all attributes, whereas the reference in a conjoint is the whole scenario.(14)

Two separate convenience samples were recruited from two support organizations, within metropolitan Baltimore, for caregivers of children with mental health needs. First, a sample of 21 caregivers participated in focus groups as part of the best-worst scaling instrument development. A second sample of 25 caregivers of children aged four to 14 with an ADHD diagnosis participated in the best-worst scaling instrument pilot. The two samples were very similar in demographics. The majority was African-American and most were the child’s biological mothers.

Attribute identification used data from a previous qualitative study of caregivers’ experiences with coming to terms with the ADHD diagnosis and medication treatment.(3, 4) This prior work generated a model of caregivers’ priorities in initiating medication that was grounded in their views of the appropriateness of treatment (i.e., is my child too young), the anticipated effects (i.e., will this harm my child), and the symbolic representation (i.e., using medicine does not mean I am a bad parent).(3) This model was cross-referenced with the published literature(5, 6) to generate a list of attribute statements for the best-worst scaling instrument.

In October 2012, a family support group leader from one of the family organizations recruited caregivers for the first sample. Caregivers were asked to participate in focus groups to assess attribute statement relevance. Fifteen caregivers participating in the first of two focus groups were presented with 26 attribute statements reflecting potential priorities caregivers may have when considering whether to initiate ADHD medication for their child. They were asked to classify the statements into one of the four categories (i.e., short-term concern; long-term impact; societal views; supportive network) or to suggest a new category if needed. Attribute statements, revised based on this feedback, were presented to a second focus group (N=6) from the same support organization for verification and relevance. No further amendments were suggested.

Sixteen attribute statements were retained for the best-worst scaling instrument. These were divided evenly into the four abovementioned categories with two positively and two negatively phrased statements per category. Two child psychiatrists reviewed the clinical and practical relevance of the attribute statements.

A balanced incomplete block design was used to construct the choice task profiles so that each attribute statement was seen the same number of times and any two attribute statements appeared together the same number of times. This ensured equal probability of selection for each attribute statement. The survey had 16 choice task profiles, each displaying six of the 16 attribute statements (see online appendix Figure 1). In each choice task profile, participants were asked to think back to when they first learned of their child’s ADHD diagnosis and some of the situations that influenced their decision to initiate ADHD medication. They were instructed to select one of the six attribute statements that reflected the best (i.e., most important) and then select one attribute statement that reflected the worst (i.e., least important) concern that influenced their decision to initiate ADHD medication for their child.

The family support group leader from a different organization helped to recruit caregivers for the pilot. A paper-and-pencil best-worst scaling instrument pilot was conducted from November 2012 to January 2013. Participants (N=25) of children aged 4–14 and diagnosed with ADHD were recruited from five family support groups in the Baltimore metropolitan area. All caregivers used medication for their child. Most also were currently using psychotherapy or had an individualized education plan.

The Principal Investigator and a graduate research assistant attended the support group meetings, explained the purpose of the survey, and provided instructions for completing the choice tasks. The pilot survey was completed, on average, in 15 minutes. At the conclusion of the meeting, participants were asked to provide feedback regarding the clarity and relevance of the choice task profiles. No further modifications were recommended.

Survey responses were coded into two binary variables: best (1/0) and worst (1/0). Best-worst scores were calculated for each attribute statement as the sum of the best minus the sum of the worst selections across all respondents,(15) divided by 150 (each attribute statement was displayed six times multiplied by 25 participants). A t-test assessed if scores differed significantly from 0 (α=0.05), which would imply that selections were not made at random but reflected stated priorities.

RESULTS

Positive best-worst scores indicated the attribute statement was selected as best more frequently than it was selected as worst, and conversely for a negative score. The number of times each attribute statement was chosen as best and worst, the best-worst score, and 95% confidence intervals are shown in Table 1. All attribute statements, except “ADHD medicine is not needed to control my child’s home behavior” were significant at p<.05.

Table 1.

Best-Worst Scores and 95% Confidence Intervals for 16 Attribute Statements, Ranked by Relative Importance Within Category

Attribute Statements^	Mean Score	95% CI	Best	Worst	P
Short-Term Concerns
ADHD medicine is needed to control my child’s school behavior	.39	.35 –.43	58	0	<.001
ADHD medicine side effects outweigh its benefits	.23	.19 –.26	37	3	.038
ADHD medicine will help my child get better grades	.07	.03 –.10	21	11	<.001
ADHD medicine is not needed to control my child’s home behavior	−.05	−.02 – −.09	9	17	.057
Long-Term Concerns
ADHD medicine will help my child be a successful adult	.41	.36 –.45	63	2	<.001
ADHD medicine has risks that will affect my child’s future health	.28	.24 –.32	43	1	<.001
ADHD medicine will help my child finish high school	.15	.12 –.19	28	5	<.001
ADHD medicine will limit my child’s career options	−.09	−.06 – −.13	8	22	.005
Supportive Network
The doctor addresses my concerns about ADHD medicine	.29	.25 –.33	46	2	<.001
The school has pressured me to use ADHD medicine in my child	−.30	−.29 – −.37	5	50	<.001
My family does not see why my child needs ADHD medicine	−.33	−.29 – −.37	2	52	<.001
My friends agree with me using ADHD medicine in my child	−.43	−.38 – −.47	4	68	<.001
Societal Views
ADHD medicine will help my child get along with others	.25	.21 – .30	45	7	<.001
ADHD medicine will hurt my child’s self-esteem	−.05	−.02 – −.07	4	11	.034
Giving my child ADHD medicine does not mean I am a bad parent	−.30	−.25 – −.35	18	63	<.001
Others will think badly of my child if he/she uses ADHD medicine	−.53	−.48 – −.57	5	84	<.001

Mean score: best – worst/150

^{^}Ranked by priority importance within category

Best-worst score ranking from largest to smallest score was used to determine relative attribute importance. Overall, medication to help their child become a successful adult (.41) was the highest and others thinking badly of the child if he/she uses ADHD medicine (−.53) was the lowest ranked score (see online appendix Figure 2). Attribute importance within each category is displayed in Table 1. Control of school behavior was the highest ranked short-term concern (p<.001). A key long-term outcome concern was “ADHD medicine will help my child be a successful adult” (p<.001). The only positive score in the supportive network category was having a doctor that addressed the caregivers’ concerns about ADHD medicine (.29; p<.001). Family, friends, and school personnel were less important influences relative to the other attributes (p<.001). The influence of societal views were generally less important relative to the other attribute statements, with the exception of the child’s peer relations (.25; p<.001).

DISCUSSION

This study demonstrated the feasibility of best-worst scaling for eliciting caregivers’ priorities in initiating medication for their child’s ADHD. Significant best-worst scores indicated that choices were not random selections. Caregivers completed the instrument with relative ease.

The caregiver-centered instrument holds great promise for advancing family-centered research and clinical practice. Children’s mental health services research has been limited by the lack of rigorous methods for eliciting caregiver priorities. Eliciting caregivers’ priorities early in the clinical encounter can guide family-centered treatment planning.

There are several limitations. The sample was limited in diversity, size, and geographic locale and may not generalize to all caregivers of children with ADHD. Although recruitment from different advocacy organizations can result in potentially different samples, convenience sampling was used to recruit caregivers from homogeneous sources. The perspectives reflect the priorities of one-parent (the mother). Despite continuous caregiver feedback, this list of relevant attribute statements may not be exhaustive. However, attribute development was an iterative feedback process with separate individuals to confirm the statements Finally, stated priorities were not correlated with treatment adherence, but this also was not the goal of this feasibility study.

The purpose of this feasibility study was to test the best-worst scaling instrument prior to use in a larger comprehensive survey. The instrument is currently being used in a study that is designed to capture clinical diagnoses and receipt of mental health care services in order to assess the association between priorities and treatment adherence.

CONCLUSIONS

Caregivers’ priorities are nuanced and impact decisions for their child’s or adolescent’s mental healthcare. The methods described here may help to better define, recognize, and understand caregivers’ priorities so that clinicians may engage caregivers in shared-decision making about treatment for their child. This could accelerate caregiver-centered outcomes research in children’s mental health.

Figure 1. Example of a Best-Worst Choice Task Question.

Think back to when you first learned that your child had ADHD.

Which of the items below was MOST IMPORTANT to you and which was LEAST IMPORTANT to you when first deciding to use or not to use ADHD medicine?

Figure 2.

Rank Order of Best-Worst Scores for Across All Attribute Statements