Adolescent-reported changes in provider behavior following pediatrician training in stimulant diversion prevention: Results from a cluster randomized controlled trial

Abstract

Objective.

To test whether pediatrician training leads to provider utilization of stimulant diversion prevention strategies as reported by adolescent patients with ADHD.

Methods.

Pediatric practices received a stimulant diversion prevention workshop (SDP) or continued treatment-as-usual (TAU) in a cluster-randomized controlled trial. Surveys were completed by 341 stimulant-treated patients at baseline and three follow-up assessments.

Results.

In intent-to-treat analyses of patient reports, SDP adolescents reported more provider use of diversion prevention strategies compared to TAU. They also reported more parent-patient communication about diversion. Provider satisfaction with the training was strong.

Conclusions.

Pediatricians can make use of clinical practice strategies for the prevention of stimulant diversion following a one-hour training; findings are novel given their reliance on confidential patient report of provider behavior and increase confidence in the results. Coupled with the positive provider satisfaction ratings, results suggest that this brief workshop may be an option for concerned providers that also has the effect of increasing discussion at home about safe use of stimulants.

Introduction

Non-medical use of prescription stimulant medications (e.g., methylphenidate, amphetamine) for cognitive enhancement or recreation is prevalent among adolescents.^1,2 Between 2005 and 2020, 7.2% of high school seniors nationally took stimulants in the past year without a prescription.³ In schools with more students reporting prescriptions of stimulants for ADHD, prevalence of non-prescribed use of stimulants is higher.³ This finding aligns with pre-existing evidence that one of the most common sources of pills for non-medical use is peers who divert (share, sell, or trade) their own stimulants prescribed for attention-deficit/hyperactivity disorder (ADHD).^1,2,4 Reaching beyond the school environment, peers and family members with ADHD are well-established as the most common source of pills for non-medical use.⁵ Such diversion confers health, social, and legal risks to giver and receiver, including under-treated ADHD and legal consequences for the patient and unmonitored side effects (which can include emergency room visits⁶) from non-prescribed use.^7,8

Pediatricians are the most common prescribers of stimulant medications for ADHD among youth.⁹ The American Academy of Pediatrics provides clinical practice guidelines for the diagnosis, evaluation, and treatment of ADHD in children and adolescents.¹⁰ Monitoring of ADHD symptoms and prescription refill requests for signs of misuse or diversion is recommended, but additional specific suggestions are not provided. Although limited to specialists (e.g., developmental-behavioral pediatricians, child and adolescent psychiatrists), a survey of physicians found that 25% and 48% did not feel qualified to educate their high school-aged patients in the health risks and legal consequences, respectively, of stimulant misuse and diversion.¹¹ Primary care pediatricians are likely to feel even less confident in their skills and in need of training. In general, little research has been available to help guide pediatric practitioners on specific recommended strategies to reduce risk of diversion and recommendations have been made to create more training opportunities.^5,8,12

We recently published a study of stimulant diversion prevention training wherein pediatricians and their staff participated in a one hour workshop on risk reduction strategies. Their self-reported knowledge and skill increased as a function of this training.¹³ Aided by developmental work conducted in pediatric and family medicine practices,¹⁴ we provided on-site education enriched with brief video demonstrations, resource materials, and ample time for discussion (see additional details below in Methods). In this randomized controlled trial (RCT) comparing stimulant diversion prevention training versus usual care, we found moderate-to-large effects of training on provider self-reported use of patient/family education strategies that were specifically included in our workshops. Example strategies included explaining to the patient that they may not have enough medication when needed if it has been shared, sold, or traded; the importance of being prepared for requests to share, sell, or trade their pills; that using stimulant medications without a prescription may be dangerous for friends. Group differences for other physician behaviors were not found (e.g., assessment of ADHD symptoms) and were believed to be a result of pre-existing use of practices already recommended by the AAP.¹⁵ Critically, these findings were based on provider self-reports which are subject to bias or just different from patient experience. In the current study, we seek to provide a much-needed validation of these findings through examination of confidential adolescent patient report of their experiences with their providers. To our knowledge, such complementary data obtained directly from patients is rarely (if ever) provided in the literature on physician practices.

In the quest to prevent adolescent health risk behaviors through primary care, parents are another potential agent of change. In prevention research, involving parents to reduce adolescent health behavior risk is not new.¹⁶ Parents are appreciated for this role due to their ability to be flexible with respect to selection of content and approach (e.g., timing and frequency of intervention delivery) and capacity to monitor unstructured time.¹⁷ Thus, primary care providers may increase their impact by involving parents in their patient interactions. Our stimulant diversion prevention training included tips on this approach (e.g., respect adolescent’s developing need for autonomy by first discussing diversion and misuse privately; video clip demonstrating adolescent-empowering interactions). In addition to examining patient report of provider behaviors, we examined an indirect target of provider training: patient report of parent behaviors including parent-adolescent discussions of diversion prevention and parent monitoring of ADHD medication use.

Data to address the impact of training on physicians were drawn from our RCT of pediatric primary care provider training in clinical practice strategies to reduce diversion risk by adolescent patients stimulant-treated for ADHD. We focus on the primary target of our training: physician behavior as reported by their adolescent patients. We also extend our analyses to patient-reported parent behavior and describe provider satisfaction with training to supplement our findings with information relevant to feasibility of dissemination.

Method

Overview

Seven pediatric practices in southwestern Pennsylvania were randomly assigned to receive a workshop on stimulant diversion prevention (SDP) or to continue treatment-as-usual (TAU) in a cluster-RCT, ClinicalTrials.gov NCT03080259. (Two small practices were yoked for randomization.) In the absence of SDP training, no changes to typical treatment were requested of TAU providers. Adolescent stimulant-treated patients from these practices completed electronic surveys at home at baseline prior to practices learning their randomized assignment, and again 6, 12, and 18 months post-training for both groups (Times 1–4; see Figure 1). Practices and patients were recruited in two cohorts in the summers of 2016 and 2017.

Figure 1.

Participant Flow Diagram

See Molina et al., 2021¹⁸ for detailed reasons for patient exclusion.

Practices

Practices were selected from a large pediatric practice-based research network hosted by the University of Pittsburgh. Those with the highest caseloads of 13–18 year-old patients stimulant-treated for ADHD were approached to participate; none declined. Across the three practices who received training, 38 prescribing providers and 59 additional staff attended.

Patients

Of 868 patients with ADHD identified from the medical records by staff at the seven practices, 357 (41%) were confirmed eligible and enrolled by research staff over the summers before each fall baseline survey launch. Inclusion criteria and reasons for exclusion (e.g., no longer taking stimulants) may be seen in Molina et al.¹⁸ Parents and patients provided informed consent that included our procedures for confidentiality, detailing privacy of survey responses from one another and from providers, bolstered by a NIDA-issued Certificate of Confidentiality. The study was approved by the University of Pittsburgh Human Research Protection Office.

Data collection

Electronic surveys were programmed in Qualtrics (Provo, UT). Data were collected over eight weeks for each of the four waves of assessment, beginning in October of 2016 and 2017 for Cohorts 1 and 2, respectively.

Intervention (SDP Training)

All providers and practice staff interfacing with patients (e.g., nurses, medical assistants, receptionists) were asked to attend a one-hour workshop (77% in person and 23% remote) on brief clinical practice strategies to reduce the likelihood of stimulant diversion by their patients. Workshop content was adapted for adolescence (including parent involvement) from our prior work.¹⁴ Workshops were conducted at each randomly assigned practice location and were led by the first author (BSGM) and clinical coordinator (HLK). A prescribing provider for all but three patients participated in training. Five-to-11 providers at each practice treated 1-to-17 patients consenting to survey completion, M(SD)=6.63 (4.16) patients/provider.

The workshop content included training in 1) brief patient and parent education and counseling (e.g., negative consequences of diversion, keeping treatment information private, how to respond to requests for pills), 2) monitoring of medication utilization and supply, and 3) assessment of patient risk. Providers were taught how to code office visits to allow billing for additional time with patients. The presentation included three 2–4 minute video demonstrations of provider:patient interactions (e.g., motivational enhancement style; empowering teens in front of parents). A supply of patient handouts was provided, with demonstration of its use, and posters of the handout material for exam rooms were provided to facilitate provider:patient discussion (a provider handout and resource binder were also supplied). Workshop discussions concluded with practice-specific conversations about where, how, and when materials could be used to address barriers to implementation. Ratings of provider satisfaction were collected in-person from attendees immediately after training. Three months post-training, patients and parents were briefly queried electronically about any exposure to handouts or posters about ADHD medication in their doctor’s office as a partial check on exposure to this element of the intervention.

Measures

Provider use of clinical practice strategies for diversion prevention.

Adolescents reported at each assessment whether, in the past six months, their provider used strategies included in the diversion prevention training, see Table 1 for items. Responses were 0=no, 1=once, 2=more than once; these were then re-coded negative/affirmative. The number of affirmative responses were summed for analysis. An exploratory factor analysis determined the presence of separate factors, with the number of affirmative responses across items loading on a given factor used for subsequent analyses (see Results).

Table 1.

Patient-reported provider clinical practice strategies: Exploratory factor analysis loadings and eigenvalues, patient endorsement, and mean number of items endorsed.

	Factor Loadings
	Diversion Discussion	Risky Behavior Discussion	Assessment of Mental Health Symptoms and Functioning	% of patients endorsing item affirmatively
1. Discuss whether you take your medication as prescribed. (18)	0.655	0.017	0.536	84%
2. Discuss being open and honest with him/her and maintaining ongoing communication. (17)	0.594	0.124	0.479	77%
3. Remind you that your medication is prescribed for you and only you. (4)	0.754	−0.057	0.184	67%
4. Discuss calling the office if you need a prescription adjustment (i.e. not changing how you take your medication without consulting your doctor). (16)	0.566	0.013	0.377	65%
5. Ask how many pills you still have at home. (19)	0.545	−0.108	0.350	58%
6. Explain his/her refill policy, if any (e.g., calling in ahead of time, how frequently refills may be obtained, replacement of lost prescriptions). (15)	0.497	0.159	0.349	57%
7. Discuss using up most of your remaining pills before filling a new prescription. (20)	0.731	−0.081	0.345	53%
8. Discuss safe storage of ADHD medication (e.g., secure, private, or locked location). (12)	0.843	0.029	−0.022	45%
9. Discuss that your medication may be dangerous for your friends to use. (14)	0.994	−0.039	−0.090	45%
10. Explain that sharing or selling ADHD medication can get you in trouble at school or with the law. (10)	0.951	0.005	−0.141	43%
11. Explain that if you sell or share your ADHD medication, you will not have enough when you need it most. (5)	0.901	−0.011	0.033	41%
12. Discuss using ADHD medication with alcohol or other drugs. (21)	0.633	0.341	0.178	41%
13. Explain that your medication may not help your friends as much as they think it will. (13)	0.979	0.019	−0.088	40%
14. Discuss keeping your ADHD diagnosis and treatment private. (11)	0.810	0.028	−0.022	38%
15. Explain that sharing or selling your ADHD medication can negatively affect your reputation. (9)	0.976	−0.004	−0.139	37%
16. Discuss what you might say if someone asked you for your pills. (7)	0.968	−0.080	0.056	36%
17. Explain that you are likely to be approached to sell or share your ADHD medication. (6)	0.980	−0.088	0.054	31%
18. Explain that if word gets out that you share your medication, people you don’t know or like will ask you for your medication. (8)	0.915	0.086	−0.063	26%
19. Discuss changing or choosing medications to reduce risk of abuse. (22)	0.485	0.376	−0.063	26%
20. Discuss increased monitoring of sharing and selling stimulant medication by schools. (24)	0.721	0.275	−0.229	21%
21. Set up a contract that states what will happen (e.g. stop your treatment) if you use your medication inappropriately. (23)	0.690	0.219	−0.286	19%
22. Ask if you smoke cigarettes or use tobacco products. (31)	0.027	0.985	−0.034	66%
23. Ask if you used alcohol. (30)	0.068	0.945	0.014	62%
24. Ask if you used marijuana or other drugs. (32)	−0.029	0.984	0.011	62%
25. Ask if you got in trouble at school or with the law. (34)	0.298	0.537	0.061	52%
26. Ask about your sexual activity and protection against STDs. (33)	0.079	0.848	−0.106	51%
27. Ask about your grades in school. (27)	−0.013	0.499	0.829	93%
28. Ask about your ADHD symptoms (e.g., attention level, over activity). (1)	0.319	−0.052	0.851	92%
29. Ask about any side effects of your prescription. (2)	0.363	−0.001	0.689	89%
30. Ask about your extracurricular activities (e.g. athletics, band). (29)	0.193	0.300	0.574	87%
31. Ask about other mental health issues such as mood or anxiety. (3)	0.283	0.142	0.534	84%
32. Ask about your sleep habits. (25)	−0.016	0.482	0.676	83%
Mean number of subscale items endorsed/total number of items (SD)	9.50/21 (6.56)	2.93/5 (2.05)	5.28/6 (1.28)
Factor eigenvalues (% variance explained)	17.913 (52.69)	2.837 (8.34)	3.896 (11.46)

Note. Original item ordering in the questionnaire is indicated by numbers in parentheses following each item. Response options were 0=no, 1=once, 2=more than once (affirmative responses were summed for analysis). For future use, the questionnaire instructional set was: “Think about the person who prescribes your ADHD medication. In the last 6 months, did he or she do the following?” We recommend revising “him/her” to “them,” “abuse” to “abuse or misuse,” “smoke cigarettes or use tobacco products” to “use products containing nicotine,” and “marijuana” to “marijuana/cannabis.”

Parenting pertinent to diversion prevention.

Two scales were used to measure adolescent reports about parenting behaviors indirectly targeted in our training due to providers being encouraged to include parents in diversion prevention discussions. Parental supervision and monitoring specific to ADHD medication was measured with four items preceded by the stem, “During the last 6 months, how much did your parents really know….” 1) if you took your ADHD medication; 2) how many pills you had left; 3) where your medication is kept; 4) if you shared, sold, loaned, or traded your ADHD medication. Response options were 1) Didn’t know to 5) Knew all of the time, α = .80. The mean response was analyzed. Item development was informed by Steinberg and colleagues’ Behavioral Supervision and Strictness subscale that assesses parental monitoring and knowledge of adolescent’s activities and whereabouts.¹⁹ Parent-adolescent communication about diversion was measured with nine items preceded by the stem “Within the last 6 months, my parents talked with me about…” Items covered strategies to lower likelihood of diversion (e.g., “Keeping my ADHD diagnosis and ADHD medication private,” and “Things I could say if someone asked me for my ADHD medication.”) and educate about negative consequences of diversion (e.g., “Ways I could get in trouble at school if I share or sell my ADHD medication,” and “Possible health problems that could occur if anyone takes ADHD medication that is not prwcribed for them.”). Response options were 0) Never to 3) Three or more times, α = .95. Items were based on Turrisi and colleagues’ measure of parent-teen communication about alcohol²⁰ and adapted to discussion of stimulant medication diversion. Like Turrisi, we analyzed binary responses (no/yes) using a count of positive endorsements greater than zero (range 0–9).

Provider satisfaction with training. Providers and staff who attended the SDP training answered seven questions about their satisfaction with the training they just received. Responses ranged from 1) Very low to 5) Very high.

Analytic Plan

We first conducted an exploratory factor analysis of the 34 items measuring patient-reported provider use of clinical practice strategies for diversion prevention. The analysis was conducted in Mplus v7.4 for binary data with an oblique rotation.

Generalized Estimating Equations (GEE) were used to analyze data for all available patients and to account for the correlated nature of the repeated measurements within each patient. Two types of GEE models were used for outcome variables: binomial logistic for limited count outcomes and normal with identity link for continuous (interval scale; mean score) outcomes. Preliminary GEE models conducted at baseline showed working correlations within providers to be nearly zero and inconsistent in sign across outcomes; thus, we did not model dependence within providers.

In addition to income (see Randomization below), baseline (Time 1) measures for each outcome were treated as covariates; Time 2–4 measures were treated as repeatedly measured dependent variables. This approach tested our hypothesis of increased use of diversion prevention strategies by primary care providers reported by patients in practices assigned to SDP vs. TAU (and also increased diversion prevention parenting). Training effects were captured through a contrast between practices in SDP vs. TAU, with all other orthogonal contrasts included to adjust for any remaining between-practice differences and capture residual dependency due to nesting within practices. Time was treated as a categorical within-subjects variable, and intervention group assignment-by-time interactions tested differences in the outcomes at each of the three post-assessment timepoints. Following the omnibus test of time*intervention group, contrasts were used to identify at which specific time points the group effect was significant.

We examined provider report of satisfaction with the training to allow discussion of dissemination feasibility. Descriptive statistics are provided separately for providers and staff.

Randomization

Randomization was conducted within matched pairs to balance SDP/TAU groups on practice demographics and structure of clinical care. Practice demographics included practice size (number of total patients at the practice, number of 13–18 year old patients with a diagnosis of ADHD and prescribed a stimulant medication) and insurance type (% Commercial, % Medicaid, % Uninsured). Structure of clinical care was number of Behavioral Health Provider FTEs, Medical Provider FTEs (Doctor of Medicine/Doctor of Osteopathic Medicine (MD/DO), and Physician’s Assistant/Nurse Practitioner FTEs (PA/NP). The three matched pairs were then provided to an independent methodologist (co-author DJB) who conducted the randomization using SAS v9.4 (random number generation). Practices (including providers), but not patients, were informed of randomization status after collection of baseline data. Across the provider and parenting variables and five demographic variables (age, sex, minoritized race or ethnicity, parent education, family income), only one group difference was observed at p<.05: family income was higher for TAU vs. SDP, p<.05. This variable was included in all analyses.

Results

Patient sample

Of 357 enrolled adolescents, 341 (M_age=14.9; SD=1.5) completed the baseline assessment and form the sample for analysis. Of these 341, 256 identified as male, 84 as female, and one as “other.” Highest parental education was bachelor’s degree or higher (64% of the sample) and household mean (and median) income was $75-$99K/year. Adolescent-identified race and ethnicity (10% Black, 6% Hispanic/Latinx, 4% more than one race, non-Hispanic/Latinx, 80% White, non-Hispanic/Latinx) mirrored the demographics of Allegheny county (population 1.2M), which includes the city of Pittsburgh, in which recruitment occurred. Retention was 92%−96% across the four waves of assessment.

Intervention exposure

After accounting for annual household income, the SDP group was 1.96 times more likely to report exposure, by patient or parent report, to handouts or posters about ADHD medication in their doctors’ offices. This finding suggested, at least for this aspect of the training, greater intervention exposure in the SDP (58%) versus TAU (37%) group, χ²(1), p<.05.

Exploratory factor analysis, provider use of clinical practice strategies for diversion prevention

Table 1 displays the full set of clinical practice strategy items rated by patients. The exploratory factor analysis showed that a 3-factor model provided the best fit to the data (best combination of acceptable fit indices and distinct factor loadings). This model accounted for 72.5% of the variance in the observed data, X²=724.48, df=462, p=.0000, RMSEA=.04, 90% CI=.035–.047, CFI=.989, TLI=.987, n=334. One subscale consisted of the items directly targeting diversion through provider discussion of diversion and medication utilization, 21 items, α=.94. Two subscales reflected relevant clinical information gathered by providers: assessment of risky behaviors such as substance use and sexual activity, 5 items, α=.89; assessment of mental health (MH) symptoms and functioning that was not expected to change appreciably as a function of training (e.g., ask about ADHD symptoms; ask about grades in school), 6 items, α=.74. Two items with low loadings (“Ask about your driving habits,” item 26, .387; “Ask about your friends,” item 28, .413) were removed before calculation of factor scores. Intercorrelations supported the distinctiveness amongst the factor scores, r = .49 (diversion discussion, risky behavior discussion); r = .37 (diversion discussion, MH symptoms and functioning); r = .33 (risky behavior discussion, MH symptoms and functioning).

Patient-reported provider strategy use

Table 1 shows the rates of endorsement at baseline for each of the individual patient-reported provider behaviors and the mean sum scores for each of the three clinical practice strategy subscales. There was a wide range of endorsements with diversion prevention behaviors occurring the least often, assessment of mental health symptoms and functioning occurring the most often, and risky behavior discussions in-between. For discussion of diversion, provider behaviors were reported by 19% (set up a contract) to 77% (discuss being open and honest) of patients. On average, patients endorsed less than half, 45.2% (M=9.50 out of 21; SD=6.56), of the diversion prevention behaviors by their physicians. Risky behavior assessments were endorsed by 51% (asked about sexual activity and STDs) to 66% (asked if smoke cigarettes or use tobacco products) of patients, with a mean of 58.6% (M=2.93 out of 5; SD=2.05) of items endorsed. Finally, assessment of MH symptoms and functioning was nearly always endorsed, with a mean of 88.0% (M=5.28 out of 6; SD=1.28) of the behaviors endorsed; as noted above, this variable was not expected to change as a function of training and was therefore not included in the GEE analyses.

Results of the GEE analyses examining group differences between SDP and TAU are shown in Table 2. Number of provider-discussed diversion prevention strategies differed between groups at Times 2, b(SE)=0.32(0.16), p<.05, and 4, b(SE)=0.45(0.18), p<.05. At both timepoints, SDP patients reported more diversion prevention strategies than TAU. Reflecting these group differences, there was a steady increase in the means across timepoints for the SDP group with 9.11 behaviors endorsed at baseline and 13.45 endorsed at Time 4 (an increase from an average of 43% to 64% provider behaviors endorsed versus 47% to 56% for TAU). No group differences were observed at Time 3. There were no group differences in patient-reported provider assessment of risky behavior.

Table 2.

Descriptive statistics by group and time and results of GEE analyses

	Time 1 (Baseline)		Time 2		Time 3		Time 4		Results of GEE Analyses
	TAUM (SD)	SDPM (SD)	TAUM (SD)	SDPM (SD)	TAUM (SD)	SDPM (SD)	TAUM (SD)	SDPM (SD)	Time 2 B(SE)	Time 3 B(SE)	Time 4 B(SE)
Provider use of clinical practice strategies for diversion prevention
Provider discussed diversion prevention	9.86(6.30)	9.11(6.83)	10.52(7.04)	11.18(7.31)	12.05(7.14)	12.40(7.05)	11.76(7.32)	13.45(7.06)	.32(.16) *	.18(.17)	.45(.18) *
Provider asked about risky behavior	2.99(2.05)	2.85(2.04)	3.17(2.06)	3.22(2.04)	3.34(1.99)	3.50(1.92)	3.35(2.06)	3.55(2.01)	.12(.22)	.06(.24)	.35(.26)
Parenting pertinent to diversion prevention
Parental supervision and monitoring specific to ADHD medication	4.39(.93)	4.39(.93)	4.48(.83)	4.39(.85)	4.36(.96)	4.28(1.04)	4.35(.85)	4.34(.96)	−.04(.09)	−.07(. 11)	−.06(.10)
Parent/adolescent communication about diversion	3.49(3.40)	3.45(3.51)	3.84(3.66)	3.99(3.67)	3.58(3.70)	4.53(3.74)	4.07(3.88)	4.23(3.80)	.15(.20)	.46(.22) *	−.01(.24)

Notes. Group n’s for TAU and SDP, respectively, ranged from 173–174 and 159–160 at Time 1, 162–164 and 144–149 at Time 2, 149–162 and 132–151 at Time 3, and 130–158 and 120–145 at Time 4. Significant results from GEE analyses are bolded.

^*p<.05.

Patient report of parenting pertinent to diversion prevention

The average response at baseline for patient-reported parental supervision and monitoring specific to ADHD medication was 4.39, sd=.93, n=333, where 4=”knew most of the time” and 5=”knew all of the time” on a 1–5 scale, providing little room for improvement. The average response at baseline for parent-adolescent communication about diversion was 3.47, sd=3.45, n=332, reflecting a low number of topics discussed at baseline (possible score range 0–9 topics).

As shown in Table 2, not surprisingly, there were no group differences in patient-reported parent supervision and monitoring of medication at any timepoint. For parent-adolescent communication about diversion, however, SDP patients reported more parent-adolescent communication about diversion compared to TAU patients, b(SE)=0.46(0.22), p<.05 at Time 3, with an average of one more topic discussed at Time 3 by those in the SDP group, 4.53, versus those in the TAU group, 3.58. Group differences were not observed at Times 2 and 4.

Provider satisfaction

All prescribing providers (38/38, 100%) and most of the non-prescribing staff (58/59, 98.3%) who completed the workshop in the SDP condition also completed satisfaction surveys immediately following training. As shown in Table 3, provider satisfaction with the SDP training was high with most indicating “High” or “Very high” satisfaction with the quality and relevance of the workshop. Only 23.7% endorsed a “High” or “Very high” need for more training to feel comfortable using the diversion prevention strategies. Similarly positive ratings were provided by the non-prescribing workshop participants, also shown in Table 3. Numbers of individuals in each professional category (e.g., physician, nurse practitioner, RN, BSN) may be seen in the Table 3 notes.

Table 3.

Prescribing providers and other staff satisfaction at the conclusion of the workshop

	1 = Very Low	2 = Low	3 = Moderate	4 = High	5 = Very High	Mean	SD
Prescribing providers (n = 38)
1. Satisfied with overall quality of educational activity	0%	0%	5.3%	39.5%	55.3%	4.50	0.60
2. Content of program was relevant to practice	0%	0%	2.6%	18.4%	78.9%	4.76	0.49
3. Program enhanced your knowledge	0%	2.6%	7.9%	36.8%	52.6%	4.39	0.75
4. As a result of participation, feel equipped to implement the strategies	0%	0%	7.9%	52.6%	39.5%	4.32	0.62
5. Time allotted for training was sufficient	0%	2.6%	10.5%	47.4%	39.5%	4.24	0.75
6. Feel that you need additional training to feel comfortable using the strategies	7.9%	31.6%	36.8%	15.8%	7.9%	2.84	1.05
7. Content of training was useful to you	0%	0%	10.5%	26.3%	63.2%	4.53	0.69
All others (n = 58)
1. Satisfied with overall quality of educational activity	0%	0%	8.6%	53.4%	37.9%	4.29	0.62
2. Content of program was relevant to practice	0%	0%	10.3%	39.7%	50.0%	4.40	0.67
3. Program enhanced your knowledge	0%	0%	17.2%	46.6%	36.2%	4.19	0.71
4. As a result of participation, feel equipped to implement the strategies	0%	1.7%	18.9%	48.2%	31.0%	4.09	0.76
5. Time allotted for training was sufficient	0%	0%	17.2%	58.6%	24.1%	4.07	0.65
6. Feel that you need additional training to feel comfortable using the strategies	13.8%	49.7%	25.9%	15.5%	5.2%	2.59	1.08
7. Content of training was useful to you	0%	1.7%	17.2%	41.4%	39.7%	4.19	0.78

Note: Prescribing providers included M.D. or D.O. (27/38, 66%) with the remainder being nurse practitioners and one physician assistant. 25/38 attended the workshop in-person and 13/38 completed the workshop online due to schedule incompatibility and received their satisfaction survey electronically. Among the remaining office staff, 43/58 held MSN, BSN, LPN, LPC, RN, BA/BS, or MA/CMA/RMA degrees; the remaining were other office staff (e.g., scheduling; office manager).

Discussion

This study describes results from the first RCT of primary care provider training in stimulant diversion prevention. Our findings demonstrated change in adolescent patients’ experiences with their providers’ up to a year and a half post-training. The findings are a critical extension of our earlier report of physician behavior change based only on providers’ self-reports.¹³ We also found, secondarily, some change in a variable indirectly targeted by our training: parenting behavior reported by adolescents, with an increase in parent-adolescent discussion about stimulant diversion when stimulant diversion training was provided to their doctor and office staff. Finally, provider and office staff satisfaction with the training, rated immediately after completion of the workshop, was very good, providing some confidence that the training resulting in physician behavior change has potential for provider uptake if disseminated.

Our primary findings, of physician behavior change reported by patients, coupled with McGuier et al¹³ (physician self-report), provide strong support for use of a brief training in stimulant diversion prevention in the pediatric setting. Analysis of blinded ratings of physician behavior would have provided the most rigorous test of intervention impact, but that would have required recording all physician interactions (for later coding) which was not practical in this study. The findings are also strengthened by the knowledge that patient reports were unlikely to be driven by awareness of study condition. We did not inform families which practices were randomized to SDP versus TAU.

Although statistically significant, the group differences in physician behavior were modest in magnitude as reflected by the average percentages of diversion prevention strategies endorsed by the patients for each group (e.g., 56% versus 64% for TAU and SDP, respectively, at Time 4). There was also a puzzling failure to find group differences at Time 3 which appears to be due to an increase (more than for SDP) in use of diversion prevention strategies in the TAU group. All physicians and their staff knew the purpose of the study which may have contributed to heightened awareness of the need to talk with patients about stimulant diversion (and misuse). The topic was also increasing in the media at the time this work was undertaken which no doubt influenced provider behavior. The magnitude and longevity of training effects that we found, relative to the brevity of the training, seems like a reasonable trade-off for practices concerned with addressing stimulant misuse. This is especially so given the many other competing demands on pediatrician time, and that providers are encouraged by the American Academy of Pediatrics to provide anticipatory guidance to young patients to reduce their risk of diversion and misuse.²¹ Significantly, physician behavior differences as a function of group assignment were noticed by patients which, to our knowledge, is unprecedented in the literature on primary care provider training.

We found intervention effects at one of the three follow-up timepoints on an indirectly targeted patient-reported parenting variable: parent-teen communication regarding stimulant diversion. We studied this effect because providers were encouraged, and shown how, to include parents in diversion prevention discussions. Specifically, adolescents reported on average one more diversion topic discussed with their parents at the one-year follow-up (Time 3) if they were in the SDP vs. TAU group. These conversations reflected topics such as keeping ADHD diagnosis and treatment private, ways that the adolescent could get in trouble from sharing or selling, and that diverting could result in not having medication when it is needed. This difference was not observed at Times 2 and 4 when effects on providers were observed. We speculate that group differences in parenting may reflect downstream effects of provider behavior change. Specifically, increased provider discussions about stimulant diversion at Time 2, which were encouraged to include parents, may have led to more parent-teen conversations at home (which would have been reflected at Time 3). Given the importance of parent involvement in adolescent healthcare, and the potential for extending the effects of provider training beyond the boundaries of the pediatrician’s office, these are hopeful findings. We observed no treatment group differences in supervision and monitoring of ADHD medication which was already high at baseline and stayed high.

Provider ratings of satisfaction collected immediately after training were generally high. Moreover, only a minority of providers and staff felt that they needed additional training to feel comfortable using the diversion prevention strategies. These findings may explain providers’ steady uptake of prevention diversion strategies across the waves of surveys. We took care to solicit and make use of provider feedback in the course of developing our workshop. This was accomplished through initial open discussions followed by review and critique of our materials by providers not participating in the study. The interactive educational approach that we used in the workshop, known to be among the most efficacious for creating provider behavior change,²² was developed with the benefit of provider consultation which is known to improve implementation.²³ As with a long-lasting school-wide behavioral program that we implemented in a public middle school, that also involved teacher training,²⁴ we are confident that including such representation in the developmental stage made a difference in the extent to which the workshop was well-received and effected provider behavior change. In addition to “involving physicians from the beginning,” we made use of recommended practices for quality improvement initiatives such as “make the right thing easy to do,” and “reduce hassles and wasted time.”²⁵ For example, we provided the billing code and scripted justification verbiage for use when extra time with the patient was required. We did this in response to providers expressing concern during workshop development about compensation for extra time spent with patients and in documentation. We also incorporated baseline data in the training materials which may have boosted motivation by ensuring relevance of the information (e.g., percent of patients across practices reporting use of their medication seven days/week). Finally, the on-site workshop delivery with provision of meals and continuing medical education no doubt enhanced participation. Although some components may be difficult to implement in larger-scale dissemination (e.g., provision of binders), the brevity of the one-hour training is not. Moreover, as promised at the outset of the study, we provided training to TAU practices after data collection was complete, but due to the onset of the COVID-19 pandemic at that time, all of this training was conducted virtually. In separate descriptive analyses available upon request, all providers rated their satisfaction with this virtual training as high or very high, suggesting that virtual delivery may be a dissemination option.

A limitation of this study, beyond those already stated, is our sampling of patients from pediatric practices embedded within one large, regional practice-based research network. In addition, our sample was well-educated and only 20% of the patients self-identified with minoritized racial and ethnic identities. Although this proportion reflects the demographics of the geographic region in which the study was conducted, it may limit generalizability to more diverse populations. The patients in our trial were also young (mean age 15) because we were targeting prevention, but older age is associated with higher scores on variables pertinent to stimulant diversion and misuse.¹⁸ Thus, results may have been different with an older sample. Although we took steps to protect patient privacy during questionnaire completion, we cannot be sure that all surveys were completed free of parental oversight. Finally, we are unable to directly compare adolescent and provider report. Providers reported about their care for all of their patients with ADHD¹³ while the adolescents reported about their provider’s behavior treating them; thus, we would not expect a strong correlation between these data.

Overall, the current study demonstrated that pediatric primary care provider training in stimulant diversion prevention resulted in provider behavior change based on confidential patient report. This novel finding provides confidence in the utility of a brief workshop to aide pediatricians and their staff with clinical practice strategies designed to prevent diversion of stimulant medication prescribed for ADHD. We are currently conducting a longitudinal follow-up study of the patients enrolled in this trial, in the age range when stimulant misuse and diversion are at their peak prevalence: early adulthood. It will be important to determine whether continued use of these clinical practice strategies by pediatric providers, as noticed by their patients, leads to reduced stimulant diversion at older ages.

Abbreviations:

ADHD

Attention-Deficit/Hyperactivity Disorder

Diversion

Sharing, selling, or trading stimulant medication

SDP

Stimulant Diversion Prevention

TAU

Treatment as Usual

RCT

Randomized control trial

MTF

Monitoring the Future Study

GEE

Generalized Estimating Equations

MD

Doctor of Medicine

DO

Doctor of Osteopathic Medicine

PA

Physician Assistant

NP

Nurse practitioner

CME

Continuing Medical Education

Data Sharing Statement:

Deidentified individual participant data supporting this manuscript will be made available to researchers providing a methodologically sound proposal, after this manuscript has been published, and upon approval of a Data Sharing Agreement with the University of Pittsburgh.