Adolescent outcomes of Post-operative opioid EXposure (APEX): Protocol for a Multisite Observational Study

Abstract

Introduction:

Adolescents accounted for 5% of fatal overdoses in the United States in 2023 (Centers for Disease Control and Prevention, 2024), yet existing research on the risk factors for non-medical opioid use (not prescribed, more frequent, or in a greater quantity than intended) among adolescent patients remains limited. Surgical procedures serve as a first exposure to opioids for many adolescents and may lead to an increased likelihood of non-medical opioid use or future opioid use disorder (OUD). Although the prevalence of OUD in adolescents ages 12–17 is low—1.2% (Center for Behavioral Health Statistics, 2024)-- identifying risk of opioid use disorder (ROUD) in adolescence could present an opportunity to intervene and support vulnerable individuals. Preventive interventions for adolescents undergoing surgical procedures could interrupt the trajectory from non-medical use to OUD, minimizing negative health effects and reducing the risk for fatal overdose.

Methods:

Using a biopsychosocial model of the development of opioid use disorder, the Adolescent outcomes of Post-operative opioid EXposure (APEX) study gathers medical record and survey data from adolescent-parent dyads to better understand the relationship between pain, pain management, prescription opioid use and recovery from surgical procedures.

Results:

In this paper, we describe objectives and design of the APEX study, which aims to develop and validate systematic tools that are designed to detect ROUD in adolescent surgical patients in the 12 months following surgery.

Conclusions:

Results of this study could provide insight into predictors of ROUD, which may inform prescribing practices in the future, help update protocols for managing pain in young people and guide future research efforts.

1. Introduction

1.1. Background

Opioid prescribing for surgical pain management drives fatal overdose from opioids and, increasingly, stimulants (Hayden et al., 2025), in the United States. Although the number of fatal overdoses from prescription opioids is decreasing nationally (Center for Disease Control and Prevention, 2022), population-level data suggests that regions with higher rates of opioid prescribing also have higher rates of opioid-related deaths (Olfson et al., 2023). Furthermore, prescription opioid use may ultimately result in non-medical opioid use (use of opioids that is greater than prescribed, more frequently than prescribed, or not prescribed), which in turn increases risk for opioid use disorder (OUD) (Han et al., 2024) and overdose from fentanyl and other opioids. Adolescents accounted for 5% of fatal overdoses in the United States in 2023 (Centers for Disease Control and Prevention, 2024).

More than 4% of US adolescents aged 12–17 undergo surgical procedures every year (Rabbitts & Groenewald, 2020), representing the first exposure to opioids for many and a potential driver of future non-medical opioid use (Lynskey et al., 2003). In 2019, 3.5% of US children and young adults aged 0–21 received at least one opioid prescription, half of whom were high-risk for non-medical opioid use (Chua et al., 2021). While the percentage of adolescent surgical patients or caregivers who filled and refilled opioid prescriptions declined between 2015 and 2020, recent data suggests that pharmacies dispense 1 in 6 prescriptions before surgery, a practice that is strongly associated with opioid-related adverse outcomes (Sutherland et al., 2024). Opioid-related adverse outcomes are defined as development of OUD and/or opioid overdose. One in 33 patients in the study filled a prescription 91–180 days after surgery, representing persistent opioid use (prescription opioid use 90+ days after surgery) and potential non-medical use (Sutherland et al., 2024). Among pediatric patients, other studies estimate the prevalence of persistent opioid use to be as high as 5% (Harbaugh et al., 2018).

Adolescents are developmentally vulnerable to substance use disorder (Levy, 2019), and adolescents who use prescription opioids have a greater risk of future OUD (Auger et al., 2021; Kelley-Quon et al., 2019; McCabe et al., 2017; Miech et al., 2015; Schroeder et al., 2019). A study using nationally representative data found that one in 315 adolescents who receive an opioid prescription experience an opioid-related adverse outcome within the following year (Hadland et al., 2021). There is an inverse correlation between the age of onset of non-medical use and the risk of developing OUD (McCabe et al., 2007); adolescents with a history of non-medical use are thirteen times more likely to initiate heroin use compared to their peers (Compton et al., 2016). Prescribing practices are associated with persistent opioid use (Vargas et al., 2023). For example, the duration of opioid prescription following a surgical procedure is a strong predictor of subsequent non-medical opioid use, with each additional week increasing the risk by 44% (Brat et al., 2018). Additionally, among youth undergoing anterior cruciate ligament repair, receiving an opioid re-prescription within six weeks post-surgery led to 160% increased odds of developing OUD within one year (Cahan et al., 2023).

Several factors can predict the transition from medical to non-medical opioid use, defined below as risk for opioid use disorder (ROUD; see section 2.4 Outcome Measures for operational definition). Previous research identified prolonged pain, particularly in adolescents, as a significant risk factor. Results from the National Longitudinal Study of Adolescent to Adult Health associate chronic pain, including post-surgical pain, with a 24% increased odds of non-medical opioid use among youth who are prescribed opioids (Groenewald et al., 2019), and a systematic review on the topic highlighted prolonged pain as a risk factor for OUD among youth prescribed opioids (Dash et al., 2018). Among surgical patients, research identifies post-surgical trauma, particularly post-traumatic stress disorder (PTSD), as a predictor of OUD even after controlling for other mental health and substance use disorders (Hassan et al., 2017). In addition, analyses of Youth Risk Behavior Surveillance System (YRBSS) data highlight the role of non-opioid substance use in increasing the risk of OUD (Barnett et al., 2019; Khalid et al., 2023).

To date, the research evaluating risk factors for OUD among patients prescribed opioids post-operatively has depended primarily on outcomes drawn from large administrative data sets. This approach relies on physician diagnoses and coding for billing purposes, which may miss incomplete OUD-related information. This study will additionally collect adolescent and parent survey data specifically designed to help identify ROUD. Furthermore, survey data collected for this study, particularly reports of drug and alcohol use and social determinants of health, will provide additional predictive power that the EMRs alone may underestimate (Iott et al., 2024; McCrory & Mayes, 2015; Rowe et al., 2021). A recent behavioral health study found that self-report data has the strongest predictive power in machine learning models, and that supplementing self-report information with data from the EMR can enhance predictive power (Nock et al., 2022).

1.2. Current Study

The Adolescent outcomes of Post-operative opioid EXposure (APEX) study, which involves a longitudinal cohort of adolescent surgical patients to evaluate the impact of opioid use on opioid-related adverse outcomes, addresses the limitations of previous work. The study uses a biopsychosocial framework that aims to integrate neurobiological and psychodynamic perspectives to enhance understanding of the development of opioid use disorders (Alvarez-Monjaras et al., 2019; McCrory & Mayes, 2015). The study design seeks to develop and validate systematic tools for predicting ROUD-- defined above as prolonged pain, prolonged opioid use, post-surgical trauma, increasing trajectory of other substance use, and non-medical opioid use-- and the trajectory of prescribed and non-medical opioid use in adolescents in the 12 months following surgery with post-operative opioid analgesia (primary objective). See section 2.4 (Outcome Measures) for the operational definition of ROUD. These tools will be useful for tailoring follow-up of higher risk adolescents rather than peri-surgical decision-making because they focus primarily on prediction and not causal inference of risk factors.

APEX has three aims: (1) to determine the frequency of ROUD, (2) to use machine learning models to develop and validate screening algorithms to detect “early warning signs” (i.e. ROUD), and (3) to develop and validate a prediction model designed to predict a clinical trajectory of post-surgical opioid use and pain. Exploratory objectives include validating the Pediatric Pain Screening Tool (PPST), evaluating the associations between parent-reported receipt of education about opioid disposal, parental substance use and the development of ROUD, and adolescent report of opioid diversion.

The APEX study is novel for its prospective design and large proposed sample size. While other studies have contributed to an understanding of the etiology and prevalence of OUD in adolescents receiving prescription opioids (Bell et al., 2019; Hadland et al., 2021), opioid prescribing patterns for adolescent surgical patients (Harbaugh et al., 2018; Sutherland et al., 2024; Vargas et al., 2023), and trajectories of post-surgical opioid use in surgical patients (Odegard et al., 2024; Wilson et al., 2021), to our knowledge, this is the first study to prospectively measure opioid consumption and risk for OUD in a post-surgical adolescent population. The proposed sample size across five diverse sites powers this study to identify important relationships and factors among domains of interest among sub-groups. With a large sample size, this study is novel in its ability to analyze outcomes of interest by demographics, patterns of substance or medication use, geographic area, and/or surgical procedure.

2. Methods

2.1. Study Sample

APEX is a longitudinal observational study. Up to 10,000 dyads comprised of an adolescent surgical patient (12–17 years old, inclusive) and a legal guardian (henceforth, parent) complete an assessment battery prior to a scheduled surgical procedure, and 4-weeks, 3-, 6- and 12 months post-operatively. Additionally, adolescent participants complete pain and opioid use diaries for the first 10 post-operative days (see supplemental material for contents of days 1–10 diaries).

Adolescents undergoing scheduled surgeries with at least a 90% likelihood of requiring opioid analgesia for at least 48 hours post-operatively are eligible. Sites reviewed and selected eligible surgeries based on historical opioid prescription EMR data from their institution. Surgical procedures were considered eligible if 90% or greater of past-year cases received either a 2+ day opioid prescription at discharge or 2+ day post-operative admission with opioids. occurred in Sites refine surgery lists iteratively during recruitment to reflect current institutional prescribing practices. The study set a 90% threshold for opioid analgesia to ensure that all adolescents had clinically meaningful exposure to opioids as part of standard care for their procedure.

Study procedures exclude adolescents with medical disorders that require chronic opioid management, who cannot comprehend English at a 4^th-grade level, or who have ever been diagnosed with OUD or have taken medication for OUD. Also excluded are adolescents who are currently in the custody of the state or incarcerated, pregnant or parenting, medically complex with a diagnosis of a degenerative or significantly impairing medical condition, or currently enrolled in a pain management intervention study. Criteria also exclude parents who cannot comprehend English or Spanish at a 4^th grade level or who are currently incarcerated. Adolescents cannot participate in the study without an eligible legal guardian and vice versa.

2.2. Recruitment

Study staff review operating room schedules up to one month in advance to identify age-eligible adolescents scheduled to undergo an eligible procedure based on site opioid prescribing practices. Research assistants conduct a preliminary medical record review to identify exclusion criteria.

Research assistants contact potential participants via a combination of remote (i.e., hospital portal message, letter, email, and/or phone call) and in-person contacts on the day of surgery and during pre-operative appointments. Interested parents and adolescents complete an eligibility survey, assent and consent procedures and the baseline survey prior to surgery. APEX is approved by the single IRB at Boston Children’s Hospital.

2.3. Data Sources

Data sources include adolescent and parent surveys and data extracted from the EMR. Participants complete surveys via an electronic data capture system (OpenClinica) (OpenClinica » Your Source for Clinical Trial Technology, n.d.). Baseline survey data provided by adolescent and parent participants includes demographics, mental health, pain, medication use, substance use, and wellbeing. Follow up surveys include measures of pain, substance use, wellbeing, support, and engagement in activities (Table 1). Data extracted from the EMR includes medical history, procedure information, medications administered, perioperative care, and surgical recovery (Table 2).

Table 1.

Domains and measures of APEX self-reported adolescent and parent surveys

Survey Domain	Description & Measures	Timepoint
Demographics	Adolescents: NIDA CTN/NIH HEAL Demographic Questionnaire(Wandner et al., 2022) Age, sex, ethnicity, race, and grade level	Baseline, 12M
	Parents: NIDA CTN/NIH HEAL Demographic Questionnaire(Wandner et al., 2022) (self) Age, sex, ethnicity, race, education, income, employment, household composition, marriage status, and preferred language	Baseline, 12M
General Wellbeing	Adolescents: Medical Outcomes Health Survey (MOS)(Ware & Gandek, 1998) Overall health at the time of survey School and Social Activity (novel) Adolescent’s involvement in school and social activities, past 30 days	Baseline, 6M, 12M
	Parents: School and Social Activity (novel-proxy) Adolescent’s involvement in school and social activities, past 30 days	Baseline, 12M
Pain	Adolescents: Brief Pain Catastrophizing Scale (BPCS) (Sullivan et al., 1995; Walton et al., 2020) Pain catastrophizing Brief Pain Inventory (BPI) (Atkinson et al., 2011) Average, current, least, and greatest pain intensity, past 7 days and 24 hours PROMIS® Pediatric Pain Interference 8a (Varni et al., 2010) Pain’s impact on life and physical functioning, past 7 days Pediatric Pain Screening Tool (PPST)(Simons et al., 2015) Pain’s impact on daily activities and quality of life, past two weeks	All Timepoints
	Parents: Brief Pain Catastrophizing Scale (BPCS) (Sullivan et al., 1995; Walton et al., 2020)(self) Parent’s catastrophizing of child’s pain PROMIS® Parent Proxy Pain Interference 8a (Irwin et al., 2012)(proxy) Impact of pain on their child’s life and physical functioning, past 7 days (also asked at 4W, 3M, 6M) Medication management and household storage (novel, self) Medication in the home and their storage Overdose Questionnaire (National Institute of Drug Abuse Clinical Trials Network, 2022b) (adapted-proxy) Child’s opioid overdose, past year Pre-operative medications (novel- proxy) Child’s medication consumption, past 7 days Education about opioid disposal (novel- self) Whether parents received information about opioid disposal/storage Thiels Opioid Prescribing Survey(Robinson et al., 2022) (adapted- proxy) Child’s opioid consumption (also asked at 4W, 3M, 6M)	Baseline, 12M
Substance Use	Adolescents: Pain medication use (novel) Frequency of opioid and non-opioid pain medication use, past 2 weeks, 2,3 months DSM-5 Opioid Use Disorder Checklist (National Institute of Drug Abuse Clinical Trials Network, 2022a) Opioid use disorder criteria, past 12 months National Survey of Drug Use and Health PRYWAYS (Substance Abuse and Mental Health Services, 2021) Non-medical use of prescription opioids, past 3 months Opioid side effects (novel) Side effects from prescription opioids, past 2 weeks Screening to Brief Intervention (S2BI) (Levy, Weitzman, et al., 2021) Substance use, past 6 months and past year National Epidemiologic Survey on Alcohol and Related Conditions (NESARC) (Grant et al., 2003) (adapted) Lifetime substance use Brief Frequency measure (Brief Freq)(Levy, Wisk, et al., 2021) (adapted) Frequency of substance use, past 3 months National Survey of Drug Use and Health(Substance Abuse and Mental Health Services, 2021) Age of first use NIAAA binge question (Donovan, 2009) Binge alcohol use Opioid prescription diversion (Canan et al., 2021) Whether adolescents have given away or sold their prescription opioids, lifetime Monitoring the Future (Johnston et al., 2022) High intensity drinking, past 3 months Pill use questions (novel) Non-medical use of prescription or over-the-counter medication, past 3 months ACLSPAIN (novel) Opioid and/or non-opioid medication use, past 24 hours Thiels Opioid Prescribing Survey (Robinson et al., 2022) (adapted) Opioid consumption, past 10 days Drug Effects Questionnaire (DEQ-5) (Robinson et al., 2022) Opioid-induced euphoria, past 24 hours	All Timepoints
	Parent: The Tobacco, Alcohol, Prescription medication, and other Substance use (TAPS-1) (Gryczynski et al., 2017) (self) Substance use, past year Single Item Screen-Cannabis (SIS-C) (Matson et al., 2022) (self) Cannabis use, past year Family history of substance use disorder (novel-proxy) Family history of substance use disorder	Baseline, 12M
Psychosocial/Mental Health	Adolescents: PROMIS® Pediatric Meaning and Purpose 4a (Ravens-Sieberer et al., 2014) Resilience and sense of purpose, present Patient Health Questionnaire-2 (PHQ-2) (Arroll et al., 2010) Symptoms of depression, past two weeks Generalized Anxiety Disorder-2 (GAD-2) (Skapinakis, 2007) Symptoms of anxiety, past two weeks DSM-5-TR Self-Rated Level 1 Cross-Cutting Symptom Measure Q3 (American Psychiatric Association, 2013) Sleep disturbance, past two weeks PROMIS® Pediatric Fatigue 10a (Lai et al., 2013) Fatigue, past 7 days Child PTSD Symptom Scale (Foa et al., 2018) (adapted) Post-surgical traumatic stress, past month	All Timepoints
	Parents: Patient Health Questionnaire-2 (PHQ-2) (Arroll et al., 2010) (self) Symptoms of depression, past two weeks Generalized Anxiety Disorder-2 (GAD-2) (Skapinakis, 2007) (self) Symptoms of anxiety, past two weeks	Baseline, 3M, 6M, 12M
Support and Engagement	Adolescents: PROMIS® Connection to Family Members 4a (Bevans et al., 2017) Connectedness to family, past four weeks Social Support Scale Q2 (adapted) (Center for HIV Identification Prevention and Treatment Services, 2012) Material support, present PROMIS® Emotional Support Item Bank V.2 (Hahn et al., 2014) Emotional support, present PROMIS® Pediatric Peer Relationships 8a (DeWalt et al., 2013) Social acceptance and support, past 7 days Satisfaction with Care Measure (novel) Satisfaction of care received and preparedness for surgery and recovery	Baseline, 6M, 12M
	Parents: PROMIS® Connection to Family Members 4a (Bevans et al., 2017) (self) Connectedness to family, self	Baseline, 12M

Table 2.

Domains of data extracted from adolescent electronic medical record

EMR Domain	Description	Timepoint
Surgery	Encounter ID, demographics (date of birth, sex, height, weight), surgery start & end time, post-anesthesia care unit start & end time, stay days, American Society of Anesthesiologists Classification, blood pressure, estimated blood loss, transfusion, insurance carrier, anesthesia technique	All EMR domains collected annually
Diagnosis	All ICD diagnosis codes available for patient: Encounter ID, diagnosis date, ICD version, diagnosis code
Procedures	Start date, procedure code, encounter ID
Opioid and Non-Opioid Medications	Date, type, name Opioid Only: Route, Quantity, Dose (Value, Units)
Derived Elements	Perioperative codes, Elixhauser comorbidity, Agency for Healthcare Research and Quality complications, CPT-surgery, anesthesia (primary, measures), healthcare facility

2.4. Outcome Measures

The development of OUD within 12 months of a surgical procedure in adolescent patients is a relatively rare event, occurring in approximately 0.7% of patients (Cahan et al., 2023), though the risk may continue to grow beyond the 12-month period (Namiranian, 2023). Thus, adolescents who eventually develop OUD may not meet the criteria for diagnosis during the study period. In this study we use risk factors that commonly occur in the post-operative period and consistently lead to the development of OUD as “early warning signs” that signal the need for close monitoring to enable early clinical intervention. For this study, the primary outcome is a novel composite measure that includes 5 constructs that are consistently associated with the later development of OUD, which we have labelled, “Risks for OUD” or ROUD. The study design considers any participant that meets one or more of these criteria as positive for the primary outcome. Components of ROUD are listed below.

• Prolonged pain: average pain score of >3/10 in the last 7 days on the Brief Pain Inventory (BPI) at 3 or more months postoperatively reported by an adolescent (Dash et al., 2018; Groenewald et al., 2019).

• Prolonged opioid use: any adolescent or parent report of opioid medication use 3 or more months postoperatively (Brat et al., 2018).

• Postoperative non-medical use of prescription opioids: any adolescent or parent report of opioid medication use that is greater than prescribed, more frequently than prescribed, or not prescribed to the adolescent at any point within the 12-month observation window (Cahan et al., 2023; McCabe et al., 2007).

• Increasing rate of change in substance use: adolescent self-report of rate of increase of substance use frequency greater than the 75^th percentile of an age- and sex-matched cohort compared to nationally representative data at any point within the 12-month observation window (Barnett et al., 2019; Khalid et al., 2023).

• Post-surgical trauma: a score of 11 or greater on the Child PTSD Symptom Scale (CPSS) 3 or more months postoperatively (Bilevicius et al., 2018; Hassan et al., 2017).

2.5. Study Sites

Five large academic pediatric surgical centers (Boston Children’s Hospital, Cincinnati Children’s Hospital Medical Center, Children’s Hospital Colorado, Children’s Hospital of Philadelphia, and Johns Hopkins All Children’s Hospital) are participating in APEX to achieve a robust sample size and representation of demographic groups in the overall sample. All sites are members of the Society for Pediatric Anesthesiology Improvement Network (SPAIN), allowing this project to build upon existing research collaboration. SPAIN is a multi-institutional collaborative developed to systematically document outcomes in children undergoing surgical procedures where pain management approaches are variable. Leaders across the country formed SPAIN as a data sharing group to connect clinical investigators at 32 major pediatric surgical centers, and participating programs have experience in conducting clinical research. Participating sites complete at least 500 surgeries annually with adolescents in the target age group.

This study followed the National Institute on Drug Abuse Clinical Trials Network (NIDA-CTN) guidelines for site selection, which required a site selection plan, a site selection survey administered to interested sites, site selection interviews, and site nomination. Eleven children’s hospitals within the SPAIN network submitted the site selection survey, providing information about the demographics of the patient population, research infrastructure, site principal investigator qualifications, volume of potentially eligible surgeries, and past participation in SPAIN research projects. Five sites were nominated and three alternate sites were identified following this process. All selected sites reported at least 1500 eligible surgeries per year; alternate sites had lower surgical volume than selected sites.

3.0. Statistical Analysis Plan

3.1. Data Extraction Plan

Electronic medical records will be abstracted at each site using a standardized data dictionary by a team of data scientists and clinicians. Experts at each institution will work with the coordinating site to identify structured fields for extraction from EMR, perioperative records, nursing records, brief operative notes, and anesthesia logs. Staff will review a small number of charts to ensure concordance with EMR data abstracts. Yearly data extractions will include all previously consented patients at each site. Once the cumulative data extract is uploaded to the server, the previous version will be deleted.

The study data dictionary leverages standard code ontologies as possible to reduce potential for extraction errors. The central coordinating group developed a three-step quality assurance process based on successful previous efforts (Hong et al., 2025; Klann et al., 2021). First, data from all sites is uploaded to a central server. The research team analyzes each dataset to identify any outliers within the site dataset or errors in coding and variable creation. This process often identifies column naming errors, variable naming errors, or incorrect reference quantities. The central coordinating group reports errors back to the site for revision.

The second step includes a manual evaluation of 20 randomly selected charts with further probing for any errors identified during this initial chart review. Specifically, a member of the central coordinating group works with a research assistant to review random elements from every category of data for an encounter. Errors identified during this process will be probed by identification of enrolled patients with similar data elements to uncover any potential coding errors or omissions. Again, the error reports are sent back to the site to address.

Finally, after all sites have submitted a revised dataset, the central coordinating group performs cross-site outlier analysis to identify any potential subtle errors in unit values. Errors are addressed by the individual sites to produce a final data submission. This quality control and assurance process will be repeated three times after yearly data uploads of the cumulative dataset.

3.2. Data Analysis Plan

Aim 1:

Using the definition of ROUD above, we will measure overall rates of ROUD at each follow-up wave (3-, 6-, and 12-months post-surgery) as well as rates of any ROUD across all follow-up waves. We will also report rates for relevant demographic subgroups.

Aim 2:

Our primary goal is to develop a clinical prediction model for ROUD. Several major domains, including demographics (age, sex, race, ethnicity, socioeconomic status), well-being (self-rated health, school and social activity), mental health (depression, anxiety, attention deficit hyperactivity disorder), pain catastrophizing, substance use, resilience, depression, anxiety, sleep, fatigue, family history, recovery course, satisfaction with care, surgical procedure and length, blood loss, anesthesia, post-operative pain and analgesia will be included in the prediction. Additionally, to account for relevant differences across sites, the prediction model will include hospital-level factors that capture the types and volume of surgeries performed at the site, as well as historical prescribing behavior of surgeons. All predictors will be assessed at baseline. Our primary outcome will be any ROUD assessed at 3-, 6-, or 12-months post-surgery, although we may explore models predicting ROUD at specific follow-up waves.

We will use an ensemble technique to build the prediction model for ROUD. The ensemble technique, known as Super Learner, finds the best combination of many component prediction models via cross-validation. We will include logistic regression, ridge regression, ordinary least squares, random forests, XGBoost, classification and regression trees, and neural networks, as component prediction models. Theoretical results suggest that the Super Learner ensemble will perform as well asymptotically as the best component model. This ensemble technique can capture complex relationships, like unspecified nonlinearity and interactions between predictors, that are difficult to capture using traditional regression models. However, if traditional models like logistic regression fit the data well, the ensemble can similarly take advantage of their good properties, e.g., low variability.

We will evaluate model performance using the area under the receiver operating characteristic curve (AUC), a common metric for assessing binary classification performance in machine learning. Most binary prediction models, including relatively simple ones like logistic regression, produce a probability or risk score for each individual. Researchers then select a threshold above/below which people are classified as 1/0. The sensitivity and specificity of a given model is specific to that threshold. AUC is a common measure of predictive performance that accounts for the sensitivity and specificity across all possible thresholds and can provide a more holistic metric of model performance. An AUC of 1.0 indicates perfect predictive performance whereas an AUC of 0.5 indicates completely random assignment. An AUC of 0.8, for example, would mean that 80% of model-assigned scores in the population with ROUD would be higher than a randomly chosen score from the population without ROUD. We will use 80% of the data to build the prediction model and 20% to calculate the AUC. Holding out 20% of the data allows fair evaluation of any prediction model built on the remaining 80%, even if the team uses complex methods to build and select the best algorithm for the model. However, a single split of the data could, by chance, yield a 20% validation sample that performs particularly well (or poorly). To ensure the most accurate estimate of AUC, we will repeatedly select different 20% subsets of the data for validation with the remaining 80% used for building the prediction model. The final AUC estimate will be the average of the AUCs estimated on the repeated splits. To assess algorithmic bias, we will compare AUC across relevant demographic subgroups and test for statistically significant differences in model performance. Note that we will not fit subgroup-specific models, only that we will assess performance in each group individually as well as overall model performance.

We anticipate both item/feature-level missingness and missingness from loss to follow-up, which we will treat differently. For item-level missingness we will use a combination of imputation and including indicators for missing data. Specifically, for categorical data we will simply include a “missing” category. For continuous observations we will use mean imputation and add a binary indicator for missingness. It is possible that missingness in these variables itself is predictive, and including these indicators can capture these relationships while imputation itself cannot. The flexibility of the Super Learner makes it particularly appropriate to take advantage of these relationships. For loss to follow-up, we will not impute missing outcomes, as outcome imputation would constitute an auxiliary prediction model and could conflate with our primary prediction task. Instead, we will explore incorporating non-response weights into the calculation of the AUC to reduce potential bias in performance estimates.

Aim 3:

Finally, for Aim 3, we will use multivariate growth mixture models (GMMs) to identify longitudinal trajectories in each of our ROUD components (pain, opioid use, non-opioid substance use, and post-surgical trauma). GMMs assume that within-person longitudinal trajectories of (possibly multivariate) outcomes arise from a finite mixture of unobserved, latent trajectories. The goal of GMMs is to identify the latent trajectories and the extent to which each latent trajectory contributes to each individual’s observed data. Unlike Aim 2, we will not dichotomize continuous or semi-continuous measures (i.e. pain scores, rates of substance use, trauma scores), which will increase statistical power and trajectory interpretability. The primary statistical challenge of growth mixture modeling is identifying the appropriate number of latent trajectories. We will use standard quantitative measures (e.g., AIC, BIC, Lo-Mendell-Rubin likelihood ratio test) as well as more qualitative approaches (interpretability, trajectory membership size) to identify the most appropriate number of latent classes.

GMMs assign each individual a probability of membership within each trajectory. Statistical practice varies on how to incorporate these probability estimates as outcomes for predictive models. While researchers often assign each individual to a particular latent class (either deterministically by selecting the class with the highest probability or randomly drawing a class from the class probabilities), we will use the latent class probabilities themselves as outcomes in subsequent statistical models. This approach appropriately accounts for uncertainty in the class assignments while not requiring costly computation from pooling results across multiple models with multiple random class assignments. Given that our outcomes lie on a unit simplex (i.e., non-negative and sum to one), we will use Dirichlet regression to identify associations between baseline predictors and the latent class probabilities. We will identify the baseline trajectory as either the most common or one with the lowest ROUD, although often these are the same. For Aim 3, we will handle missing data in a similar way as with the Aim 2 analyses. GMMs do not require complete outcome data and naturally accommodate missing data.

3.2. Sample Size Estimates

We focus power calculation on Aim 2 as it is our primary analysis. The sample size of up to 10,000 dyads (20,000 participants total) ensures ample power for the primary outcome and allows separate analyses even in relatively small segments of the population. Similar studies have yielded relatively high AUC estimates (0.85 – 0.97) for similar outcomes in adults with sample sizes ranging from ~300 to ~600,000 (Song et al., 2024). High AUCs for the smallest studies suggest that even the most pessimistic recruiting estimates would allow the study to achieve its primary goal, including in situations with low rates of ROUD and high loss to follow-up. We are also well-powered to detect simple associations between ROUD and individual potential predictors outside of the prediction model. Figure 1 shows the minimum detectable effect size at 80% power under varying sample sizes and prevalence of ROUD in the population. As is evident from the figure, we maintain the ability to detect “small” effects (Cohen’s d < 0.20) for sample sizes as low as 1,000 if overall rates of ROUD are high (> 27%). For lower rates of ROUD (e.g., 10%), minimum sample sizes would need to be at least 2,200 to maintain the same power to detect effect sizes less than d = 0.20.

Figure 1.

Minimum Detectable Effect Size by Sample Size and ROUD Proportion

A common concern in machine learning prediction models is that they may perform worse for small subgroups of the data than they do for the whole sample. AUC is often mapped to qualitative performance in bins of width 0.1 (e.g., an AUC of 0.8 – 0.9 is often considered “good”). As such, we define a meaningful decrease in AUC as a 0.1 reduction. The following power calculations assume an overall AUC of 0.85 (the lower end of what similar studies were able to achieve) and have a baseline ROUD frequency of 20%. The table below shows the minimum frequency of the subgroup in the overall population required to have 80% power to detect this meaningful reduction in model performance.

As is evident from the table, we are well powered to detect meaningful differences in predictive performance in small subpopulations, even if recruitment is substantially lower than anticipated.

4.0. Discussion

4.1. Limitations

Although the combination of EMR and self-report data from patients and parents used in APEX address the limitations of past studies of post-surgical opioid use outcomes, the study design is not without limitations. For one, the follow-up period of one year is not long enough to observe the development of OUD in a meaningful number of participants. Although participants who endorse past opioid use do complete an OUD DSM-5 checklist at baseline and 12 months, the incidence of OUD in this sample is expected to be too small to conduct analyses of OUD directly. For this reason, we selected ROUD as the composite outcome measure instead of OUD. Although the outcomes included in ROUD each have significant evidence suggesting a connection with future development of OUD, the creation of this composite measure was novel. It may be useful to extend the follow-up period so that the incidence of OUD can be estimated and predictors for OUD, rather than ROUD, can be identified.

As with all longitudinal studies, there is risk of loss to follow-up, and it is possible that participants with higher OUD risk selectively drop-out at higher rates. For example, patients with significant prolonged pain or escalating trajectories of substance use may stop completing surveys before their study participation window ends. The final sample that completes the 12-month survey may be skewed to represent those with no or less ROUD than participants who are lost to follow-up. Additionally, some self-reported ROUD will activate study safety protocols; for example, a report of opioid use 3 or more months after surgery would result in the participant being provided with national resources and, possibly, clinical intervention. Although past research by our group suggests that receiving a safety intervention does not impact study participation rates (Levy, Tennermann, et al., 2021), receiving a safety intervention that targets a measured outcome of the study may confound results.

Our sample draws from five large pediatric medical centers which is a strength. However, the final sample may not be nationally representative of the adolescent population undergoing surgery, and generalizability to other groups will be unknown. While we anticipate that this study will provide important information on the association between biopsychosocial parameters and the development of risk for opioid use disorder among adolescents undergoing surgery, generalizability to adolescent populations who are treated with opioids for other indications will be unknown.

4.2. Future Directions

Findings from this study are expected to yield information about potential risks which may lead to new screening tools to assess risk prior to opioid prescribing, updated protocols for managing pain intra-operatively and post-operatively in high-risk populations, updated patient and family education materials to reduce risks for those undergoing painful procedures, and revised recommendations for monitoring patients being treated for pain. Furthermore, the project enables the establishment of an infrastructure among pediatric surgical centers that can be used for future projects to evaluate the best post-operative outcomes for youth. Future projects could include qualitative interviewing of patients identified with ROUD to learn more about the context of their experience, a longer observation period for patients identified with ROUD, adding a genetic sample to look for genetic risks for developing OUD, brain imaging to look for anatomical correlates of evolving OUD, metabolomics and proteomics samples that could help decipher the pathway from painful experience, medication consumption and behavioral outcomes. While the primary focus of the proposed project is on patients undergoing surgery with planned postoperative opioid analgesic use, our emphasis on the link between pain and behavior could be inform research questions addressing patients with non-surgical pain.

Table 3:

Minimum Subgroup Frequency for 80% Power

Total Sample Size	Minimum Subgroup Frequency
2,000	16%
3,000	10%
4,000	8%
5,000	6%
10,000	3%

Highlights:

• Surgical procedures are the first exposure to opioids for many youth and the initial experience may become a driver of future opioid use disorder (OUD).

• Administrative data suggests that prescribing details (formulation, dose/duration of prescription) are associated with risk for opioid use disorder, but these databases are not able to assess individual risk factors.

• The Adolescent outcomes of Post-operative opioid EXposure (APEX) study will establish a prospective cohort to provide greater insight into OUD risk after post-surgical opioid exposure during adolescence.

• N=10,000 dyads (20,000 participants) consisting of an adolescent aged 12–17 undergoing a planned surgical procedure requiring peri- and post-operative opioid pain management and a parent/caregiver will complete surveys for one year pre- and post-operatively.

• Self-report and extracted electronic medical record data will be combined to identify variables that contribute to risk for OUD.