Methodological and Reporting Concerns Regarding “Adverse Drug Events Associated with Opioid Treatment for Pain After Motor Vehicle Crash” [Letter]

Dear editor

With great interest I read the article by Zullo et al, entitled “Adverse Drug Events Associated with Opioid Treatment for Pain After Motor Vehicle Crash,” published in the Journal of Pain Research.¹ The study addresses an important clinical question concerning the safety of opioid prescribing strategies following motor vehicle crashes among older adults. I particularly appreciate the authors’ use of a target trial emulation framework and inverse probability weighting to address confounding in this observational study. However, upon reviewing the authors’ official statistical analysis code made available in the Brown Digital Repository,² I identified several methodological and reporting inconsistencies that require clarification, arranged here by analytical significance.

Inconsistency in the Number of Bootstrap Replicates

The manuscript describes the use of a non-parametric bootstrap approach to estimate 95% confidence intervals. However, the analysis code does not appear to use a consistent number of bootstrap replicates across all analyses. For most of the reported risk, risk-difference, and risk-ratio estimates, the code specifies 500 bootstrap replicates. For example, the intention-to-treat and most per-protocol analyses use bootstrap_replicates = 500. However, the per-protocol risk estimates for the ≥6-day opioid supply group uses only 10 bootstrap replicates, and the per-protocol risk estimate for the ≥30 MME group also uses only 10 replicates.

This distinction is important because a bootstrap confidence interval is based on the empirical distribution generated by repeated resampling. Ten replicates provide an extremely limited empirical distribution and would generally be insufficient for reliable estimation of a 95% percentile confidence interval.³ In contrast, 500 replicates provide a substantially more stable approximation, although a larger number of replicates may be preferable when computationally feasible. I therefore suggest that the authors clarify whether the use of 10 replicates was intentional or resulted from a coding or reporting error. If 10 was unintended, the affected analyses should be rerun using the number of bootstrap replicates used for the other estimates, followed by corresponding updates to the reported confidence intervals.

Inconsistency in the Reported Method for Confidence Intervals

I also noted an apparent inconsistency between the statistical methods described in the manuscript and the footnote accompanying Table 2. The analysis code uses the boot procedure and specifies the percentile method (type = “perc”) for calculating the 95% confidence intervals. Thus, the code appears to implement a bootstrap percentile confidence interval.

However, the Table 2 footnote states that confidence limits were estimated using Huber–White robust standard errors. Huber–White robust SEs and bootstrap percentile CIs are different variance/CI procedures. Bootstrap percentile confidence intervals are obtained from the empirical distribution of bootstrap estimates, whereas Huber–White standard errors provide a robust variance estimate within a model-based framework. Because these approaches can produce different confidence intervals, the authors should clarify which method was actually used to generate the estimates presented in the manuscript and Table 2. The Methods section, statistical code, table footnotes, and Results section should describe the same procedure.

Discrepancies in the Per-Protocol Risk Ratios

I identified several discrepancies between the per-protocol risk ratios reported in the Results section and those presented in Table 2. For the opioid supply strategy, the per-protocol risk ratio is reported in the Results as 1.29 (0.64, 4.57), whereas Table 2 reports 1.29 (0.43, 4.07). For the dosing strategy, the Results report a risk ratio of 0.83 (0.45, 1.76), whereas Table 2 reports 0.89 (0.40, 2.28). Similarly, for the tramadol strategy, the Results report 0.54 (0.21, 4.01) for the per-protocol analysis, whereas Table 2 reports 0.39 (0.14, 3.04).

Of particular concern, the values reported in the Results for the dosing and tramadol comparisons correspond to the intention-to-treat estimates shown in Table 2. This raises the possibility that the ITT estimates may have inadvertently been repeated in the narrative description of the per-protocol results. I recommend that the authors verify the statistical output and revise the Results section so that the reported estimates correspond exactly to the appropriate analysis.

Percentage Reported for Opioid Prescriber Specialty

I also identified a numerical inconsistency in the reporting of opioid prescriber specialty. The Results section narrative reports that 47 individuals (0.9%) received an opioid prescription from a practitioner in another specialty. However, among 510 individuals, 47 corresponds to approximately 9.2%, not 0.9%. Moreover, Table 1 reports this value as 47 (9.2%). Therefore, the percentage in the Results narrative appears to be a typographical or calculation error and should be corrected to 9.2%.

Importance of Reconciling the Analysis Code, Tables, and Results

Taken together, these discrepancies highlight the importance of ensuring consistency between the statistical analysis code, reported methodology, tables, and narrative results. This is particularly important in an observational study using inverse probability weighting and per-protocol analyses, where the analytic procedure is relatively complex and small differences in implementation can affect both point estimates and confidence intervals.⁴

The study provides valuable information regarding adverse drug events associated with different opioid treatment strategies following motor vehicle crashes. Clarifying the bootstrap implementation, particularly the use of 10 versus 500 replicates, and reconciling the confidence-interval methodology and reported per-protocol estimates would improve the reproducibility and interpretability of the findings. I respectfully encourage the authors to review the analysis code and reported results, rerun the affected analyses if necessary, and provide corrected and internally consistent estimates.

Use of Generative Artificial Intelligence

Generative artificial intelligence (ChatGPT, OpenAI) was used to assist with language refinement, organization, and editing of the manuscript. The author reviewed, verified, and takes full responsibility for the final content, interpretations, analyses, and conclusions presented in this correspondence.

Disclosure

The author declares no conflicts of interest related to this communication.