Challenges in conducting an individual participant data meta-analysis of non-steroidal anti-inflammatory drugs for patients with (sub)acute low back pain: a systematic review and report on data availability

Abstract

Objective

We planned to conduct an individual participant data meta-analysis (IPD-MA) to investigate the efficacy of non-steroidal anti-inflammatory drugs (NSAIDs) compared with placebo in patients with (sub)acute low back pain (LBP). This report describes the challenges encountered in conducting this IPD-MA and their implications.

Design

A systematic review.

Data sources

Medline ALL, Embase and the Cochrane Central Register of Controlled Trials were searched through January 2024.

Eligibility criteria

We included randomised clinical trials (RCTs) from the Cochrane review on NSAIDs for (sub)acute LBP and trials identified through the aforementioned data sources. We compared NSAIDs (regardless of administration route) to placebo on effectiveness outcomes. Our primary outcomes were pain intensity, physical functioning and health-related quality of life at 1-, 3- and 12-week follow-ups, with secondary outcomes including adverse events.

Data extraction and synthesis

Two independent reviewers screened and included eligible studies. Risk of bias (RoB) was assessed using the Cochrane RoB tool 2. Original authors of included RCTs were contacted based on a hierarchy: Corresponding > First > Last > Other authors via email, social media or institution. For contributors, we requested their study protocol, codebook and IPD, and provided our data-sharing agreement and transfer protocol. One-stage IPD-MA was planned with a generalised linear mixed model.

Results

We identified 10 RCTs. Seven authors responded, but only one author agreed to share IPD. Data were unavailable or lost due to the age of the trials, dissolution of research departments, lack of records by sponsoring companies, market withdrawal of the studied drug, or contractual restrictions on data sharing. Two RCTs had a ‘low’ overall RoB, four had ‘some concerns’ and four had ‘high’. With only one IPD set obtained, the planned IPD-MA could not be conducted.

Conclusions

This IPD-MA on NSAIDs for (sub)acute LBP could not be completed due to challenges in data acquisition. In future IPD research, researchers should focus on clearer rationale, recent RCTs, improved data-sharing and storing practices.

STRENGTHS AND LIMITATIONS OF THIS STUDY.

• We used a detailed and systematic approach to search for eligible randomised controlled trials, with professional librarian support, and followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses of Individual Patient Data (PRISMA-IPD) guidelines to ensure transparent and rigorous study selection, quality checking and data retrieval.

• We clearly documented the challenges in data retrieval, offering insights that could help future researchers plan their individual participant data meta-analysis projects better.

• Due to difficulties obtaining data from original studies, we could not perform the planned IPD meta-analysis, limiting our ability to draw conclusions on the effectiveness of non-steroidal anti-inflammatory drugs for acute or subacute low back pain.

• Most of the identified trials were older, which made data retrieval challenging and affected the generalisability and usefulness of our results.

• Our review is limited by a small sample size and the highly specific study setting.

Introduction

Low back pain (LBP) is a widespread and economically burdensome condition affecting people globally.¹ LBP is the leading cause of productivity loss globally,² contributing to significant economic costs, including £2.8 billion in the UK, over $A 4.8 billion in Australia, and more than US$100 billion annually in the USA.^{1 3 4} Although an episode of acute LBP is self-limiting in a majority of cases,⁵ it can have a high recurrence rate, with 50% to 80% of individuals experiencing a return of symptoms within a year.^{6 7}

An overview of clinical practice guidelines reported that non-steroidal anti-inflammatory drugs (NSAIDs) and paracetamol are recommended as the first pharmacological choice for acute LBP.⁸ A Cochrane review concluded that NSAIDs are slightly more effective than placebo in patients with acute or subacute LBP for the outcomes of pain intensity, physical functioning and global improvement.⁹ This Cochrane review used a traditional pairwise meta-analysis (MA) which combines aggregate data from different randomised clinical trials (RCTs) to estimate the average effect of a treatment. In a traditional MA, data are extracted at study-level, which gives rise to two major limitations. First, key information may be unpublished or reported inconsistently. Therefore, the investigator is dependent on how the data are presented and analysed. For example, subgroup effects may be presented as a forest plot without accompanying numerical data. Second, investigators cannot adjust for baseline covariates or assess risk of bias in raw data, such as differential loss to follow-up.

Individual participant data (IPD)-MA can be used to address these challenges.¹⁰ In an IPD-MA, researchers retrieve the raw IPD from different RCTs, including unpublished or unreported information.¹¹ Therefore, the new data analysis is not dependent on what is analysed and reported in the included RCTs, and it allows for more precise analysis, such as adjusting for baseline covariates and applying consistent analytical methods across studies.¹¹ Furthermore, IPD-MA is currently considered the ‘gold standard’ to study the efficacy or effectiveness of treatments in subgroups of patients.¹⁰ In fact, in an IPD-MA, interactions between treatment effects and participant characteristics can be analysed, providing insights that RCTs cannot,^{12 13} since they are often underpowered for subgroup analyses.¹³ Therefore, IPD-MA is more effective in detecting genuine treatment-covariate interactions.

Although IPD-MA offers several advantages over traditional aggregate data MA, it also presents certain challenges, such as the effort required to obtain IPD from the original study teams, potential bias if data are unavailable,¹⁴ preparing datasets and the lack of standardised methods for data collection.¹⁵ The aim of our study was to conduct an IPD-MA to (1) estimate the overall treatment effect of NSAIDs when compared with placebo for reducing pain, improving physical functioning and enhancing health-related quality of life in adults with acute or subacute LBP, and (2) identify any baseline patient characteristics that might be potential moderators of observed treatment effects. Nevertheless, we encountered challenges and obstacles in collecting the original data while working on this IPD-MA project. Therefore, the aim of this article is to report on these challenges and obstacles, and based on these findings, we propose several suggestions to improve the successful conduct of future IPD-MA projects.

Methods

This IPD-MA project was registered in PROSPERO (with registration ID number of CRD42024503235), and the protocol was also published.¹⁶ We reported this study in line with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses of Individual Patient Data (PRISMA-IPD) statement.¹⁷ Being a systematic review, this study was exempt from ethical approval. A summary of the methods is provided in this section. For further details, see the published protocol.¹⁶

Study inclusion/exclusion criteria

We included all relevant studies without language limitations, adhering to the following criteria:

1. The study design was RCT.

2. The participants were adults (ie, aged 18 years or older) experiencing acute or subacute non-specific LBP (ie, the duration of LBP <12 weeks), with or without leg pain. In case a study involved a mixed population, we included it and extracted IPD from people with acute and subacute LBP for analysis.

3. The treatments of interest were NSAIDs regardless of the route of administration (eg, oral, topical).

4. The comparator treatment was placebo.

5. The outcomes of interest included pain intensity, physical functioning and health-related quality of life (HRQoL), which are core outcome domains for LBP RCTs.¹⁸ We had no restriction on the follow-up period.

We included RCTs without restrictions on study characteristics such as research year, amount of missing data and the overall study risk of bias (RoB). We excluded RCTs focusing on chronic LBP, LBP caused by serious underlying conditions (eg, tumours, vertebral fractures, infections or axial spondyloarthritis), which may require specific treatments.¹

Search strategy

We included RCTs that were searched up to January 2020, as deemed eligible in the Cochrane review by van der Gaag et al.⁹ Additionally, we extended our literature search with the help of a professional librarian to cover acute and subacute LBP from the last search date of the Cochrane review (7 January 2020) to 15 January 2024 in databases of Medline ALL, Embase and the Cochrane Central Register of Controlled Trials∗ (*Registry-only records were removed manually after the CENTRAL search). The details of the search strategy can be found in online supplemental appendix 1.

Two independent reviewers (YF and SDM) screened titles, abstracts and full texts. When the two reviewers could not reach a consensus on eligibility, a third reviewer (AC) was consulted for a final decision. The reasons for excluding full-text articles were recorded in online supplemental appendix 2. The study selection process was summarised using the PRISMA flowchart.

Quality assessment of included studies

All RCTs were assessed by at least two authors for each relevant outcome using an adapted version of Cochrane RoB2 tool, excluding signalling questions in domain 5 (ie, questions 5.2 and 5.3) which are not applicable to IPD-MAs.¹¹ The quality assessment was an important step for checking IPD because the RoB indicated the likelihood of biased results in each trial.

Strategies to retrieve the data

We reached out to the authors following a prioritised order and preferred contact methods, outlined below:

Priority order of authors contacted:

1. Corresponding author.

2. First author.

3. Last author.

4. Other co-authors.

Preferred contact methods:

1. E-mail (primary method).

2. Social media platforms (eg, ResearchGate and LinkedIn; used when an author’s e-mail address could not be found).

3. Institutions (when an author’s e-mail address could not be found or when the author did not respond directly).

We attempted to retrieve the IPD in three time-windows. During the first time-window (2 weeks from 3 May 2024 to 17 May 2024), we sent emails with our study protocol to invite original authors to contribute to our IPD-MA by sharing data (on 3 May 2024) and set 2-week response time (until 17 May 2024). If there was no response from the authors or institutions within 2 weeks, data retrieval entered the second time window (10 working days, 30 May 2024 to 11 June 2024). For authors or institutions who had not responded yet, we contacted them for the last time in the third time-window (3 weeks, 27 June 2024 to 18 July 2024).

Strategies for data sharing

For authors who agreed to share the data, we had planned three sequential follow-up emails: (1) to request the protocol and codebook of the included trial, (2) to request the author to sign and return a proposed data sharing agreement and (3) to provide the data transfer protocol and formally request data.

Data analysis

We had planned a verification process of the data completeness and accuracy on the receipt of the IPD, before harmonising the datasets.¹⁶ We originally considered a one-stage IPD-MA under a generalised linear mixed model framework to estimate the overall treatment effect of NSAIDs compared with placebo, and to identify the moderation of NSAID treatment effect.¹⁶ Since only one study agreed to share data, we were unable to proceed with a formal IPD-MA.

Patient and public involvement

None.

Results

We included the nine eligible RCTs identified in the Cochrane systematic review. For the updated systematic search, we screened 2055 records by title and abstract. Six RCTs were screened in full text, and after this, we included one new RCT. In total, we identified 10 eligible RCTs, as shown in figure 1. The reasons for excluding trials based on full-text review are listed in online supplemental table S1.

Figure 1. PRISMA flowchart. PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

Characteristics of the included studies

The 10 eligible RCTs spanned several decades, with five published before 2000, three between 2000 and 2010 and two between 2010 and 2020. Six of the studies were conducted in primary care settings, two in tertiary care, while the settings for the remaining two studies were not specified. The sample sizes varied between 69 and 746 participants across the studies (with the median of 263 patients). Nine RCTs measured back pain intensity using different questionnaires or scales, seven studies assessed physical functioning, and one study reported on HRQoL with the Short Form-12 questionnaire. Further details on the study characteristics and the patients included can be found in online supplemental table S2.

Quality assessment

Based on the quality assessment results of the included studies using the adapted RoB 2 in table 1, the overall RoB varied across RCTs and outcomes. Two RCTs demonstrated a ‘low’ overall RoB, four exhibited ‘some concerns,’ and four were rated as having a ‘high’ overall RoB. Concerns were frequently noted in domain 2 (ie, deviations from intended interventions) and domain 5 (ie, signalling question 5.1 describing possible selection of reported results due to the absence of a study protocol). In contrast, domains 1 (randomisation process), 3 (missing outcome data) and 4 (measurement of the outcome) were often rated as having a ‘low’ risk in the majority of RCTs.

Table 1. Quality assessment of included trials with modified Risk of Bias 2 (Rob 2).

Studies	Assessed outcome(s)	Domain 1. Randomisation process	Domain 2. Deviations from intended interventions	Domain 3. Missing outcome data	Domain 4. Measurement of the outcome	Domain 5. Selection of the reported result (adapted)*	Overall judgement
Lacey et al27	Affected part pain	Some concerns	High	Low	High	Some concerns	High
Amlie et al26	Pain in lying, standing and sitting	Some concerns	High	Low	High	Some concerns	Some concerns
Postacchini et al19	Pain severity	Some concerns	High	High	High	High	High
	Daily activity ability	Some concerns	High	High	High	High
Babej-Dolle et al20	Pain intensity	Low	Low	Low	High	Some concerns	Some concerns
Szpalski and Hayez23	Pain intensity	Some concerns	Low	Low	Low	Some concerns	Some concerns
Nadler et al21	Pain relief	High	Some concerns	Low	High	Low	High
	Disability	High	Some concerns	Low	Low	Low
Dreiser et al25	Pain intensity	Some concerns	Low	Low	Some concerns	Some concerns	Some concerns
	Disability	Some concerns	Low	Low	Some concerns	Some concerns
Hancock et al22	Pain intensity	Low	Low	Low	Low	Low	Low
	Physical function	Low	Low	Low	Low	Low
	Disability	Low	Low	Low	Low	Low
von Heymann et al24	Pain intensity	Low	Low	Low	Low	Some concerns	Low
	Disability	Low	Low	Low	Low	Some concerns
	Quality of life	Low	Low	Low	Low	Some concerns
Predel et al28	POM	Some concerns	low	Low	High	Low	High

^*The adapted domain 5 excluding signal questions: ‘5.2 Is the numerical result being assessed likely to have been selected, on the basis of the results, from multiple eligible outcome measurements (eg, scales, definitions, time points) within the outcome domain?’ and ‘5.3 Is the numerical result being assessed likely to have been selected, on the basis of the results, from multiple eligible analyses of the data?’.

POM, pain on movement.

Data collection

Two out of 10 authors responded at our initial contact, resulting in a 20% response rate during the first time window. In the second window, three more authors responded, increasing the response rate to 50%. Finally, in the third time window, two additional authors responded, bringing the total response rate to 70%. The three non-responding RCTs concerned older publications, published in 1988, 1994 and 2002, respectively.19,21 For the RCTs with no response, we also contacted all authors and affiliated institutions for whom we had contact details, but we did not receive any reply during any of the time windows.

Among the seven responding trial teams, one²² agreed to share their original data with us for inclusion in our IPD-MA project. The other six trials declined to share data for different reasons. These reasons were as follows: (1) data loss due to the RCT being conducted a long time ago²³; (2) data loss as far as co-authors’ knowledge²⁴; (3) data loss because the research department was dissolved and the study was conducted a long time ago²⁵; (4) data unavailability because the cited pharmaceutical company reported no record of the department or of involvement in the study²⁶; (5) the medication studied (ie, Piroxicam) had been withdrawn from the Dutch market²⁷ and (6) contractual obligations preventing data sharing.²⁸ Details of communication with the trial teams are shown in table 2.

Table 2. Contact details.

Study	Time to reply (from 3 May 2024 to 18 July 2024)	Response	Contact information	Result	Rejection reason(s) (if possible)
Lacey et al27	31 days	Institution	Email	Rejection	Feldene (piroxicam) is withdrawn from the market
Amlie et al26	59 days	Institution	Email	Reject	Data unavailability: The institution listed in the publication—a pharmaceutical company—responded that they had no record of the department mentioned in the article or any involvement for this study
Postacchini et al19	Null	Null	Email ResearchGate message	No response	Null
Babej-Dolle et al20	Null	Null	Email Phone	No response	Null
Szpalski and Hayez23	1 day	First author	Email	Reject	Data loss: The data had originally been recorded on paper and could no longer be located. The respondent also could not recall which contract research organization (CRO) was involved, preventing data retrieval
Nadler et al21	Null	Null	Email	No response	Null
Dreiser et al25	27 days	Institution	Email	Reject	Data loss: The CRO (Clinica Statistica) ceased to exist in 2006, and the database could not be located. Given the 20-year time lapse, the current sponsor (Novartis) was also unable to retrieve the data
Hancock et al22	1 day	First author	Email	Agree	No
von Heymann et al24	60 days	Third author	Email	Reject	Data loss: the data from the 2003–2008 trial were no longer available and therefore could not be shared
Predel et al28	31 days	Corresponding author	Email Vivli system	Reject	Contractual obligations

Discussion

This study aimed to evaluate the overall treatment effect of NSAIDs for patients with acute or subacute LBP and to identify potential moderators of NSAID effectiveness using an IPD-MA. Through a systematic literature search, we identified 10 eligible RCTs for inclusion. Five of these studies were published before 2000. A low data acquisition rate of 10% of the original data prevented us from conducting the IPD-MA and led to the project’s discontinuation.

As the gold standard for synthesising evidence across clinical studies,²⁹ IPD-MA uses raw participant-level data from published and unpublished studies, including those excluded from original trial analyses, enabling more comprehensive analysis.¹¹ However, during the conduct of an IPD-MA project, challenges are often encountered, such as availability bias, the time-intensive nature of data collection and the requirement for advanced analysis techniques.30,32 The biggest challenge we encountered was retrieving IPD for the identified RCTs. Despite increasing demand for IPD sharing and supportive policies, many IPD requests, especially for studies published a long time ago, failed to receive responses.³⁰ In our study, half of the trials were published over 30 years ago, with the oldest dating back 40 years, making it challenging to contact original authors or institutions. Three studies, published in 1988, 1994 and 2002, were contacted multiple times without response.

Of the seven studies that responded, six declined to share their original data for various reasons. The most common reason for declining the invitation to contribute was the inability to retrieve the IPD because the studies were conducted a long time ago. This is consistent with findings from the literature, which show that the availability of data and author contact information decreases significantly with article age, with the odds of data being extant decreasing by 17% per year and the availability of a working email address of the first, last or corresponding author decreasing by 7% per year.³³ In our study, one research team did not exist anymore, further contributing to the unavailability of the required IPD. In hindsight, when we planned to conduct the IPD analysis, it would have been wise to assess how many datasets as well as the number of participants were available, in particular whether they had been published in recent years. In our case, 10 trials, with only 5 published after 2000, were insufficient. We had hoped to obtain at least three datasets from recently published trials. Although there is no fixed minimum number of included studies for conducting an IPD-MA, a greater number of studies typically provides more robust estimates, particularly when assessing treatment-covariate interactions. However, whether the time invested and the resulting outcomes justified pursuing this project remains debatable.

Although older RCTs still contribute valuable knowledge to research, the standards (eg, Cochrane RoB 2 for assessing the methodological quality of RCTs) and methodologies (eg, blinding techniques like double-dummy blinding) for conducting RCTs have evolved significantly over time. For RCTs published a long time ago, we need to check if their standards or methods are outdated. If so, careful considerations about whether the potential low quality of the data and the difficulties in obtaining it from the authors justify the efforts required. However, when IPD is available, we are less dependent on the methods used in the original RCTs. The advanced statistical methods in IPD meta-analyses, such as one-stage analysis, allow researchers to reanalyse data using raw, participant-level information and reduce biases present in the original analysis.

Another challenge is creating a data-sharing agreement applicable for international data transfers to safeguard the sharing of IPD in the data acquisition process. A RCT reported that implementing a data-sharing agreement can increase the likelihood of data sharing by 24%.³⁴ Health research is becoming increasingly data-intensive projects in global collaborations. IPD-MA is a research method that relies on international collaboration and access to large amounts of raw data. However, transferring data across countries and regions is particularly challenging due to varying legal frameworks governing the protection of personal data protection.³⁵ This makes it difficult to establish a universal data-sharing agreement. Even within the same country, universities often have their own data-sharing policies and rules. In this study, a data-sharing agreement provided by the Erasmus MC Legal Department was prepared specifically to allow for data transfer to our institution. However, when the sharing of IPD is restricted by the policies and rules of the original author’s institution or country, the request for IPD is likely to be declined. In the present attempted IPD-MA, one trial published in 2020²⁸ was unable to provide IPD due to their own contractual obligations.

Alongside data-sharing agreements, future clinical trials should ensure that participants approve IPD sharing during the informed consent process to uphold ethical standards. While data sharing enhances research and can lead to valuable findings, it also introduces potential risks, such as the re-identification of patients’ personal information.³⁶ Therefore, it is necessary to fully inform patients of the potential benefits or risks of data sharing and seek their consent. If data sharing is not addressed in the informed consent, the original authors may be restricted from sharing IPD. However, in some cases, if the secondary analysis addresses the same research question as the original study, this may not constitute a barrier—though policies vary at both organisational and national levels.

Even with a data-sharing agreement in place, the transfer and storage of data for an IPD-MA can still present challenges. Once the data-sharing agreement is signed, the original authors should encrypt the IPD and share it with the researchers through a secure platform. Several platforms, such as AxCrypt and VeraCrypt, can facilitate encrypted data transmission. However, many of these require payments and necessitate that both the uploader and the recipient download the software. While these platforms offer strong security, the process of encryption, decryption and file handling can be quite cumbersome. To streamline the IPD collection, it is advisable to develop a data transfer protocol in advance and opt for a free, user-friendly platform. In this study, we had planned to use the free platform SURFfilesender. This platform allows authors to share files easily and securely without requiring additional software installations and payment. However, the platform is only available within Dutch academic institutions, and we were unable to test the system due to challenges in obtaining the necessary datasets.

IPD needs to be stored in reliable and long-term data repositories. While we aimed to use existing repositories to facilitate data access and sharing, this came with its own difficulties. Accessing these repositories often requires meeting strict criteria, such as institutional affiliations and subscription fees, which can complicate the process.^{37 38} Furthermore, keeping these platforms adequately funded and up to date is essential to ensure they remain secure and effective for data storage.³⁹ To support the long-term success of IPD sharing, sustainable funding and regular technological updates for these repositories are crucial.

Implications for future research

The challenge of acquiring sufficient IPD presents significant hurdles in conducting an IPD-MA. To facilitate smoother project progress, we recommend a set of strategies based on the findings of this study. First, for IPD-MA researchers, it would be helpful to select a reasonable timeframe and exclude outdated RCTs (eg, before 2000, when electronic data capture systems were uncommon and not yet widely used, and research data are often stored for a limited period and may subsequently be destroyed to protect participant confidentiality⁴⁰ when screening eligible trials. However, even among more recent studies, availability of IPD may still vary by the size or resources of the original team or institution. This approach can increase the response rate from original authors, improve the likelihood of acquiring IPD and reduce the time spent on attempting to retrieve old datasets. Second, at the initial stage of an IPD-MA project, a preliminary search for RCTs that may meet the inclusion criteria in literature databases can help assess whether enough eligible RCTs have been conducted in the past decade. If the search reveals that most studies are older, with few recent studies available, careful consideration should be given before deciding to proceed with an IPD-MA project. Third, a data-sharing agreement should be discussed and shared with potential data providers early in the process, as early transparency around data-sharing terms fosters trust and increases the likelihood of successful collaboration.

For researchers who plan to conduct new RCTs, we suggest obtaining participants’ consent to share their original data for future research purposes during the recruitment process. It is essential that participants are fully informed of both the benefits and risks associated with data sharing. For raw data shared, we recommend that researchers store it in an independent repository managed by a secure and reliable data storage platform or academic institution. The data repositories ensure long-term preservation and accessibility of clinical study data while enhancing security, discoverability and efficient data sharing and tracking.⁴¹

Conclusions

Obtaining IPD and conducting a MA is often both time-consuming and complex. In this study, an IPD-MA on NSAIDs for patients with (sub)acute LBP could not be completed due to challenges in the data acquisition process. Most of the eligible RCTs (8 out of 10) were published over 15 years ago, leading to issues such as data unavailability and a lack of response from the original authors. Moreover, the absence of a unified data-sharing agreement that aligns with the diverse legal frameworks of different countries or institutions further complicated data acquisition. Here are some recommendations to improve the efficiency of future IPD-MA research projects.

IPD-MA researchers (1) can clearly define the rationale and necessity for conducting an IPD approach before starting; (2) can select a reasonable timeframe to obtain enough recent eligible studies and improve data accessibility and (3) should share and discuss a data-sharing agreement upfront to build trust and facilitate collaboration.

Trial teams can facilitate data sharing by (1) obtaining patient consent during the informed consent process and (2) using data repositories to ensure the long-term preservation and accessibility of the IPD.

Data availability statement

Data sharing is not applicable as no datasets were generated and/or analysed for this study. All data relevant to the study are included in the article or uploaded as supplementary information.