Abstract
In this article we briefly examine the unique features of Single-Case Designs (SCDs) (studies in a single participant), their history and current trends, and real-world clinical applications. The International Collaborative Network for N-of-1 Trials and Single-Case Designs (ICN) is a formal collaborative network for individuals with an interest in SCDs. The ICN was established in 2017 to support the SCD scientific community and provide opportunities for collaboration, a global communication channel, resource sharing and knowledge exchange. In May 2021, there were more than 420 members in 31 countries. A member survey was undertaken in 2019 to identify priorities for the ICN for the following few years. This article outlines the key priorities identified and the ICN's progress to date in these key areas including network activities (developing a communications strategy to increase awareness, collecting/sharing a comprehensive set of resources, guidelines and tips, and incorporating the consumer perspective) and scientific activities (writing position papers and guest editing special journal issues, exploring key stakeholder perspectives about SCDs, and working to streamline ethical approval processes for SCDs). The ICN provides a practical means to engage with this methodology through membership. We encourage clinicians, researchers, industry, and healthcare consumers to learn more about and conduct SCDs, and to join us in our mission of using SCDs to improve health outcomes for individuals and populations.
Keywords: N-of-1 trials, Clinical trials, Network, Clinical trials network, Single Case Experimental Design
1. Introduction
In recent years, there has been a spotlight on personalised medicine, patient-centred healthcare and digital health. This has led to a growing focus on the value of Single-Case Designs (SCDs). SCDs focus on a single participant to draw conclusions that are specific to that participant. But it is common for investigators to conduct a series of SCDs that have the same protocol and pool the data across them to draw conclusions about a larger group of participants. There can be a misconception that SCDs are “qualitative” or “descriptive” case studies, but they are quantitative studies that are designed prospectively and collect data from an individual repeatedly and systematically over time. SCDs include Single-Case Experimental Designs (SCEDs) and Single-Case Observational Designs (SCODs). N-of-1 trials are a particular kind of SCED typically applied in the field of medicine. Fig. 1 shows how the various types of SCDs inter-relate.
Fig. 1.
Taxonomy of common single-case experimental designs using a single participant.
Reproduced from Tate RL, Rosenkoetter U., Wakim D., Sigmundsdottir L., Doubleday J., Togher L., McDonald, S., Perdices, M. (2015) The Risk of Bias in N-of-1 Trials (RoBiNT) Scale. An expanded manual for the critical appraisal of single-case reports. Sydney, Australia: The authors. ISBN: 978-0-9945369-0-7.
SCDs have wide applicability in health-related research and practice [[1], [2], [3], [4], [5]]. Box 1 presents a brief description of common SCD sub-designs and their main features. See supplementary material for more detail on these designs. They have been published in high-profile medical journals including Nature [8], the Journal of the American Medical Association (JAMA) [9]. and the New England Journal of Medicine (NEJM).10. Several CONSORT reporting guidelines for N-of-1 trials and SCEDs have been published (e.g., CENT [6] and SCRIBE [7]) and study quality assessment tools are available [11]. SCDs have not yet been used extensively in either research or practice. For example, studies have shown that clinicians may encounter barriers to conducting SCDs in practice [12]. To facilitate wider adoption of SCDs, we established a formal collaborative network called the International Collaborative Network for N-of-1 Trials and Single-Case Designs (ICN), which aims to disseminate information and stimulate global discussion about SCDs to promote awareness and use of SCDs, and to assist clinicians and researchers in the design, conduct, and analysis of individual and pooled (aggregated) SCDs.
Box 1. Brief description of common SCD sub-designs.
Single-Case Experimental Designs (SCEDs) are experimental designs that are used to test the effect of an intervention for an individual participant (who acts as their own control) using repeated outcome measurements, sequential, randomised or non-randomised introduction of an intervention. Outcome data is evaluated using specialised analysis methods for SCDs, including visual and statistical techniques. Simultaneous or sequential replications are possible with more individuals.
N-of-1 trials are a subtype of SCED and involve evaluating individual participant responses to interventions, by randomly allocating different time periods within an individual to repeated intervention and control (e.g., standard care, placebo, or alternative intervention) conditions. Outcomes are compared across the time periods to determine whether the intervention has a favourable effect on the outcome compared to the control condition.
Single Patient Open Trials (SPOTs) are another subtype of SCED with the key distinction that they are an open (unblinded) design. SPOTs are less rigorous than other types of SCEDs (e.g., N-of-1 trials), but they may be more practical for some clinicians to conduct [13].
Single-Case Observational Designs (SCODs), also referred to as N-of-1 observational designs, involve repeated measurements (e.g., pain severity ratings) from an individual participant over time, in the absence of an intervention implemented by the investigator. Their purpose is to draw conclusions about naturally occurring patterns and predictors of the outcomes measured. Individual-level data can provide insight into highly personalised potential intervention strategies.
Alt-text: Box 1
1.1. Unique features of Single-Case Designs
A major advantage for all types of SCDs is that participants’ individual data can be shared with their clinicians for discussion and can facilitate shared decision-making about future management. These data are often lost in group-based methods, because their focus is on identifying average responses to treatments, rather than individual responses. In aggregated (pooled) SCD studies, data are pooled from individual specific drug or non-drug treatment studies to provide clinicians and patients with average outcome data, like in group-based randomised controlled trials (RCTs) and population-level epidemiological studies. In aggregated (pooled) SCD studies, fewer participants may be needed for similar levels of statistical power as studies that use a group-based design [14].
Collecting individual SCD data allows tracking of changes and treatment response in individual patients, whereas pooling SCD data addresses generalisability of findings. Although SCDs have certain advantages and provide unique opportunities such as providing specific and reliable information about individual response to treatments, and the proportion of treatment responders in a series of SCDs, they can, depending on the context, also have potential drawbacks such as an extended length of study, and the need to allow for and deal with any carryover effects. Traditional group-based RCTs continue to have an important role, for example where it suffices to obtain a precise estimate of an average causal effect (e.g., for national health policy or reimbursement for health insurance purposes). There are particular contexts, conditions, interventions and objectives suitable for one type of study design or the other, and sometimes both.
1.2. History and current trends in single-case research
SCDs have been used in several fields. They have a long history in behavioural and psychological science, dating back to the nineteenth century with Ebbinghaus' pivotal memory research and Stratton's study on the effect of wearing inverting lenses in a single participant, both of which were influential in these fields [15] and this work was built on by Sidman [16] in the early 20th century. Mirza et al. have provided a comprehensive description of the history of N-of-1 studies in medicine [17]. Briefly, before Hogben and Sim's methodological landmark study (using blinding, placebos, multiple crossovers and wash in/out) in the International Journal of Epidemiology [18] there were a few studies of individual responses [19,20]. Baskerville et al. [21] then applied principles of adaptive design, allowing treatment period length to vary depending on adverse events, clinical deterioration, and patient preference. In 1986, Guyatt published a landmark paper on using ‘N of 1 randomized control trials’ for a poorly controlled asthmatic patient treated with inhaled beta agonists, theophylline, and prednisone [22]. For a comprehensive description of the history of SCDs please see Mirza et al. [17] and Hurtando & López-López [23].
There has been renewed interest in N-of-1 trials and SCDs recently [17]. Gordon Guyatt published 11 articles from 1986 to 1995, and there was a steady publication rate of N-of-1 trials and SCDs from 1986 to about 2002, followed by a rapidly increasing publication rate from 2002 to 2010. Early adopters between 2002 and 2010 were on the shoulder of the wave of the technology adoption lifecycle [24], which now in the context of personalised medicine, patient-centredness, self-tracking and digital health, is rising more quickly. Fig. 2 shows exponential growth in SCD studies since 2010, making a definitive case for increased use. This renewed interest is also reflected in other areas such as an increase in published books, media interest and special journal issues. For example, the book “Essential guide to N-of-1 trials in health” [25] has had over 13,000 chapter downloads, and there have been at least 9 special journal issues on SCDs in the last 7 years.
Fig. 2.
Number of SCD publications in Pubmed since 1986. The graph shows exponential growth in SCD articles in Pubmed especially since 2002. The search used was Search query: Single-case experimental design[tiab] OR SCED[tiab] OR N-of-1[tiab] OR N-of-one[tiab] OR "single participant"[tiab] OR "ABA design"[tiab] OR "Single-Case Studies as Topic"[Mesh] OR "single-case design" OR "single subject design" OR multiple baseline design[tiab] OR (("single subject"[ti] OR "single case"[ti] OR "single patient"[ti]) AND (trial*[ti] OR design*[ti])).
There have been some important developments in the last 10–20 years. N-of-1 designs are endorsed by Health Canada for natural health products [26], the UK Medical Research Council for developing and evaluating complex interventions [27], US Food and Drug Adminstration [28] Therapeutic Goods Administration and European Medicines Agency [29] for clinical trials in small populations and the USUS National Pharmaceutical Council for comparative effectiveness research [30]. Among others, the National Institutes of Health, National Health and Medical Research Council, and Patient Centred Outcomes Research Institute (PCORI) have all funded projects for N-of-1 trial research. N-of-1 trials have been cited in systematic reviews [31], healthcare policy documents [32] and clinical [33] and methods guidelines [34]. It is possible that these developments do not fully capture the increasing use as clinicians may be conducting N-of-1 trials and SCD studies and not reporting them as research in scholarly journals because they consider them as patient care [35].
1.3. Real word clinical application and clinical impact of Single Case Designs
SCDs are useful when there is variability in treatment response; this applies to many conditions and situations [17]. Identifying the individual patient response to an intervention is important for clinician-patient dyads for shared treatment decision-making. Studies using SCDs have provided information about the number of treatment responders for various treatments and interventions [[36], [37], [38], [39]]. SCDs have also been used to objectively demonstrate benefit to the patient and their families or carers which represents another advantage [36]. N-of-1 trials have been used to correctly attribute side effects experienced by patients.10, 37 N-of-1 studies have been especially useful in rare diseases, where numbers are too small for conventional RCTs to be readily conducted [9]. The reduced sample size needed for aggregated (pooled) N-of-1 trials, when compared to standard RCT sample sizes [14], has positive implications for recruitment, time, and costs because each person contributes multiple observations. Some real world examples of these uses have been provided in Box 2.
Box 2. Real world examples of uses of SCDs.
Testing treatment effectiveness in replicated SCDs
Simons et al.: Sequential replicated and randomized single-case experimental design with multiple measures evaluating Graded Exposure Treatment (GET) for 27 youth with chronic pain. By follow-up, over 80% of participants had improved across all primary and secondary outcomes. Avoidance, pain acceptance, and pain intensity improved during GET over the no-treatment randomized baseline period, whereas fear and pain catastrophizing did not improve. All 5 outcomes were significantly improved at 3- and 6-month follow-ups. The results support the effectiveness of graded exposure for youth with chronic pain and elevated pain-related fear avoidance [38].
Stunnenberg et al.: An RCT of around 60 people found that mexiletine was effective for treating muscle stiffness in nondystrophic myotonia [39]. a comparable level of statistical support for similar efficacy was found after analysing aggregated data from just 11 N-of-1 trials [9]. These studies are comparable studies of mexiletine.
Roustit et al.: 38 patients with Raynaud's phenomenon completed 2 to 5 treatment blocks of on-demand sildenafil. Aggregated data showed that the probability that sildenafil at 40 mg or 80 mg was more effective than placebo was greater than 90% for all outcomes except for one. However, the aggregated effect size was not clinically relevant. Substantial heterogeneity in sildenafil's efficacy was observed among participants, with clinically relevant efficacy in some patients and no efficacy in others [40].
Identifying individual treatment responders and overall response rate
Mitchell et al.: Forty-three participants completed 84 cycles of methylphenidate and placebo in random order, exceeding sample size estimates. Overall, MPH did not improve fatigue (mean difference 3.2; 95% credible interval −2.0, 9.0; posterior probability of favourable effect 0.890). Eight (18.6%) participants showed important improvement, and one participant (2%) showed important worsening of fatigue on methylphenidate [41].
Yelland et al.: Of 55 participants who completed at least one cycle in an N-of-1 trial, the response to gabapentin (defined by aggregating individual subscale scores (pain, sleep interference and functional limitation; frequency of adverse events and medication preference) was better than placebo in 16 (29%), of whom 15 continued gabapentin posttrial. No difference was shown in 38 (69%), and 1 (2%) showed a better response to placebo [42].
Duggan et al.: An N-of-1 trial of stimulant versus placebo can show parents of a child with ADHD if the child's ADHD symptom scores (measured by Conners' Comprehensive Behavior Rating scales) are better (lower) on stimulants or placebo [36]. Three of four patients studied were clear responders to dexamphetamine (including a non-completer, as his results still demonstrated a clear response). The results were clinically useful in each case. Management was confirmed for three patients and changed for one (who ceased dexamphetamine).
Correctly attributing drug side effects
Herrett et al.: Patients often discontinue statins because of side effects [43], even though some blinded trials have not shown an excess of symptoms with statins as compared with placebo [44]. In a recent study using randomised, placebo-controlled N-of-1 trials with 151 participants who reported severe muscle symptoms when taking statins, no difference in muscle symptom frequency or intensity was found between statin and placebo periods. Two thirds of people completing the trial intended to restart treatment with statins [37].
Wood et al.: Patients who had previously discontinued statins because of side effects that occurred within 2 weeks after the initiation of treatment were enrolled in a double-blind, three-group, N-of-1 trial to compare symptoms induced by a statin or placebo. In patients who had discontinued statin therapy because of side effects, 90% of the symptom burden elicited by a statin challenge was also elicited by placebo. Half the trial completers chose to successfully restart statins [10].
Orloff et al.: In a similar N-of-1 trial looking at metformin gastrointestinal side effects, metformin was associated with significantly lower global treatment satisfaction scores compared to placebo but participants could not distinguish metformin from placebo and did not report higher rates of gastrointestinal side effects on metformin [45].
Alt-text: Box 2
Examples of the clinical impact of personalised SCDs, with changes in subsequent management, include testing the use of psycho‐stimulants for Attention Deficit Hyperactivity Disorder (ADHD) [6,47]; paracetamol vs non‐steroidal anti‐inflammatory agents (NSAIDs) for osteoarthritis [48,49]; and gabapentin vs placebo for chronic neuropathic pain [42]. The work of ICN chair JN and team has led to the following achievements: established feasibility for certain N-of-1 tests and collected evidence of variability in individual drug response [36,42,46,48]; demonstrated usefulness for bridging significant evidence gaps, especially in difficult to research populations where recruitment and retention are difficult, such as pediatrics and palliative care [41,46,47,[50], [51], [52]]; confirmed that N-of-1 trials can improve treatment decisions [[47], [48], [49]] as well as reduce health costs [53,54]; explored patient perspectives of N-of-1 trials [55]; followed up patients to examine impact of N-of-1 trials on immediate and long term management and how patients and doctors who have completed N-of-1 trials use individual high quality evidence to make management decisions [47,49]; and showed that the N-of-1 trial is superior to traditional RCTs in obtaining individual effectiveness results [14].
1.4. Single Case Designs and digital health
SCDs and digital health are a synergistic pairing. The term digital health includes eHealth and mHealth (e.g., telemedicine, electronic health records and wearable sensors) as well as developing areas such as the use of advanced computing sciences in the fields of big data and artificial intelligence, for example. Digital SCDs harness the power of individual patient data, facilitating collection of real world data that can be translated into real world evidence, which healthcare clinicians can use to make informed decisions about patient care [56]. SCDs using digital technologies for outcome assessment and intervention delivery provide an important opportunity to enhance the utility of individual health data (see for example Frontiers in Computer Science Research Topic Creating Evidence from Real World Patient Digital Data (https://www.frontiersin.org/research-topics/10089/creating-evidence-from-real-world-patient-digital-data). Apps for analysing SCD data, such as Shiny (https://shiny.rstudio.com/) and m-Path (https://m-path.io/landing/), are available.
1.5. Vision and mission for the International Collaborative Network for N-of-1 Trials and Single-Case Designs
The need for a SCD community providing opportunities for collaboration, a global communication channel, resource sharing, and knowledge exchange was discussed at a gathering of 26 N-of-1 and SCD experts working on the Single-Case Reporting In Behavioural interventions (SCRIBE) guidelines [7], in Sydney, Australia in 2015. The idea of developing a formal collaborative network for individuals with an interest in SCDs was met with solid support, and in 2017 a formal collaborative network called the International Collaborative Network for N-of-1 Trials and Single-Case Designs (ICN) was established (www.nof1sced.org). The ICN's vision is a world where personalised clinical studies are an integral part of clinical practice and clinical research. Its mission is to promote, support, and advance the use of personalised clinical studies using SCDs, and to share relevant knowledge, experience, expertise, resources, and data through global partnerships between clinicians, researchers, industry, healthcare consumers, and healthcare consumer organisations. Full details about the ICN's objectives can be found on the website (www.nof1sced.org/about). In brief they include:
-
•
Raise awareness of and promote use of personalised clinical studies using N-of-1 trials and other types of SCDs.
-
•
Support clinicians, researchers, industry, and healthcare consumers to conduct and translate personalised research into clinical care.
-
•
Promote the use of digital technologies to facilitate data collection, management, and analysis in personalised clinical studies.
1.6. ICN committee and membership
The ICN has a multi-disciplinary executive committee of clinicians, researchers, industry representatives and healthcare consumers, including two network chairs, 10 expert members, 10 theme coordinators (for medicine, psychology, digital health, statistics, complementary and alternative medicine, nursing, health economics, physiotherapy, occupational therapy, and nutrition), two early career representatives, and two healthcare consumers. The committee meets virtually to discuss activities and strategy. Several administration officers manage social media, membership, communications, website, external relations, and blog coordination.
In May 2021, there were more than 420 ICN members in 31 countries (Fig. 3), from a range of health and education disciplines. The ICN provides a platform for members to discuss shared priorities, learn, generate ideas, share knowledge, and inspire each other, without the barrier of membership fees since it is free to join.
Fig. 3.
World map showing countries of origin of ICN members (31 countries in May 2021).
1.7. What the ICN does
Theme coordinators regularly organise short blogs on relevant topics and recommend recently published papers to feature. A monthly newsletter has links to new blog posts, selected papers, new SCD resources, upcoming events, and other topics of interest, for example relevant journal special issues on SCDs. The ICN's website allows clinicians, researchers, industry, and healthcare consumers to interact, and provides a platform to encourage collaboration and discussion. Members are encouraged to share new SCD ideas and projects, provide updates on current projects, events, and activities, discuss key research papers and ask and answer important SCD questions (methodological, clinical, applied, and theoretical).
1.8. ICN activities and progress to date
The strategic direction of the ICN during its first few years of operation was informed by an online survey sent to members in 2019 to elicit their views about key priorities. Seventy-five respondents viewed the following areas as top priorities for the following few years:
1.9. Network activities
-
1.
Developing a communications strategy to increase awareness of the ICN
-
2.
Collecting and sharing a comprehensive set of SCD resources, guidelines and tips
-
3.
Incorporating the consumer perspective in the ICN's objectives and activities.
1.10. Scientific activities
-
4.
Writing position papers; and guest editing special journal issues
-
5.
Exploring key stakeholder perspectives about SCDs
-
6.
Working to streamline ethical approval processes for SCDs
The ICN's executive committee developed strategic plans for addressing these priorities, which are briefly described below with a summary of progress to date.
1.11. Network activities
The ICN has developed a communications strategy to increase awareness of SCDs. The communication strategy includes:
-
1.
Sharing SCD news, promoting members' publications, resources, videos, posters, and events through the ICN's website (www.nof1sced.org), monthly newsletters, and the ICN Twitter (www.twitter.com/nof1SCED) and LinkedIn (www.linkedin.com/company/nof1sced) accounts.
-
2.
Involvement in teaching SCD methods to bachelor, Master and PhD students and clinicians.
-
3.
Explaining SCD methodology with case study examples to consumer groups to raise the profile of SCD and citizen science.
-
4.
Raising the profile of SCDs through involvement in conferences, symposiums, and events to showcase methods, applications and benefits of SCDs.
The ICN collects and shares resources, guidelines and tips for designing, conducting and analysing SCDs. The ICN's resources webpage provides lists of useful guidelines, books, websites, analysis resources, e-courses, literature reviews, and special journal issues that cover various aspects of the design, conduct, and analysis of SCDs (www.nof1sced.org/resources). Several members of the ICN have published accessible “how-to” guides [57,58] and tutorial papers [[59], [60], [61]] to foster increased knowledge and skills in relation to the design and analysis of various types of SCDs. The ICN also offers an advice service to answer specific queries about how to design, conduct, and analyse SCDs.
The ICN's members have delivered workshops where tips for designing, conducting, and analysing SCDs are shared. A successful SCD symposium in Stockholm in 2019 attracted over 250 participants. This was a conference focussing solely on Single Case Designs, run by ICN members PO, JV, and RW. A second international SCD symposium was held online in 2021 (https://ppw.kuleuven.be/ogp/smallisbeautifulagain) attended by 261 participants. Other planned workshops are listed on the ICN events webpage (www.nof1sced.org/events). Another ICN member hosted a virtual symposium consisting of 10 video talks from experts in the SCD field, including ICN chairs JN and SM, which have been posted online for general viewing (www.nof1sced.org/virtual20). These activities represent a useful way to raise awareness and knowledge about SCDs.
The ICN supports consumer engagement and partnership to incorporate the healthcare consumer perspective. Healthcare consumers frequently test treatments and try different management strategies to improve health and/or quality of life, often without good tools beyond a diary, a mobile phone, and perhaps a heart rate or activity monitor. SCDs offer a more rigorous method. Consumers are part of the ICN's executive committee, are involved in setting and implementing the ICN's agenda, and have contributed to this article. The ICN's consumers provided crucial input into ICN's consumer and partnership engagement strategy (www.nof1sced.org/consumers). We will develop healthcare consumer accessible materials to facilitate understanding about the ICN's objectives and the relevance of individualised health research. We will continue to elicit consumers' perspectives about ICN priorities and activities going forward.
1.12. Scientific activities
Organisers of the “Small is Beautiful” symposia on SCDs have written a position paper summarising the 2019 conference presentations and pre-conference expert meeting [62]. The ICN co-chairs JN and SM have guest-edited a special issue on N-of-1 trials in Healthcare, [63] and with others are co-editors of a Frontiers special research topic “Creating evidence from real world patient digital data,” [64] which showcases how SCDs are consistent with the movement towards the collection of real-world data to develop real-world evidence [56], by collecting patient-generated data from in-home-use settings and via mobile devices. As of May 2021, these special issues have had >22,000 [63] and >50,000 views [64] (the latter in particular from US, Germany, China, France, and UK) respectively). The interest generated by these special issues supports the view that there is increasing interest in SCDs.
An important aspect of the ICN's work is exploring barriers and concerns in relation to SCDs. Despite the unique advantages of SCDs and availability of how-to articles for clinicians [57,65] and researchers [66], they are not yet broadly adopted. Barriers identified by doctors are lack of time and awareness, and insufficiently valuing reduced therapeutic uncertainty against inconvenience [12]. The ICN co-chairs SM and JN and other ICN members have conducted a systematic review of stakeholder perspectives about SCDs [67] and are conducting a qualitative study to identify the issues impeding wider uptake of SCDs in healthcare from the perspective of various stakeholder groups including health professionals, patients, clinical trial coordinators, research methodologists, biostatisticians, Institutional Ethics Review Board members, journal editors, regulators and health research funders [68]. The findings from these projects will inform the design and implementation of strategies to address the barriers identified.
In addition, ICN members including JN, SM, and JM addressed some common statistical and design concerns regarding aggregated (pooled) N-of-1 trials (e.g., carry-over effects, selection bias), comparing them to traditional group-based parallel and cross-over RCTs using statistical simulation. Findings showed appropriately designed aggregated N-of-1 trials offer substantial advantages over these alternative designs [14]. N-of-1 trials outperformed both traditional parallel RCTs and crossover designs when trial designs were simulated in terms of power and required sample size to obtain a given power and allowed better estimation of patient-level random effects.
Simplifying ethics applications for SCDs is important to facilitate future SCD clinical research, especially when only one participant is involved [69,70]. In addition, ethics approval is not needed if the SCD is solely for clinical care, which has previously led to uncertainty and debate about whether SCDs require ethical approval. ICN members are addressing this issue in various countries (e.g., The Netherlands) and have developed a practical a flowchart to support decision making about whether ethics approval is needed for a particular context [71].
1.13. Future directions for the ICN
SCD databases, platforms, and registries are an important part of the ICN's future direction. An open, transparent, deep phenotype data bank is needed, where SCD protocols, Case Report Forms and data can be deposited; a collaborative proposal for this has been developed, facilitated by the availability of SCD compatible disease registries. The databank will have implications for data management, collaborations, consent models, method development, and utility of digital tools for data collection. The ICN is also supporting the development of Trial-Ready Registry Framework, a digital infrastructure to support N-of-1 and adaptive clinical trials [72]. To facilitate clinicians becoming more amenable to engaging with trial-ready protocols, this open-source solution will enable seamless capture and linkage of clinician-entered and patient-reported data with health system administrative data, improving efficiencies for assessing and connecting eligible patients to trials, supporting efficient systematic capture of trial data, and enabling near real time Bayesian analysis for novel trial designs. The platform is designed to facilitate and simplify the process for clinicians and researchers to conduct N-of-1 (and other) clinical studies using trial-ready or template-generated protocols, and to responsibly store and share data. A trial-ready system will facilitate clinician involvement, but other issues may remain, such as clinicians' lack of awareness or understanding about SCDs and difficulties obtaining funding. Management of consent and privacy in data banks and registries is of high concern to consumers, so a priority area to focus on is establishing a relationship of trust with healthcare consumers.
2. Conclusion
The future looks bright for SCDs, especially now that digital health, precision medicine, and consumer engagement in research are strongly shaping the healthcare and clinical research landscapes. SCDs offer unique advantages for enhancing clinical service delivery and clinical research for individuals and groups of patients and should be more widely used. The ICN exists to facilitate engagement with SCDs through a range of activities to raise awareness, knowledge and skills, as well as through the conduct of empirical collaborative research to identify and address the challenges and barriers that currently prevent widespread adoption. We hope our work, facilitated by leveraging the utility of digital tools [73], will lead to wider adoption of SCDs across healthcare. We call on clinicians, researchers, industry, and healthcare consumers to learn more about SCDs and encourage them to join our growing community that is collectively committed to utilising SCDs to produce better health outcomes for individuals and populations.
Ethical approval information
Not applicable.
Funding information
Johan W.S. Vlaeyen is supported by the “Asthenes” long-term structural funding Methusalem grant by the Flemish Government, Belgium (METH/15/011).
Contributorship
Jane Nikles is a registered Medical Practitioner with 20 years’ experience in conducting N-of-1 trials. Jane is co-chair of the ICN.
Patrick Onghena is Professor of educational and behavioural statistics and methodology and has expertise in the design and analysis of studies using single-case experimental designs.
Johan W.S. Vlaeyen, PhD is Professor of behavioural medicine and has expertise in the development and evaluation of customized cognitive-behavioural management strategies for individuals suffering chronic bodily symptoms.
Rikard Wicksell is Associate Professor in psychology and has expertise in the evaluation, predictors and moderators of change, the role and function of biological processes in pain and behavioural treatment and measurement development in pediatric chronic pain.
Laura E. Simons is Associate Professor of anesthesiology, perioperative and pain medicine (pediatric). She has expertise in pediatric chronic pain, pain psychology and pain rehabilitation.
James M. McGree is Professor of Statistics and the ICN statistics theme leader with expertise in Bayesian statistics in N-of-1 trials.
Suzanne McDonald is a clinical trial coordinator and has expertise in the application of N-of-1 trials and SCDs in psychology, medicine and digital health. Suzanne is co-chair of the ICN.
JN drafted and revised the paper and is guarantor. PO, JV, RW, LS, JM and SM provided detailed editorial input.
Sources of information
Pubmed.
Data sharing statement
Not applicable.
Declaration of competing interest
Authors declare that the article was prepared in the absence of any financial relationships that could be construed as a conflict of interest. Authors JN and SM have a commercial interest in N-of-1 Hub Pty Ltd consultancy company. N-of-1 Hub Pty Ltd has no commercial or financial relationship with the International Collaborative Network for N-of-1 Trials and Single-Case Designs (ICN).
Acknowledgements
We thank ICN consumer representatives Noel Muller and Kathy Dallest for sharing their valuable insights and helping to shape the ICN and this paper. Noel Muller and Kathy Dallest provided input to this article through reviewing drafts and providing recommendations to strengthen the consumer perspective and for identifying ways that SCDs can lead to beneficial outcomes for consumers' self-management and self-advocacy. We also thank Samuel Tan and Dr Tom Harris for assistance with setting up the ICN and Katie Davis, Nick Bell, Tze Phang, Jared Ong, Garrett McDonald-Robson, Alex Fixler, Chiharu Kankanamalage, Daniel Ng, Vilany Le, Vidvi Malik, Stela Untoro and Nikola Poli for administrative assistance with the ICN.
Footnotes
Supplementary data to this article can be found online at https://doi.org/10.1016/j.conctc.2021.100826.
Appendix A. Supplementary data
The following is the Supplementary data to this article:
References
- 1.Gabler N.B., Duan N., Vohra S., Kravitz R.L. N-of-1 trials in the medical literature: a systematic review. Med. Care. 2011 Aug;49(8):761–768. doi: 10.1097/MLR.0b013e318215d90d. [DOI] [PubMed] [Google Scholar]
- 2.Shaffer J.A., Kronish I.M., Falzon L., Cheung Y.K., Davidson K.W. N-of-1 randomized intervention trials in health psychology: a systematic review and methodology critique. Ann. Behav. Med. 2018 Aug 16;52(9):731–742. doi: 10.1093/abm/kax026. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.McDonald S., Quinn F., Vieira R., O'Brien N., White M., Johnston D.W., Sniehotta F.F. The state of the art and future opportunities for using longitudinal n-of-1 methods in health behaviour research: a systematic literature overview. Health Psychol. Rev. 2017;11(4):307–323. doi: 10.1080/17437199.2017.1316672. [DOI] [PubMed] [Google Scholar]
- 4.Smith J.D. Single-case experimental designs: a systematic review of published research and current standards. Psychol. Methods. 2012;17(4):510–550. doi: 10.1037/a0029312. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Punja S., Bukutu C., Shamseer L., Sampson M., Hartling L., Urichuk L., Vohra S. The design, analysis and meta-analysis of N-of-1 trials: a tapestry of heterogeneity. J. Clin. Epidemiol. 2016 doi: 10.1016/j.jclinepi.2016.03.023. [DOI] [PubMed] [Google Scholar]
- 6.Vohra S., Shamseer L., Sampson M., Bukutu C., Schmid C.H., Tate R., Nikles J., Zucker D.R., Kravitz R., Guyatt G., Altman D.G., Moher D. CENT Group CONSORT extension for reporting N-of-1 trials (CENT) 2015 Statement. J. Clin. Epidemiol. 2016 Aug;76:9–17. doi: 10.1016/j.jclinepi.2015.05.004. [DOI] [PubMed] [Google Scholar]
- 7.Tate R.L., Perdices M., Rosenkoetter U., Shadish W., Vohra S., Barlow D.H., Horner R., Kazdin A., Kratochwill T., McDonald S., Sampson M., Shamseer L., Togher L., Albin R., Backman C., Douglas J., Evans J.J., Gast D., Manolov R., Mitchell G., Nickels L., Nikles J., Ownsworth T., Rose M., Schmid C.H., Wilson B. The single-case reporting guideline in BEhavioural interventions (SCRIBE) 2016 statement. Neuropsychol. Rehabil. 2017 Jan;27(1):1–15. doi: 10.1080/09602011.2016.1190533. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Schork N. Personalized medicine: time for one-person trials. Nature 520, 7549. [DOI] [PubMed]
- 9.Stunnenberg B.C., Raaphorst J., Groenewoud H.M., Statland J.M., Griggs R.C., Woertman W., Stegeman D.F., Timmermans J., Trivedi J., Matthews E., Saris C.G.J., Schouwenberg B.J., Drost G., van Engelen B.G.M., van der Wilt G.J. Effect of mexiletine on muscle stiffness in patients with nondystrophic myotonia evaluated using aggregated N-of-1 trials. JAMA. 2018 Dec 11;320(22):2344–2353. doi: 10.1001/jama.2018.18020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Wood F.A., Howard J.P., Finegold J.A., Nowbar A.N., Thompson D.M., Arnold A.D., Rajkumar C.A., Connolly S., Cegla J., Stride C., Sever P. N-of-1 trial of a statin, placebo, or no treatment to assess side effects. N. Engl. J. Med. 2020;383(22):2182–2184. doi: 10.1056/NEJMc2031173. [DOI] [PubMed] [Google Scholar]
- 11.Tate R.L., Perdices M., Rosenkoetter U., Wakim D., Godbee K., Togher L., McDonald S. Revision of a method quality rating scale for single-case experimental designs and n-of-1 trials: the 15-item Risk of Bias in N-of-1 Trials (RoBiNT) Scale. Neuropsychol. Rehabil. 2013;23(5):619–638. doi: 10.1080/09602011.2013.824383. [DOI] [PubMed] [Google Scholar]
- 12.Kravitz R., Duan N., Niedzinski E., Hay M.C., Subramanian S., Weisner T.S. What ever happened to N-of-1 trials? Insiders' perspectives and a look to the future. Milbank Q. 2008 Dec;86(4):533–555. doi: 10.1111/j.1468-0009.2008.00533.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Smith J., Yelland M., Del Mar C. Single patient open trials. In: Jane, editor. The Essential Guide to N-Of-1 Trials in Health. Springer; Dordecht, Netherlands: 2015. pp. 19–41. [Google Scholar]
- 14.Blackston J.W., Chapple A.G., McGree J.M., McDonald S., Nikles J. Comparison of aggregated N-of-1 trials with parallel and crossover randomized controlled trials using simulation studies. Healthcare. 2019 Nov 6;7(4):E137. doi: 10.3390/healthcare7040137. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Dukes W.F. N = 1. Psychol. Bull. 1965;64:74–79. [PubMed] [Google Scholar]
- 16.Dougher M.J. Murray Sidman’s contributions to clinical behavior analysis. J. Exp. Anal. Behav. 2021 Jan;115(1):36–43. doi: 10.1002/jeab.644. PMID: 33185279. [DOI] [PubMed] [Google Scholar]
- 17.Mirza R.D., Punja S., Vohra S., Guyatt G. The history and development of N-of-1 trials. J. R. Soc. Med. 2017 Aug;110(8):330–340. doi: 10.1177/0141076817721131. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Hogben L., Sim M. The self-controlled and self-recorded clinical trial for low-grade morbidity. Br. J. Prev. Soc. Med. 1953 Oct;7(4):163–179. doi: 10.1136/jech.7.4.163. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Cushny A.R., Peebles A.R. The action of optical isomers. II. Hyoscines. J Physiology. 1905;32:501–510. doi: 10.1113/jphysiol.1905.sp001097. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Stoll S. JLL Bulletin: Commentaries on the History of Treatment Evaluation. 2011. Paul Martini's methodology of therapeutic investigation. [Google Scholar]
- 21.Baskerville J.C., Toogood J.H., Mazza J., Jennings B. Clinical trials designed to evaluate therapeutic preferences. Stat. Med. 1984;3:45–55. doi: 10.1002/sim.4780030107. [DOI] [PubMed] [Google Scholar]
- 22.Guyatt G., Sackett D., Taylor D.W., Ghong J., Roberts R., Pugsley S. Determining optimal therapy – randomized trials in individual patients. N. Engl. J. Med. 1986;314:889–892. doi: 10.1056/NEJM198604033141406. [DOI] [PubMed] [Google Scholar]
- 23.Hurtado-Parrado C., López-López W. Single-case research methods: history and suitability for a psychological science in need of alternatives. Integr. Psychol. Behav. Sci. 2015;49:323–349. doi: 10.1007/s12124-014-9290-2. [DOI] [PubMed] [Google Scholar]
- 24.Rogers Everett M. Free Press; Glencoe: 1962. Diffusion of Innovations. [Google Scholar]
- 25.Nikles J., Mitchell G., editors. The Essential Guide to N-Of-1 Trials in Health. ” Springer; 2015. [DOI] [Google Scholar]
- 26.Health Canada Guidance for Clinical trials for natural health products. 2016. https://www.canada.ca/en/health-canada/services/drugs-health-products/natural-non-prescription/legislation-guidelines/guidance-documents/clinical-trials.html#a1.2.3
- 27.Medical Research Council Developing and evaluating complex interventions. 2000. https://mrc.ukri.org/documents/pdf/complex-interventions-guidance/
- 28.Woodcock J., Marks P. Drug regulation in the Era of individualized therapies. N. Engl. J. Med. 2019;381:1678–1680. doi: 10.1056/NEJMe1911295. [DOI] [PubMed] [Google Scholar]
- 29.European Medicines Agency (EMA) Guideline on clinical trials in small populations. https://www.ema.europa.eu/en/documents/scientific-guideline/guideline-clinical-trials-small-populations_en.pdf
- 30.National Pharmaceutical Council . In: Making Informed Decisions: Assessing the Strengths and Weaknesses of Study Designs and Analytic Methods for Comparative Effectiveness Research. A Briefing Document for Stakeholders. Velengtas P., editor. 2017. p. 6.https://npcnow.org/system/files/research/download/experimental_nonexperimental_study_final.pdf [Google Scholar]
- 31.Müller A., Brands M., van de Ven P., Roes K., Cornel M., van Karnebeek C. Systematic review of N-of-1 studies in rare genetic neurodevelopmental disorders: the power of 1. Neurology. 2021 Mar 16;96(11):529–540. doi: 10.1212/WNL.0000000000011597. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Warwickshire North Clinical Commissioning Group Commissioning policy: experimental and unproven treatments. July 2015. https://www.warwickshirenorthccg.nhs.uk/mf.ashx?ID=6953bc79-b939-4e17-a02b-2ed07bf16e7d
- 33.Faltinsen, E., Zwi, M., Castells, X et al. Updated 2018 NICE guideline on pharmacological treatments for people with ADHD: a critical look. BMJ Evidence-Based Medicine 24(3), pp. 99-102. [DOI] [PubMed]
- 34.Vohra S., Shamseer L., Sampson M., Bukutu C., Tate R., Nikles J., Kravitz R., Guyatt G., Altman D. Moher D for the CENT group. CONSORT statement: an extension for N-of-1 trials (CENT) BMJ. 2015;350:h1738. doi: 10.1136/bmj.h1738. [DOI] [PubMed] [Google Scholar]
- 35.Curro F.A., Robbins D.A., Naftolin F., Grill A.C., Vena Don, Terracio L. Person-centric Clinical trials: defining the N-of-1 clinical trial utilizing a practice-based translational network. Clin. Invest. 2015;5(2):145–159. doi: 10.4155/cli.14.126. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Duggan C.M. Managing ADHD in general practice. N of 1 trials can help! Aust. Fam. Physician. 2000;12:1205–1209. [PubMed] [Google Scholar]
- 37.Herrett E., StatinWISE Trial Group Statin treatment and muscle symptoms: series of andomised, placebo controlled n-of-1 trials. BMJ. 2021 Feb 24:372. doi: 10.1136/bmj.n135. n135. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Simons L.E. 2020. Avoid or Engage? Outcomes of Graded Exposure in Youth with Chronic Pain Using a Sequential Replicated Single-Case Randomized Design. Pain. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Statland J.M., Bundy B.N., Wang Y. Consortium for Clinical Investigation of Neurologic Channelopathies. Mexiletine for symptoms and signs of myotonia in nondystrophic myotonia: a randomized controlled trial. JAMA. 2012;308(13):1357–1365. doi: 10.1001/jama.2012.12607. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Roustit M., Giai J., Gaget O., Khouri C., Mouhib M., Lotito A. On demand sildenafil as a treatment for Raynaud phenomenon: a series of n-of-1 trials. Ann. Intern. Med. 2018 Nov 20;169(10):694–703. doi: 10.7326/M18-0517. [DOI] [PubMed] [Google Scholar]
- 41.Mitchell G. The effect of methylphenidate on fatigue in advanced cancer: an aggregated N-of-1 trial. J. Pain Symptom Manag. 2015;50(3):28996. doi: 10.1016/j.jpainsymman.2015.03.009. [DOI] [PubMed] [Google Scholar]
- 42.Yelland M. N-of-1 randomized trials to assess the efficacy of gabapentin for chronic neuropathic pain. Pain Med. 2009;10(4):754–761. doi: 10.1111/j.1526-4637.2009.00615.x. [DOI] [PubMed] [Google Scholar]
- 43.Stulc T. Statin intolerance: the clinician's perspective. Curr. Atherosclerosis Rep. 2015;17 doi: 10.1007/s11883-015-0552-3. 69-69. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Finegold J.A. What proportion of symptomatic side effects in patients taking statins are genuinely caused by drug? Systematic review of randomized placebocontrolled trials to aid individual patient choice. Eur J Prev Card. 2014;21:464–474. doi: 10.1177/2047487314525531. [DOI] [PubMed] [Google Scholar]
- 45.Orloff J., Touhamy S., Truong W., Casper A., Shukla A., Ige L., Flory J.H. Trial of restarting and tolerating metformin (TreatMet) Diabetes Obes. Metabol. 2020 Nov;22(11):2189–2192. doi: 10.1111/dom.14137. [DOI] [PubMed] [Google Scholar]
- 46.Nikles J. An N-of-1 trial service in clinical practice: testing the effectiveness of stimulants for attention-deficit/hyperactivity disorder. Pediatrics. 2006;117(6):2040–2046. doi: 10.1542/peds.2005-1328. [DOI] [PubMed] [Google Scholar]
- 47.Nikles C.J. Long-term changes in management following N-of-1 trials of stimulants in attention-deficit/hyperactivity disorder. Eur. J. Clin. Pharmacol. 2007 Nov;63(11):985–989. doi: 10.1007/s00228-007-0361-x. [DOI] [PubMed] [Google Scholar]
- 48.Yelland M.J. Celecoxib compared to sustained-release paracetamol for osteoarthritis: a series of N-of-1 trials. Rheumatology. 2007 Jan;46(1):135–140. doi: 10.1093/rheumatology/kel195. [DOI] [PubMed] [Google Scholar]
- 49.Nikles J. Do Individualised Medication Effectiveness Tests (N-of-1 trials) change clinical decisions about which drug to use for osteoarthritis and chronic pain? Am J Ther. Jan-Feb 2005;12(1):92–97. doi: 10.1097/00045391-200501000-00012. [DOI] [PubMed] [Google Scholar]
- 50.Nikles J. Aggregating single patient (n-of-1) trials in populations where recruitment and retention was difficult: the case of palliative care. J. Clin. Epidemiol. 2011 May;64(5):471–480. doi: 10.1016/j.jclinepi.2010.05.009. [DOI] [PubMed] [Google Scholar]
- 51.Nikles J. Aggregated n-of-1 trials of CNS stimulants versus placebo for paediatric traumatic brain injury - a pilot study. Trials. 2014;15 doi: 10.1186/1745-6215-15-54. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Nikles J. A series of n-of-1 trials of stimulants in brain injured children. NeuroRehabilitation. 2017;40(1):11–21. doi: 10.3233/NRE-161386. [DOI] [PubMed] [Google Scholar]
- 53.Yelland M. Are N-of-1 trials an economically viable option to improve access to selected high cost drugs? – the Australian experience. Value Health. Jan-Feb 2008;11(1):97–109. doi: 10.1111/j.1524-4733.2007.00218.x. [DOI] [PubMed] [Google Scholar]
- 54.Scuffham P. Using N-of-1 trials to improve patient management and save costs. J. Gen. Intern. Med. 2010 doi: 10.1007/s11606-010-1352-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55.Nikles C.J. Using n-of-1 trials as a clinical tool to improve prescribing. Br. J. Gen. Pract. 2005 Mar 1;55(512):175–180. [PMC free article] [PubMed] [Google Scholar]
- 56.U.S Food & Drug Administration Framework for FDA's real-world evidence program. 2018. https://www.fda.gov/media/120060/download
- 57.Krasny-Pacini A., Evans J. Single-case experimental designs to assess intervention effectiveness in rehabilitation: a practical guide. Ann. Phys. Rehabil. Med. 2018;61(3):164–179. doi: 10.1016/j.rehab.2017.12.002. [DOI] [PubMed] [Google Scholar]
- 58.Levin J.R., Ferron J.M. Different randomized multiple-baseline models for different situations: a practical guide for single-case intervention researchers. J. Sch. Psychol. 2021;86:169–177. doi: 10.1016/j.jsp.2021.03.003. [DOI] [PubMed] [Google Scholar]
- 59.Vieira R., McDonald S., Araújo-Soares V., Sniehotta F.F., Henderson R. Dynamic modelling of n-of-1 data: powerful and flexible data analytics applied to individualised studies. Health Psychol. Rev. 2017;11(3):222–234. doi: 10.1080/17437199.2017.1343680. [DOI] [PubMed] [Google Scholar]
- 60.McDonald S., Vieira R., Johnston D.W. Analysing N-of-1 observational data in health psychology and behavioural medicine: a 10-step SPSS tutorial for beginners. Health.Psychol. Behav. Med. 2020;8(1):32–54. doi: 10.1080/21642850.2019.1711096. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61.Manolov R., Moeyaert M. How can single-case data be analyzed? Software resources, tutorial, and reflections on analysis. Behav. Modif. 2017;41(2):179–228. doi: 10.1177/0145445516664307. [DOI] [PubMed] [Google Scholar]
- 62.Vlaeyen J.W.S., Wicksell R.K., Simons L.E., Gentili C., Kumar De T., Tate R.L. From boulder to Stockholm in 70 Years: single case experimental designs in clinical research. Psychol. Rec. 2020;70:659–670. [Google Scholar]
- 63.McDonald Suzanne, Jane Nikles . Healthcare; Basel: 2021. N-of-1 Trials in Healthcare.https://www.mdpi.com/journal/healthcare/special_issues/N_of_1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 64.Nikles J., Daza E.J., McDonald S., Hekler E., Schork N. Frontiers in Computer Science. 2021. Creating evidence from real world patient digital data.https://www.frontiersin.org/research-topics/10089/creating-evidence-from-real-world-patient-digital-data [Google Scholar]
- 65.Margolis A., Giuliano C. Making the switch: from case studies to Nof-1 Trials. Epilepsy Behav Rep. 2019;12:100336. doi: 10.1016/j.ebr.2019.100336. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 66.Percha B., Baskerville E.G., Johnson M., Dudley J.T., Zimmerman N. Designing robust N-of-1 studies for precision medicine: simulation study and design recommendations. J. Med. Internet Res. 2019 Apr;21(4) doi: 10.2196/12641. e12641. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 67.McDonald S., Koudmani Y., Nguyen T., Ralph N., Nikles J. Perspectives on N-of-1 studies in health care: a systematic review and meta-synthesis of the literature. PROSPERO: CRD42020173473. 2020. https://www.crd.york.ac.uk/PROSPERO/display_record.php?RecordID=173473
- 68.McDonald S., Niemeijer M., Ralph N., Nikles J. Open Science Framework. 2021. Exploring stakeholders' perspectives about N-of-1 clinical trials and single-case designs in healthcare.https://osf.io/sb4qf [Google Scholar]
- 69.Cen R., Hussain A., Pak K.J., Mitchell G., Nikles J., Gaudreau S., Bazzano L.A., Breault J.L. Do N-of-1 trials need IRB review? J Empir Res Hum Res Ethics. 2016 Jul;11(3):250–255. doi: 10.1177/1556264616662560. [DOI] [PubMed] [Google Scholar]
- 70.Hesselink J., Kopsky D., Bhaskar A. Ethical justification of single-blind and double-blind placebo-controlled response tests in neuropathic pain and N-of-1 treatment paradigm in clinical settings. J. Pain Res. 2019;12:345–352. doi: 10.2147/JPR.S180792. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 71.Stunnenberg B.C. Healthcare (Basel) 2020. N-of-1 trials: evidence-based clinical care or medical research that requires IRB approval? A practical flowchart based on an ethical framework. PMID: 32120865. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 72.Bellgard M., Snelling T., McGree J.M. RD-RAP: beyond rare disease patient registries, devising a comprehensive data and analytic framework. Orphanet J. Rare Dis. 2019;14:176. doi: 10.1186/s13023-019-1139-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 73.Nikles J., Daza E.J., McDonald S., Hekler E., Schork N. Editorial: creating evidence from real world patient digital data. Front. Comput. Sci. 24 February 2021 doi: 10.3389/fcomp.2020.636996. [DOI] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.



