Privacy rights and improving knowledge are not hierarchical needs: data protection and good epidemiologic standard (DP_GOES) checklist for retrospective observational studies using secondary data

Abstract

Starting with the Nuremberg Code in 1947, several guidelines were developed to formulate rules to guide research on humans and safeguard the rights and well-being of subjects participating in clinical research. In recent years, retrospective observational studies based on disease and drug registries, surveillance systems, hospital-based data lakes and platforms, and unstructured data have gained progressively greater attention in the medical literature. Although several guidelines and checklists are currently available to develop and evaluate a protocol for observational studies, issues concerning ethical considerations, data protection and data access have been often ignored. We propose the Data Protection and Good Epidemiologic Standard (DP_GOES) checklist for the development and evaluation of the protocol of observational, retrospective studies based on secondary data. The checklist is divided into four parts, 9 sections and 68 items, and should help to verify whether the study protocol respects the constraints of the regulatory requirements and provisions of data protection authorities, while ensuring that the study may generate robust evidence potentially useful to promote health, supplying more effective healthcare, and guaranteeing system sustainability. The DP_GOES checklist represents a novel and integrative contribution, as it systematically combines epidemiological research standards with data protection principles. Its practical value lies in offering a structured and operational tool that supports both researchers and evaluators in conducting and assessing retrospective observational studies based on secondary data in a rigorous, transparent, and ethically accepted manner.

Background

After World War II, a trial was conducted on 23 Nazi doctors and scientists at Nuremberg for the murder of concentration camp inmates who were used as research subjects. The trial brought to light the torture that was conducted, and in 1947 the judgement culminated in the formulation of rules to guide research on humans, known as the Nuremberg Code [1].

Subsequent violations of these principles, such as the 1956 Willowbrook Hepatitis Study [2] and the 1964 Jewish Chronic Disease Study [3] highlighted the ongoing need for ethical oversight. In response, the World Medical Association (WMA) General Assembly introduced the Declaration of Helsinki in 1964 [4], setting guidelines to protect the rights and welfare of participants in clinical research, with the latest amendment approved in 2024 [4]. According to the Helsinki Declaration, in any research involving humans, it must be ensured that all participants are adequately informed of the expected benefits and potential risks associated with the study, and that the research conforms to universally accepted scientific principles.

To unify the many overlapping guidelines issued worldwide, the International Conference for Harmonisation (ICH) introduced the Good Clinical Practice (GCP) in 1996 [5], providing an international standard for the ethical and scientific conduct of trials involving human subjects. According to ICH-GCP, a trial shall only be initiated once an ethics committee concludes that suitable informed consent was submitted and offers guarantees on study quality [6]. These guidelines have been adopted into law in several countries [7].

In summary, recent history explains much of the natural and linear evolution of our ability to generate robust knowledge from studies requiring the direct involvement of human beings while respecting their individual rights [8]. However, as history advances, the paradigm of conventional clinical research seems to be faltering.

Several basilar questions that need to be answered to improve the value of healthcare cannot always be fully investigated (or only have partial inquiries) by primary studies involving humans, i.e., by randomised controlled trials (RCTs) and prospective (or cross-sectional) observational studies. Three topics will help us understand the foundation of these questions.

First, RCT and observational investigations allow to generate knowledge about how “average” patients react to therapy, but not to who is most likely to receive benefits from it. Precision medicine [9] and personalized healthcare [10], the new waves of modern medicine, need to identify patients who will benefit from targeted therapies and to tailor the healthcare pathway strategy based on individual biomolecular, clinical, demographic, and socioeconomic features. Second, evidence-based guidelines cannot be translated into health benefits, regardless of being suitably implemented in clinical practice [11]. However, a gap exists between evidence-based recommendations and clinical practice which must be filled with additional knowledge. Third, due to several reasons, most medical actions are not supported by robust evidence. Frail patients are usually excluded from RCTs, so that our knowledge is based on patients who do not reflect those seen in clinical practice [12]. RCTs can be limited to duration of only a few years, so that we are not able to predict the effect of treatments lasting several years, i.e., potentially lifelong therapies [13]. Finally, the early approval from regulatory agencies aimed at making drugs with innovative mechanisms of action quickly accessible [14], implies that available evidence is increasingly obtained through simplified research approaches, e.g. from single-arm study designs and surrogate endpoints. The need to manage the unknown [15] and effectively enforce a total product lifecycle, i.e., to generate evidence after accelerated/conditional approval of new therapeutics, must be considered a priority knowledge requirement.

In summary, stratifying the population to tailor interventions to health needs, monitoring adherence with evidence-based recommendations, and assessing the impact of healthcare in the real world, are new, compelling knowledge needs that conventional clinical studies can only partially fulfil.

In recent years, observational studies in which both healthcare exposure and the resulting outcomes have already occurred when the study started (known as retrospective observational studies [16, 17]) using data collected for purposes other than medical research, which is fed with secondary data [18], have gained progressively greater attention in the medical literature and have become a crucial avenue to bridge these evidence gaps. This approach allows insight into the best ways to treat the patients within the healthcare system to generate the required data [19]; a goal which can significantly contribute if available real-world data correctly identifies between-patient heterogeneity in biomolecular, clinical, demographic, and socioeconomic features, as well as in experienced healthcare pathways and clinical and cost outcomes experienced by patients.

Two basic questions should be considered to understand the potential of the approach. First, do we have enough data to investigate clinical practice? Health digitisation increasingly involves both the management of patients (through electronic medical/health records [20]), and health system (through electronically stored healthcare utilisation databases to supply payments to health service providers [21]). Furthermore, disease and drug registries [22, 23], surveillance systems [24], and hospital-based data lakes [25] and platforms [26] provide increased availability of electronically stored health data. Unstructured data regarding images (diagnostic tools [27]), texts (medical records [28]), biomolecular sequences [29], signals generated by wearable devices [30, 31], and search engines [32], among others, constitute a wealth of potentially informative electronically recorded data.

Second, are we entitled to use personal data for research purposes? Both the above quoted principles of individual right and community altruism should be recalled and adapted to the new research setting. Individual right to privacy must always be suitably guaranteed. A social licence and public mandate are essential components of big data research that can provide societal benefit, address the concerns of participants, and ensure data protection [33]. Conversely, the principle of community altruism retains the same meaning already expressed for medical research, conventional and otherwise. That is, we do research to further our understanding and improve the treatment of future patients. To do this, we must guarantee that reliable evidence is generated from patient data. However, as we are moving from sole reliance on conventional, randomised trials to inclusion of observational, retrospective studies, we must consider that potential biases that are always lurking, and the considerably weaker guarantee [34]; the generation of credible evidence is not guaranteed, even when good quality real-world big data is available and accessible.

The principles of privacy rights and community altruism - the ethical and scientific quality standard which must be promised to study participants - translate into parallel ways of approaching the planning of a retrospective observational study, based on secondary data. On one hand, as far as the ethical quality standard is of concern, the field has long been debated and explored mainly by data scientists skilled in data protection [35]. The Five Safes framework is a set of principles which enable provision of safe research access to data [36]. Originating in the 2010s, this framework has become best practice in data protection while fulfilling the demands of open science and transparency. Briefly, safe data (data is treated to protect any confidentiality concerns), safe people (researchers are trained and authorised to use data safely), safe settings (a secure environment to preventing unauthorised access), safe outputs (screened and approved outputs that are non-disclosive), and safe projects (research projects approved by data owners for the public good) are the five pillars on which the framework is built to enable researchers to access and use datasets in a secure and responsible way [37]. Interestingly, a safe project refers to the “legal, moral, and ethical considerations surrounding use of the data” [38] without addressing how the naïve use of high-quality data can lead to misleading findings that may pose risks to health if not properly handled [39]. It is almost as if, after 70 years of good clinical practice, the generation of credible (unbiased as far as possibly) findings that are useful to improve patient treatment in the future is no longer a concern that the study protocol needs to address!

On the other hand, as far as the scientific quality standard is of concern, the field has also been long debated and explored. The amount of attention placed by epidemiologists on this topic has generated several guidelines, checklists, and recommendations [40–47] to develop and evaluate a protocol for observational studies, some of which are dedicated to the field of pharmacoepidemiology [48–51]. However, issues such as ethical considerations and data access within observational studies are too often ignored [52]; the safe use of data is no longer of great enough concern.

To summarize, existing epidemiological guidelines often focus on methodological rigor but provide limited guidance on data protection, while data protection frameworks emphasize legal and ethical safeguards without considering scientific quality. We propose the DP_GOES checklist merging Data Protection (privacy-by-design) and GOod Epidemiologic Standard (scientific issues), specifically developed to bridge this gap and intended to supportthe development and evaluation of the protocol of observational, retrospective studies based on secondary data.

Methods

Extensive research literature was performed in order to identify the existing guidelines, checklists, and recommendations to develop and to evaluate a protocol for observational studies. We searched PubMed to find published guidelines, checklists, and recommendations to develop and to evaluate a protocol for observational studies. We used the following research string: (“checklist“[Title/Abstract] OR “recommendation“[Title/Abstract] OR “guideline“[Title/Abstract]) AND (“protocol“[Title/Abstract]) AND (“observational“[Title/Abstract]), and we filtered the results based on the English language. At the time the research was performed (i.e. December 2023), we obtained more than 500 articles, which were initially screened by MF via title and abstract, and then via the full text. Second, in order to retrieve best practices and guidelines not indexed on PubMed, such as those issued by the FDA and the EMA, we performed a hand-search on Google, using the following search terms: “guidelines”, “recommendations”, and “checklist”, combined with “observational” and “protocol”. Overall, 13 items were selected and cited in the main text manuscript. The results of the research were shared and discussed with the other authors of this article.

Starting from the research literature, the authors of this manuscript identified essential items deemed important to be included in the checklist. The heterogeneity of the expert team ensured that multiple perspectives were incorporated throughout the development of the checklist. For example, GC and OL contributed regulatory and compliance insights, MF provided an academic and methodological viewpoint, and MG represented the patient perspective. This multidisciplinary composition allowed the team to consider the checklist items from scientific, ethical, regulatory, and patient-centered angles, enhancing both the relevance and applicability of the final tool. Each author independently prepared a preliminary list of items in advance, based both on specific elements already addressed in the existing literature and on areas that were not previously covered. Then, each draft checklist was discussed by all authors, who met on a 1-day workshop, which agreed on the definitive items to be included in the checklist. Consensus was reached through discussions among the expert team. For each item, differing opinions were carefully debated, and decisions were made by considering the available evidence, relevance to both scientific rigor and data protection, and practical applicability. Items for which initial agreement was not achieved were revisited until a majority agreement was reached, ensuring that all perspectives were adequately considered before final inclusion in the checklist. Finally, the authors submitted via e-mail the checklist, to be validated by different categories of stakeholders, i.e., leading experts in observational research, government agency executives, and patient association leaders (see the Acknowledgements section for a list of contributors). In detail, five stakeholders returned their comments and suggestions via e-mails within one month, and the authors met again in person in order to implement their comments. Specifically, we obtained 14 feedbacks on section A (Preliminary remarks), 28 on section B (Study architecture), 12 on section C (Data sources and protection) and 4 on Section D (Dissemination). Finally, during an on-line plenary meeting with both the authors and the stakeholders, the checklist was reviewed and discussed item by item. Any further disputed items were debated openly. Decisions were made based on consensus, with items being revised in real time when necessary to incorporate suggestions from all participants. The process ensured that every item was critically evaluated from multiple perspectives, before final agreement was reached.

Results

The checklist is divided into four parts, nine sections, and 68 items that we consider fundamental for a good development of the protocol of observational, retrospective studies based on secondary data (Table 1).

Table 1.

The DP-goes checklist

	A – Preliminary remarks	Fully	Partially	None	Not pertinent	Section (page)
1.	Starting items
1.1.	Has there been a review by an ethics committee?	□		□
1.2.	Has there been any consultation with patient representatives / a patient advisory board in the context of the development and implementation of the study?	□		□
1.3.	Are details of the study registration clearly reported (site, date, code) in such a way to guarantee public availability of the protocol?	□	□	□
1.4.	Are study fundings clearly declared?	□	□	□
1.5.	Is the study promoter and sponsor clearly reported?	□	□	□
1.6.	Are members of the research staff listed and their roles and responsibility clearly reported?	□	□	□
1.7.	Does the research title reflect the PICO (Population, Intervention, Comparator, Outcome) approach?	□	□	□
2.	Background
2.1.	Is the available knowledge on the topic the study intends to address adequately described and well documented with available and up-to-date literature?	□	□	□
2.2.	Is the knowledge that the study will add (not available or, although available, inconsistent and/or unsuitable for the study setting) clearly reported?	□	□	□
2.3.	Is it clear to what body and to whom the knowledge/evidence of the study will be reported? Are stakeholders informed of this?	□	□	□
3.	Aims
3.1.	Is the main objective of the study (specific question the study intends to answer and the hypothesis the study intends to test) clearly stated?	□	□	□
3.2.	Are secondary objectives of the study clearly stated? Is it explicit for each objective whether it refers to hypotheses to be tested or is it an exploratory objective?	□	□	□
3.3.	Are exploratory objectives of the study clearly stated?	□	□	□

	B - Study architecture	Fully	Partially	None	Not pertinent	Section (page)
4.	Study design
4.1.	Is the study design to answer the main objective clearly reported (e.g., cohort, nested case-control, case-cohort, case-crossover, self-controlled case-series) along with the rationale for choosing it?	□	□	□
	Population
4.2.	Is the target population clearly defined?	□	□	□
4.3.	Is the recruitment period clearly stated?	□	□	□
4.4.	Are the eligibility criteria clearly defined and justified?	□	□	□
4.5.	Are the exclusion criteria clearly defined and justified?	□	□	□
4.6.	Does the study population adequately reflect the population that should ideally be reached (see 4.2)?	□	□	□
	Follow-up
4.7.	Is the starting point of the follow-up clearly defined?	□	□	□	□
4.8.	Are the reasons for censoring clearly reported?	□	□	□	□
4.9.	Does the planned duration of follow-up adequately reflect the knowledge available (see 3.1) and that which the study will add (see 4.1, 4.2, and 4.3)?	□	□	□	□
	Healthcare exposure
4.10.	Is the exposure of interest clearly defined and consistent with the clinical/conceptual basis of the research question (see 2.1) as well as with stakeholder needs (see 2.3)?	□	□	□	□
4.11.	Is the time window for exposure assessment clearly reported and consistent with the clinical/conceptual basis of the research question (see 2.1)?	□	□	□	□
4.12.	Is any justification provided for the validity of the exposure measure or the range within which sensitivity, specificity, and/or positive and negative predictive values may vary?	□	□	□	□
4.13.	Is the direction of differential and non-differential exposure misclassification sources discussed and how these sources might influence the acceptance or rejection of the null hypothesis?	□	□	□	□
	Comparator
4.14.	Is the exposure comparator clearly defined (e.g., exposed to the same therapy but with lower intensity, individuals exposed to a different therapy with the same indication as the one under study, unexposed)?	□	□	□	□
4.15.	Does the comparator fit the needs of stakeholders (see 2.3)?	□	□	□	□
	Covariates
4.16.	Are the covariates of interest clearly defined?	□	□	□	□
4.17.	Is the time window for covariate assessment clearly reported?	□	□	□	□
4.18.	Is any justification provided for the validity of the covariates measure or the range within which sensitivity, specificity, and/or positive and negative predictive values may vary?	□	□	□	□
4.19.	Is the direction of differential and non-differential misclassification sources of covariates discussed and how these sources might influence the acceptance or rejection of the null hypothesis?	□	□	□	□
	Clinical outcome(s)
4.20.	Are the clinical outcomes of interest clearly defined and consistent with the clinical/conceptual basis of the research question (see 2.1) as well as with stakeholder needs (see 2.3) and the study aims (see 3.1–3.3)?	□	□	□	□
4.21.	Is the time window between start of exposure and onset of clinical outcome justified by the clinical/conceptual basis of the research question (see Item 2.1)?	□	□	□	□
4.22.	Is any justification provided for the validity of the clinical outcome measure or the range within which sensitivity, specificity, and/or positive and negative predictive values may vary?	□	□	□	□
4.23.	Is the direction of differential and non-differential outcome misclassification sources discussed and how do these sources influence the acceptance or rejection of the null hypothesis?	□	□	□	□
	Costs
4.24.	Do the costs, if provided in the protocol, refer to medical care only (direct costs) or also to indirect costs? Which ones?	□	□	□	□
4.25.	Are the start, endpoint, and duration of follow-up for cost measurement clearly defined?	□	□	□	□
5.	Data analyses
	Definition of the study variables
5.1.	Are the independent variables clearly defined, indicating the following for each item:
	✓the metric (dichotomous, ordinal, continuous, other)	□	□	□	□
	✓ when they are measured (and whether time-fixed or time-dependent)	□	□	□	□
	✓ whether they are measured at a higher level than that of the individual (e.g., hospital, physician, region, etc...)	□	□	□	□
	✓ whether they will be considered as (healthcare) exposure, confounders, effect modifiers, or other	□	□	□	□
5.2.	Are the dependent variables clearly defined, and a metric for each of them (continuous or categorical, single, or repeated measure, time to event) clearly indicated?	□	□	□	□
	Descriptive analysis
5.3.	Are the descriptive analyses clearly defined and consistent with the primary and secondary objectives of the study (see Section 2)?	□	□	□	□
	Modelling
5.4.	Does the choice of the functional form of the model consider the objectives of the study (see section 3), the study design (item 4.1), the nature of independent and dependent variables (items 5.1 and 5.2)?	□	□	□	□
5.5.	Are the model’s assumptions clearly stated, including if/how will they be empirically verified, and what alternatives will be used if they are violated?	□	□	□	□
	Sensitivity analysis
5.6.	Is consideration given to changing definitions of exposure, covariates, and outcome to assess how they affect the main results of the study?	□	□	□	□
	Other methods of systematic uncertainty control
5.7.	Are there plans to adopt any strategies to correct, or at least account for, the impact of sources of selection bias?	□	□	□	□
5.8.	Are there plans to adopt any strategies to correct, or at least account for, the impact of sources of exposure misclassification?	□	□	□	□
5.9.	Are there plans to adopt any strategies to correct, or at least account for, the impact of sources of outcome misclassification?	□	□	□	□
	Covariates effect
5.10.	Are there plans to adopt any strategies to correct, or at least account for, the impact of unmeasured confounding?	□	□	□	□
5.11.	Is there a clear and explicit provision for assaying the action of one or more covariates in modifying the effect of treatment exposure on the outcome under study? Are planned assays consistent with previous knowledge (items 2.1 and 2.2) and study objectives (items 3.1–3.3)?	□	□	□	□
5.12.	Is consideration given to the possibility that covariates may act as mediator, collider, or in some other form that explains the causal pattern underlying the process under study? Is there provision for the use of Directed Acyclic Graphs (DAGs) to clarify which models will be assayed? Are planned assays consistent with previous knowledge (items 2.1 and 2.2) and study objectives (items 3.1–3.3)?	□	□	□	□
6.	Study size and power considerations
6.1.	Are the study size considerations adequately reported?	□	□	□	□
6.2.	Is the link to 1.1, 1.2, the main objective of the study (item 3.1), the exposure (4.10–4.13) and main outcome of interest (see 4.20–4.26) clear and explicit?	□	□	□	□
6.3.	Are these considerations consistent with the expected risk at baseline? Is the minimum detectable effect sufficiently relevant from a clinical and/or public health perspective? Is the power of the study adequate?	□	□	□	□
6.4.	Are methods for dealing with the problem of multiple comparisons reported?	□	□	□	□

	C - Data sources and protection	Fully	Partially	None	Not pertinent	Section (page)
7.	Data sources and data linkage
7.1.	Are the data sources for selecting target population (items 4.2–4.6) and measuring exposure (4.10–4.13), covariates (4.16–4.19), clinical outcome(s) (4.20–4.24), and costs (4.25–4.26) of interest clearly reported?	□	□	□
7.2.	Are all data details required by study design (e.g., eligibility and exclusion criteria, date, and time-windows items, censoring information such as death date, see Sections 4 and 5) available from data sources?	□	□	□
7.3.	Are data sources linkable according with the needs of study design (e.g., record linkage between drug prescription and hospital admission databases) to obtain a person-based database to perform the analyses?	□	□	□	□
7.4.	Is the personal identification code unique for each data source?	□	□	□	□
8.	Data protection
8.1.	Have individuals expressed consent for collection and processing of the data?	□	□	□
8.2.	Will protection be undertaken to prevent unauthorized persons from accessing the environment where the data is stored?	□	□	□
8.3.	Will data minimization practices limit individual recognition (e.g., removing personal data that is not useful for the purposes of the study, such as the day and month of the date of birth)?	□	□	□
8.4.	Do the result minimization practices adopted limit recognition of the individuals who generated the data (for example, removing the number of individuals who meet specific conditions if <10)?	□	□	□
8.5.	Are the procedures/clauses that persons who will access the data will have to adhere to (for example, limitation of data transmission to third parties) clearly reported?	□	□	□
8.6.	Is the rationale for the study design and data analysis clearly stated (according to background and aims, see Sections 2 and 3) and is it ensured that the analysis will be carried out to optimise the strength of the resulting evidence (see Sections 4 and 5)?	□	□	□

	D - Dissemination	Fully	Partially	None	Not pertinent	Section (page)
9.	Dissemination, communication, and reporting
9.1.	Has a dissemination and communication plan (policy makers, regulatory authorities, associations of citizens and patients) been developed with key stakeholders?	□	□	□
9.2.	Is it made explicit that the results of the study will be submitted for publication in a peer reviewed scientific journal, possibly open access?	□	□	□
9.3.	Does the protocol make it clear that contact with journalists will be made only after the manuscript has been accepted for publication, and preferably published?	□	□	□	□
9.4.	Are quality assurance and quality control mechanisms put in place ensure complete, accurate, accessible, and interpretable data reporting?	□	□	□

Guidance note: The DP_GOES checklist is designed to support the development, evaluation, and review of protocols for retrospective observational studies using secondary data. It integrates both scientific rigor and ethical/data protection considerations to ensure studies are ethically and methodologically appropriate, and compliant with data protection regulations

The checklist is divided into four parts:

• Part A – Preliminary Remarks: Ethics, transparency, justification, and objectives

• Part B – Study Architecture: Study design, data analysis, and sample size/power considerations

• Part C – Data Sources and Protection: Data adequacy assessment (scientific need) and privacy-by-design (ethical/legal need)

• Part D – Dissemination: Transparent reporting and communication of results to stakeholders

Each item should be evaluated according to whether it is addressed in the protocol:

• Fully: The item is completely addressed and clearly specified

• Partially: The item is mentioned but lacks sufficient detail or clarity; additional information or clarification is required

• Not at all: The item is absent from the protocol and must be explicitly incorporated

The first part (Part A; Preliminary remarks; 13 items) is designed to ensure that the study protocol meets fundamental ethical standards and has a well-defined purpose before proceeding. Specifically, it verifies whether (i) the protocol was reviewed by an ethics committee, and transparency is guaranteed through registration details, promoter, possible sponsor, and research team (Sect. 1); (ii) knowledge gaps on the topic of interest justify the study (Sect. 2), and (iii) objectives are clearly reported (Sect. 3).

Part B (Study architecture; 41 items) is designed to ensure the scientific rigor and validity of the methodology, assessing whether the study design and analysis will generate robust results to inform policy or decision-making. It includes three sections, nominally study design (Sect. 4), data analysis (Sect. 5), and study size and power considerations (Sect. 6). The question: “will the rules with which the study will be carried out allow for sufficiently robust results to guide the policies underlying the research question?” is the implicit purpose of Part B. For example, several sources of systematic uncertainty, such as misclassification (of exposure, covariates, and outcome) and confounding, are widely considered in both the design and analysis, and random uncertainty is considered. As widely discussed above, the scientific quality of the study protocol is intended to be verified by Part B through the study architecture.

The third part (Part C, Data sources and protection; ten items) is entirely devoted to the study data and addresses the adequacy of the study data and the implementation of data protection measures. Having clarified the research question (Sect. 3) and how it is intended to be addressed (Sects. 4 and 5), two questions can be answered: (i) do we have enough (real-world) data to carry out the study in the detail and with the quality and completeness required by the planned design and analysis? (ii) how is data protection guaranteed by the study protocol (that is, how privacy-by-design issues are considered)? These two issues considered in Sects. 7 and 8, respectively. This section illustrates the integrative nature of the checklist by simultaneously addressing both scientific and ethical/legal considerations. It evaluates data adequacy to ensure the study can generate robust and reliable results while also assessing privacy-by-design measures to guarantee compliance with ethical and legal data protection requirements. In this way, this section reinforces the checklist’s unique role in bridging methodological rigor and responsible data governance.

Finally, Part D (Dissemination; four items) ensures that study results are communicated transparently, accurately, and appropriately to all relevant stakeholders. It verifies whether transparency is guaranteed by provision within the protocol of how the results will be disseminated (Sect. 9). For findings that could have a significant impact on public health, there may be legal, as well as ethical requirements, to report the results immediately to the appropriate regulatory authorities and to healthcare professionals and patients concerned. The scientific community should be informed of study results in a timely fashion by publication in the literature and presentations at conferences, workshops, or symposia. Contact with journalists are an optional part of the dissemination process. Authors should make efforts to ensure that the text to be published in the journalistic product is accurate, precise, correct, and understandable for the readers [40].

Each item of the checklist can be rated as fully, partially, or not at all addressed in the study protocol. The “partially” category indicates that the item is considered but some information is incomplete or insufficiently specified and therefore requires clarification. In contrast, the “not at all” category indicates that the item is entirely missing from the protocol and must be explicitly addressed.

Discussion

According to the extensive literature search and the contribution of several stakeholders (academic researchers, patients’ representatives and public health professionals), a set of standards to develop and evaluate the protocol of observational, retrospective studies based on secondary data have been listed. The aim was not to duplicate existing work, but rather to bring together data protection (privacy-by-design) and scientific (good epidemiologic practice) issues in one overarching checklist with a focus on practical implications for researchers.

DP_GOES checklist verifies whether the study protocol respects the constraints of the regulatory requirements and provisions of data protection authorities, while at the same time ensuring that the study may generate robust evidence potentially useful to promote health and better address patients’ needs, supplying more effective healthcare, and guaranteeing system sustainability. Patient involvement is also an important aspect that must be explored in the context of this framework to ensure greater acceptance and awareness of data-sharing and of the outcomes of observational studies to benefit patient care. To the best of our knowledge, DP-GOES is the most complete and exhaustive checklist among those published.

Compared to other lists, the DP_GOES has several distinctive features that need to be acknowledged. First, it specifically focuses on retrospective observational studies based on secondary data, which represent a very particular type of observational study. Second, the checklist emphasizes the validation of the research protocol. Indeed, it begins with “starting items” designed to assess whether both the ethical and methodological aspects have been clearly stated in the research protocol and approved by an ethics committee prior to the start of the study. Third, in addition to evaluating the effectiveness of the intervention under study (i.e. measured by patient-relevant outcomes), the DP_GOES checklist also considers its sustainability for the National Health Service perspective and the costs required to achieve patient outcomes, thus addressing the cost-effectiveness profile within a “value-based” approach [53]. Fourth, and most importantly, an entire section is dedicated to data protection, a dimension not addressed by any other reporting guideline or checklist. Fifth, a concluding section is devoted to the dissemination and communication of study results to institutional stakeholders, the scientific community, and patients. This, too, represents a novel and innovative aspect of our approach. Finally, it is expected that each individual requirement (item) of the checklist can be fully, partially, or not at all convincing in the protocol. In particular, the category “partially” indicates that, although the item has been considered in the research protocol, some information is missing or not fully specified, thus they need to be better clarified. In contrast, the category “Not at all” indicates that no mention was made in the protocol about that item, thus it is required to be clearly taken into account. Taken together, these features highlight several novel dimensions of the DP_GOES checklist that are not covered by other existing checklists, distinguishing our tool from previous reporting guidelines.

From the overall reading of the protocol, and the research question that underlies it, balancing methodological strictness and confidentiality depends on the urgency inherent in the answer to the question. For example, during a particularly aggressive epidemic, and with the availability of an effective vaccine, the urgency with which it is necessary to identify individuals at greatest risk for priority protection, should suggest that privacy protection rules may be somewhat looser. In other words, the legal principle should apply accordingly that, in the event of a competitive conflict between individual rights and public interest, the latter prevails [54]. Of course, the application of the principle requires objective judgement on urgency of the study. Moreover, just because a study takes place in the context of an emergency does not mean that all methodological guidance and considerations should not be considered as mandatory; a lot of observational studies during the COVID-19 pandemic turned out to be biased and useless, with considerable waste of resources.

Both protocol authors and evaluators should be interested in the DP_GOES checklist. The former can verify its completeness in all its aspects related to transparency, epidemiological and statistical methodology, the source of the data and the requirements to ensure security and privacy, and the dissemination of the results, before submitting the protocol for ethical evaluation. The evaluators can verify the criticisms and request clarifications from the authors.

The main obstacle in the diffusion of good practices in this context is the absence of a clear and explicit mandate of the ethics committees in the appraisal of observational, retrospective studies based on secondary data. For example, the legal framework for clinical trials in Europe, Regulation (EU) No 536/2014 covers only RCTs with pharmaceutical products [55], although the European Medicines Agency started a reflection and provided initial guidance on the use of real-world evidence in the regulatory decision-making. As the process has been initiated by regulatory agencies whose exclusive interest is for drugs and devices, a substantial legislative vacuum exists on studies carried out with this methodology; mainly, whether they do not evaluate, or only partially concern, drugs and/or devices. This leaves ample space for “data torture” [56] that has no scientific basis, and which represents real attacks on the ethical principle of community altruism.

Conclusions

In conclusion, the intent of the DP-GOES checklist is not to cripple researchers’ freedom or to impose inflexible rules on the conduct of studies, but rather to streamline efforts to harmonise actions aimed to protect natural persons from the risk of violation of sensitive data (principle of privacy rights), with those aimed at improving knowledge aimed at promoting and protecting health (principle of community altruism).

Authors’ contributions

GC, MG, OL and MF contributed to the conception of the study, independently prepared a draft checklist and approved the final checklist, participated to a 1-day workshop and interacted with stakeholders, both by emails and in an on-line plenary meeting. MF performed the initial research literature. GC coordinated all the activities and wrote the first draft of the paper. All authors read and approved the final manuscript.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Data availability

Not applicable.

Declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

GC received research support from the European Community (EC), the Italian Agency of Drugs (AIFA), and the Italian Ministry for University and Research (MIUR). He took part in a variety of projects that were funded by pharmaceutical companies (i.e. Novartis, GSK, Roche, AMGEN and BMS). He also received honoraria as a member of the advisory board to Roche. The other authors do not report any competing interest.