Table 2.

CODE-EHR framework: best practice checklist to report on the use of structured electronic healthcare records in clinical research

Date of completion:		Study name:
Item	Objective	Framework standards	Minimum information to provide	Lead author acknowledgment
1. Dataset construction and linkage	To provide an understanding of how the structured healthcare data were identified and used.	Minimum: Flow diagram of datasets used in the study, and description of the processes and directionality of any linkage performed, published within the research report or supplementary documents. Preferred: Provided within a pre-published protocol or open access document.	State the source of any datasets used. Comment on how the observed and any missing data were identified and addressed, and the proportion observed for each variable. Provide data on completeness of follow-up. For linked datasets, specify how linkage was performed and the quality of linkage methods.	Choose one from: (1) Minimum standard not met (2) Minimum standard met OR (3) Preferred standard met
2. Data fit for purpose	To ensure transparency with the approach taken, with respect to coding of the structured healthcare data.	Minimum: Clear unambiguous statements on the process of coding in the methods section of the research report. Preferred: Provided within a pre-published protocol or open access document.	Confirm origin, clinical processes, and the purpose of data. Specify coding systems, clinical terminologies, or classification used and their versions, and any manipulation of the coded data. Provide detail on quality assessment for data capture. Outline potential sources of bias.	Choose one from: (1) Minimum standard not met (2) Minimum standard met OR (3) Preferred standard met
3. Disease and outcome definitions	To fully detail how conditions AND outcome events were defined, allowing other researchers to identify errors and repeat the process in other datasets.	Minimum: State what codes were used to define diseases, treatments, conditions, and outcomes prior to statistical analysis, including those relating to patient identification, therapy, procedures, comorbidities, and components of any composite endpoints. Preferred: Provided within a pre-published protocol or open access document prior to statistical analysis.	Detailed lists of codes used for each aspect of the study. Date of publication and access details for the coding manual (please add to box below). Provide definitions, implementation logic and validation of any phenotyping algorithms used. Specify any processes used to validate the coding scheme or reference to prior work.	Choose one from: (1) Minimum standard not met (2) Minimum standard met OR (3) Preferred standard met
4. Analysis	To fully detail how outcome events were analysed and allow independent assessment of the authenticity of study findings.	Minimum: Describe the process used to analyse study outcomes, including statistical methods and use of any machine learning or algorithmic approaches. Preferred: Provide a statistical analysis plan as a supplementary file, locked before analyses commencing.	Provide details on all statistical methods used. Provide links to any machine code or algorithms used in the analysis, preferably as open source. Specify the processes of testing assumptions, assessing model fit and any internal validation. Specify how generalisability of results was assessed, the replication of findings in other datasets, or any external validation.	Choose one from: (1) Minimum standard not met (2) Minimum standard met OR (3) Preferred standard met
5. Ethics and governance	To provide patients, who might or might not have given consent, and regulatory authorities the ability to interrogate the security and provenance of the data.	Minimum: Clear unambiguous statements on how the principles of Good Clinical Practice and Data Protection will be/were met, provided in the methods section of the research report. Preferred: Provided within a pre-published protocol or open access document with evidence of patient and public engagement.	State how informed consent was acquired, or governance if no patient consent. Specify how data privacy was protected in the collection and storage of data. Detail what steps were taken for patient and public involvement in the research study. Provide information on where anonymised source data or code can be obtained for verification and further research.	Choose one from: (1) Minimum standard not met (2) Minimum standard met OR (3) Preferred standard met
6. Coding manual	DOI of publication or website address: Date published:
7. Comments
8. Summary declaration	Choose one from: One or more minimum standards not met OR All minimum standards met Number of preferred standards met: / 5

Directions for use:

Research team: To complete the checklist, authors will need to consider these points during the design of the research to ensure that coding protocols and coding manuals are pre-published. Where applicable, it is advisable that all five minimum standards are met for an individual research study, whether observational or a controlled trial. If any component is not applicable to the study, the corresponding author can indicate why this is the case in the comment box. This checklist can accompany the article as a supplementary file on submission to the journal, with the ability for readers to review responses. A comment on the meeting of standards in the text of the method section is suggested, eg; “this study meets all five of the CODE-EHR minimum framework standards for the use of structured healthcare data in clinical research, with two out of five standards meeting preferred criteria https://doi.org/10.1093/eurheartj/ehac426”; OR “this study meets four out of five of the CODE-EHR minimum framework standards for the use of structured healthcare data in clinical research; one of the five minimum standards was not met as coding schemes were not specified prior to analysis https://doi.org/10.1093/eurheartj/ehac426.” Note, easy to complete form versions of this checklist are available in appendix 4 (word version) and appendix 5 (pdf version), and also at https://www.escardio.org/bigdata.

Research appraisers (patients, clinicians, regulators, guideline task forces): Where applicable, it is advisable that all five minimum standards are met for the research study to be considered robust.