Data protection legislation: interpretation and barriers to research

Research has been described as “a powerful means of achieving” the objectives of the Department of Health, namely “to improve the health and well-being of the population and to secure high quality care.”¹ There is, however, a need to find a balance between facilitating important research and protecting the confidentiality of patients. As the capabilities of information technology grow, legal frameworks and professional guidance need to be created or refined to safeguard the rights of patients.

Some areas of the common law duty of confidentiality and the new Data Protection Act 1998 (box, p 891), which constitutes the United Kingdom's implementation of the relevant European Union directive,² are causing difficulties of interpretation within the NHS. With few exceptions, broad debate about the implications of the new act is lacking, particularly in the context of epidemiological research that uses patients' records.^6–8 Questions of consent, anonymisation of data for research, and access to medical notes for research purposes (rather than audit) have been addressed in a range of literature.^9–13 Some of these documents are being updated; this may indicate that there are uncertainties about the legal issues involved in implementing the act. Local variations in interpretation may cause particular difficulties for researchers conducting multicentre epidemiological studies, as the case study that will be described in this article shows.

In the meantime, those who must make decisions about confidentiality are still confused. This confusion exists for several reasons. Firstly, there is the interpretation of the act (and to an extent the common law duty of confidentiality). The interpretation is subject to debate, and no case law exists which might clarify the interpretation. Secondly, there is a dearth of up to date and clear policy guidance. Thirdly, the new system of “Caldicott guardians” (box) is untried, and guardians as well as others are only beginning to learn to exercise their new responsibilities. Fourthly, clarification is needed about the role that research ethics committees should have in data protection and confidentiality. Guidance recently issued by the NHS should help clarify some of these areas.¹⁴ We highlight issues for future discussion that have arisen in a case study of a multicentre epidemiological project that sought to use patients' records.

Summary points

• The interpretation of the Data Protection Act 1998 and how it affects the NHS, healthcare, and epidemiological research is riddled with uncertainties

• Clarification is needed to determine how the common law duty of confidentiality affects the health sector in terms of using patients' data for research

• Different interpretations of the act and the duty of confidentiality may adversely affect the ability of researchers to conduct multicentre studies

Case study

Regional NHS sources funded our department in collaboration with clinicians from five NHS trusts to undertake a retrospective pragmatic study of the effectiveness and cost effectiveness of a new drug treatment. In the initial phase it was expected that a registered nurse employed by the university would extract data on treatment and on the utilisation of health services from the routine records of patients seen in collaborating trusts.

The relevant multicentre research ethics committee approved the study but advised the researchers that the question of whether explicit consent was needed from patients to allow the researchers to have access to the medical records needed to be clarified with data protection officers at the five hospital trusts. The responses to this request are shown in the box (p 891). The trusts' decisions varied considerably and usually involved complex internal discussions and consultation; consequently, this led to delays.

Does the diversity in the outcomes mean that some trusts made erroneous judgments or that the law is ambiguous, or can the situations in individual trusts be sufficiently different for them to reach contrary decisions? Although the latter case seems unlikely, there are individual circumstances under which trusts may arrive at a different decision about the same project. One such condition may involve cases in which trusts have in place routine mechanisms to obtain consent from patients for the use of their personal data for future research, a procedure which would be subject to the approval of a research ethics committee.

Explanation of terms

Data Protection Act 1998—This brings into UK law European Directive 95/46/EC on the processing of personal data. It came into effect on 1 March 2000, and in comparison with the 1984 act (which it replaces) it is concerned with both records on paper and records held on computers. The act is based on eight principles the first of which stipulates that “personal data shall be processed fairly and lawfully.” Interpretation of the phrase “fairly and lawfully” may give rise to different opinions about implementation.

Common Law Duty of Confidentiality—This legal duty applies to information entrusted to someone in confidence. The duty of confidentiality applies independently of the Data Protection Act. The Department of Health acknowledges that there are conflicting legal views on applying this duty and is trying to interpret it for the health sector.³ In particular, the issue of consent and the conditions under which consent can be implied or waived need to be clarified.

Caldicott guardian—In 1997 the Caldicott Committee reported on its review of information that identifies NHS patients.⁴ In keeping with the report's main recommendations each health authority, trust, and primary care group in the United Kingdom appointed a “Caldicott guardian.” One key responsibility of the guardians is to agree and review internal protocols for the protection and use of identifiable information obtained from patients.⁵

Trusts' decisions on whether patients needed to give explicit consent

Trust 1—This trust decided that the researcher could have access to patients' records without explicit consent from patients as long as no identifiable information was removed from the hospital (for example, the researcher could extract information from records and retain it in coded form but the key for decoding would be kept at the hospital). (Time to decision: <3 weeks.)

Trust 2—The Caldicott guardian decided that consent from patients was required. This decision was later revised after the trust sought legal advice, and the researcher was then permitted to have access to patients' records because the Data Protection Act 1998 only came into force after the start of the study (1 March 2000). (Time to decision: 4-5 months.)

Trust 3—The data protection officer and the Caldicott guardian advised the researcher to obtain explicit consent from patients because the researcher was not a staff member of the trust and no explicit consent exists from patients to permit the use of their data for research (for example, no agreements are signed by patients when they are first seen). (Time to decision: 6 weeks.)

Trust 4—The data protection officer immediately decided that the proposed study required explicit consent from patients since only staff with a duty of care to the patient are permitted to have access to that patient's medical records, and, unlike audit, research is not seen as part of the healthcare process. (Time to decision: immediate.)

Trust 5—The data protection officer made a formal decision only about records held on the computer. The outsider status of the researcher was problematic. The case of deceased patients (which is not covered by the Data Protection Act) would have to be decided by the research ethics committee. (Time to decision: no formal decision at 7 weeks.)

As a result of the trusts' decisions there seemed to be three options available to the researchers: abandon the project entirely, seek explicit consent from patients who have been treated in the trusts that demand explicit consent, or alter the design of the study so that only anonymised data are used.

Discussion

Issues of consent, anonymisation, and access to patients' records for research need to be more widely discussed and evaluated in terms of the 1998 act and the Common Law Duty of Confidentiality. Well meaning clinicians may be passing anonymised or non-anonymised data to researchers without realising the legal implications.

It is not easy to answer questions about data protection requirements for particular research projects, and many individuals within trusts who are responsible for tackling these questions face difficulties in answering them. Because of the current uncertainty, insurmountable problems may arise in cases in which researchers hope to conduct their studies at a variety of centres, especially since they may have to comply with conflicting interpretations of the existing law and conflicting guidance from various bodies. This situation has created inconsistencies in the access to routine NHS data allowed to researchers. Additionally, the appropriate interactions between the new Caldicott guardians, the data protection officers in each trust, clinicians, and research ethics committees has not yet been fully clarified; however, a revision of the guidance for local research ethics committees is expected to be published later in the year and may partially address this problem.¹⁵ Also, anxieties about the requirements for consent have increased as a result of the exposure of cases in which organs were retained for research and medical research procedures were performed on children.¹⁶,¹⁷

One of the options for resolving the issue of consent in our case study was to use anonymous data. A High Court decision in May 1999 increased uncertainty in the healthcare and medical research communities about the legality of processing even fully anonymised data without consent¹⁸: in this case the judge held that confidentiality can be breached even when anonymisation is used if the patient has not consented and the research is not in the public interest (in this case, data were being sold by pharmacists indirectly to the pharmaceutical industry). The Court of Appeal overturned the judge's decision in December 1999, ruling that as a reasonable pharmacist's conscience would not be troubled by the proposed use of the information any claim for breach of confidentiality was unlikely to be successful.¹⁹ Unfortunately this aspect of the law remains unresolved because leave may be given to appeal to the House of Lords.

The view of the data protection commissioner is that any personal data which has been encoded remains personal data in the sense of the Data Protection Act 1998 provided that the key for decoding it remains in existence. Thus, coded data falls within the scope of the Data Protection Act even if the key for decoding it is not accessible to the researcher. The new NHS number being assigned to patients is an example of such a code, and chronic disease registers and reporting systems or postmarketing surveillance systems of new drug treatments might use codes that can be linked to individuals. Much epidemiological research and research into health economics would simply be impossible to conduct if completely anonymous data had to be used because updating, linking, or validating data is impossible without using codes.

The processing of coded personal data (sometimes called “pseudonymised” data to distinguish it from fully anonymised data) for research does not necessarily contravene the act. However, in considering whether data processing is “fair and lawful” routine mechanisms to merely inform patients in advance about the potential use of their personal data for future research (for example, through form letters or notices posted in waiting rooms) may not be seen as constituting sufficient consent. It is also unclear whether patients who do not register their refusal can be said to have consented. Neither the Data Protection Act 1998 nor the confidentiality law give sufficient guidance as to what constitutes explicit and implied consent and when each ought to be used.

Strict, clear criteria are urgently needed to determine under which limited situations such consent requirements for research using patient data might be waived; these must take into account the degree of anonymisation. The Department of Health's proposal to set up a national confidentiality and security advisory body, which was announced on 15 March, is welcome.²⁰ This new body should have the potential to provide the necessary clear guidance for research, similar to the guidance in the United States on disclosure of individually identifiable health data for research under specified conditions.²¹

Conclusion

Researchers performing epidemiological studies in the United Kingdom need clear guidance in several areas. Firstly, the definition of explicit consent and the situations in which it is required need further explanation. Secondly, there is an unacceptable amount of uncertainty over when consent can be considered to have been implied or when it may be waived on grounds of public interest. Research ethics committees may be asked to advise on whether processing identifiable data without consent is in the public interest. This is an onerous responsibility, especially in light of the uncertainties described in this paper. The Department of Health's ongoing review of guidelines for local research ethics committees will help illuminate this situation. The legal responsibility lies ultimately with the trust, and any decision regarding disclosure must be able to be justified as being in the public interest. Thirdly, anonymisation and its effects need to be clarified especially taking into account the court case described earlier. Fourthly, issues of access to confidential data must be resolved. The effect of a contract between the NHS and outside research staff also needs to be clarified (for example, in cases in which research staff are funded by the NHS itself or when they have an NHS contract with some, but not all, of the trusts involved in a multicentre research study). Ultimately, the legality of any guidance or decision can only be determined by the courts.

SUE SHARPLES

In the meantime, a workable solution that respects patients' rights may be to ensure that data are fully anonymised whenever possible. In this case, the data is not personal and does not fall within the scope of the Data Protection Act. If full anonymisation is not possible or the design of the study does not permit it, the use of pseudonymous data (created using codes and carefully restricting access to them) should be considered, bearing in mind that it is still seen by the data protection registrar as personal data. To facilitate future research, trusts need to ensure that sufficient mechanisms are in place to inform patients about any potential use of their data for research and to obtain consent when necessary. Finally, researchers should agree their project design with those responsible for data protection well in advance.