Abstract
General practice research databases are increasingly used to study intended and unintended effects of treatments. However, confounding by indication remains a major problem. The randomized database study methodology has been proposed as a method to combine the strengths of observational database (generalizability) and the strength of the randomized clinical trial (RCT) design (randomization). We developed an infrastructure that enables the execution of randomized database studies with treatment randomization facilitated by a general practice research database. The requirements posed by the methodology of randomized database studies were facilitated by software components. Our assessment showed that it is technically possible to conduct randomized trials in general practice according to the randomized database design. The infrastructure facilitated the conduct of randomized database studies in general practice but some practical difficulties and methodological issues remain. The technical infrastructure seems to be both promising and potentially feasible to facilitate future randomized database studies, although the methodology needs to be evaluated in more detail.
Background
An increasing number of general practitioners (GPs) replace traditional paper-based patient records with electronic patient records (EPRs). A typical EPR contains information about patient identification, demographics, type of visits, prescriptions, diagnoses, reasons for visits, referrals, laboratory findings, and other notes. In the Netherlands, for example, more than 90% of the GPs have replaced their paper records with EPRs.1 In automated general practices, the EPR facilitates many processes such as patient care, management, billing, planning of care processes, and education.2
Researchers have recognized the potential value of data collected with the EPR, and this realization has resulted in a number of so-called general practice research databases. These general practice research databases contain longitudinal data from the EPRs. In countries where the GP has a gatekeeper role in the health care system (e.g., the Netherlands, United Kingdom), these databases contain almost complete medical data. Examples are the General Practice Research Database3, Mediplus UK,4 and Integrated Primary Care Information database.5
Pharmacoepidemiology is an example of a research area that takes advantage of the availability of general practice research databases since the databases contain information on the population, drug use, and mild and severe outcomes. However, a major issue in the conduct of observational studies, especially concerning intended drug effects, is confounding by indication.*6 Such confounding occurs when the physicians' selection of a treatment is related to the severity of the underlying disease or to the prognosis of the patient.7
In the randomized clinical trial (RCT) design, confounding is dealt with by random allocation of treatments.8 Even though the RCT design is considered the gold standard in assessing treatment effects, it has some limitations. RCTs are often conducted in controlled environments with selected and limited patient groups. One of the key challenges in the interpretation of RCT results, therefore, is to determine whether the study results also apply to other settings and populations. The term generalizability is used to describe the degree to which the results can be generalized to other settings and populations.9 For example, a RCT conducted in a hospital environment controlled by a strict protocol with a specific patient population may not be generalizable to the general practice population.
Due to the lack of generalizability of RCTs to primary care settings, there has been a request for large simple trials or pragmatic trials that provide measures of treatment effectiveness (rather than efficacy) in this setting.10 In pragmatic RCTs, the patient sample is more heterogeneous and the evaluation and follow-up criteria are similar to those used in clinical practice. Compared to conventional RCTs, pragmatic RCTs are conducted with fewer restrictions and enable researchers to use study designs and data that are representative of the natural patient care setting.11
A number of researchers have argued that combining the strengths of observational studies in databases (generalizability) and the strength of the pragmatic RCT design (randomization) will result in a new method of research: randomized database studies.12 A randomized database study is described as a study in which the EPR is used to select eligible study candidates, to randomize patients, and to collect data on the course of the treatment.
Researchers use data collected during daily care to assess the outcome, as is done in pharmacoepidemiologic studies with general practice research databases. In a randomized database study, however, the randomization procedure needs to be incorporated in the daily care workflow, preferably when the treatment is prescribed.
Although the advantages of this approach have been recognized,13 no research has been conducted to further develop the randomized database study approach. In this paper, we describe our attempt to develop an infrastructure that enables the execution of randomized database studies with treatment randomization in the context of a general practice research database.
We first briefly describe the changes that have to be made to the EPR in order to generate data for a general practice research database. We then describe the additional system requirements posed by the randomized database study. Finally, we describe the different additional software components that we built to enable the execution of randomized database studies.
Methods
From the EPR to a General Practice Research Database
Physicians in primary care mainly use the EPR to document patient treatment. Researchers using data from the EPR have concluded that data in the EPR are not always suitable for their needs5 because the data requirements for research and clinical care are not always congruent. For example, researchers report that EPRs often contain in detail the actions performed by the physician, but often not the underlying rationale14,15; physicians often use the EPR to record what was done rather than why it was done.
In the early 1990s, we were involved in the development of a general practice research database based on EPRs used in general practice, the Integrated Primary Care Information (IPCI) database.5 When we developed the IPCI database, we analyzed the requirements of the researchers and built additional software to address the limitations of routinely recorded data when using such data for research purposes.
Requirement for Observational Research with the IPCI Database
Researchers intended to use the IPCI database primarily for pharmacoepidemiological research; the database should be the data source to test hypotheses about both adverse and beneficial effects of drugs. To enable investigators to conduct this type of research, we formulated the following requirements:
Researchers should have access to all medical data on the patients. Since GPs may record data in the EPR and on paper, we required that the GPs record all medical data in the electronic records. The general practices that supply data to the IPCI database should be paperless to ensure that all relevant events are recorded in the EPR.
Researchers should be able to follow treatments over time including changes in treatment. Therefore, GPs should record the indication for each prescription and switches to other treatments.
Researchers should not have to obtain informed consent from each individual patient for each study to avoid selective participation. Dutch law stipulates that patient data can be used for research without the patient's consent only if the data are anonymized.16 Therefore, we required the data in the IPCI database to be anonymous. This means that the identity of both the physicians and patients should be concealed to the researchers.
Researchers anticipated that it would be impossible to predict all data requirements for future studies. For some studies, the data might be incomplete. We therefore required the ability to obtain additional data from the GP.
Changes in the Information Processing with the EPR
In addition to the requirement that practices work paperless, three types of changes had to be made in the (information processing within the) EPR to enable the development of the IPCI database: changes in the data recorded by the GP, changes in the communication with the EPR, and finally changes in the organization of the database.
Changes in the data recorded by the GP involved adding software to link prescriptions to indications. When the GP prescribes a treatment, the software asks the GP additional data about the indications and therapy changes.
We added communication software that ensures the anonymity of the patient data and assigns a unique patient identification number that would allow researchers to follow the patient over time. The GP is the only person who is able to translate that identification number to the potential identity. After the patient has been anonymized, the communication software sends all data to the gatekeeper. The gatekeeper is a person responsible for the anonymity of the GPs. Finally, the data are stored in the central IPCI database.
The organization of the IPCI database uses a board of supervisors, which has the responsibility to ensure the maintenance of the anonymity of patients and GPs. In addition, the board of supervisors has to approve each study proposal and researchers' request to collect additional data. After the board of supervisors has approved a study, all individual GPs are informed about the study. The technical infrastructure of the IPCI project allows individual GPs to withdraw data on patients or specific data elements for studies.5 Patients are informed of the existence of the IPCI project by leaflets and posters in the office of the participating GPs.
Currently, the IPCI database contains information from EPRs of about 150 general practitioners (GPs) covering more than 600,000 patients and provides data for studies with various epidemiological study designs, e.g., case-control design, cohort design, and cross-sectional design.17,18,19,20,21,22 Conducting a study with the randomized database study method, however, was not possible in the IPCI database.
Toward Randomized Database Studies with the IPCI Database
To enable researchers to conduct randomized database studies, we first analyzed the requirements posed by randomized database studies. Second, we built additional software to solve the shortcomings of the GP information system with respect to the conduct of randomized database studies.
Requirements for a Randomized Database Study
We analyzed the procedures in the conduct of RCTs in general in order to integrate them with the daily care process. Four essential steps were distinguished in the conduct of randomized trials that would apply to the randomized database study as well (▶): patient selection, patient recruitment, randomization of treatment, follow-up of patients.
Figure 1.
Steps in a randomized clinical trial and the software modules built to integrate the steps with the workflow of general practices working with an electronic patient record.
Patient selection comprises the identification of patients who are eligible for participation in a specific trial. Completeness of identification of eligible patients is necessary to be able to assess whether the included group is representative for the total eligible population.23 There may be large differences between the included patients and the total eligible population, for example, whether the included patients are healthier or whether there is a large overrepresentation of one gender. In conventional randomized studies, the selection methods are often are not standardized and there is no information about the nonincluded persons, which severely limits the possibility to evaluate generalizability.23
Patient recruitment in randomized trials involves the assessment of inclusion and exclusion criteria and obtaining informed consent from the patient. Researchers and recruiters are required to adhere to the good clinical practice (GCP) guidelines.24 GCP is an international ethical and scientific standard for designing, conducting, performing, analyzing, and reporting clinical trials. One of the most important ethical principles of GCP is informed consent from participating patients: the recruiter should fully inform the patient about the study before the patient gives written consent to participate. In addition, the patients should have enough time to reflect before consent is given.
In randomized database studies, patients may be recruited during routine care visits.13 The constraints of the GP in recruiting patients during regular visits (i.e., time or disruption of the daily care process), however, needs to be addressed. The procedure itself should not be a disproportional disruption of the normal interaction between the GP and the patient.
Randomization in randomized database studies comprises the random allocation of treatment on a patient level. Randomization of treatments in a multicenter trial is often done centrally or by a local randomization procedure (i.e., envelopes, random number generator). To minimize disruption of the normal care process, the randomization procedure should preferentially be integrated in the workflow of the normal prescription routine.
Patient follow-up in randomized trials consists of scheduled return visits to assess outcome parameters. Data are usually collected on case report forms. In the randomized database study, researchers use daily care data as recorded in the EPR to assess outcomes. This requirement poses two problems related to the GCP guidelines and the quality of the data. First, the GCP guidelines require that the documentation of follow-up is accurate, complete, legible, time stamped, and available for auditing. Second, study data derived from the source documents should be consistent with the source documents. If there are inconsistencies, the researchers should document and be able to explain them. There is a theoretical possibility that data in the EPR can be changed retrospectively, which could go unnoticed if time stamping does not occur accurately. An audit trail of the EPR, therefore, is an essential feature to comply with the GCP documentation guidelines.
Outcome assessment in the randomized database study approach will be done from the EPR data, but the quality and completeness of the data might not be optimal for all types of outcomes. For that reason, researchers required the possibility of collecting data from the patient as well. At the same time, the researchers should maintain the level of confidentiality required by use of the IPCI database.
Data on adverse drug reactions need to be collected according to regular spontaneous reporting schemes. Sudden unexpected serious adverse reactions and serious reactions have to be reported within 24 hours to the research center and the Netherlands Pharmaco-vigilance Center in accordance with the newest European guidelines.25
Results
Changes in Infrastructure to Enable Randomized Database Studies
We aimed to integrate the randomized database study with the daily care process in general practice by means of adding software to the general practice information system. The software consisted of different modules corresponding with the four essential steps in randomized studies: selection, recruitment, randomization, and follow-up of the patients.
Selection Module
After installation of the software, the information system of the GP activated the selection module. The selection module contained the query that identifies potential patients for a specific study based on data that were already available in the general practice information system (e.g., diagnosis, demographics, laboratory findings). The query may contain coded and free-text searches but the latter required manual validation of the results to reduce the false-positively selected patients prior to having them marked for recruitment. After the selection of potential patients, the selection module generated a reminder with a special message in the EPR of the selected patients. Whenever the GP opened the patient's EPR, the message reminded the GP that the patient had been selected as a potential subject for a study.
Recruitment Module
GPs were confronted with reminders in the EPRs of the selected patients whenever the patients' EPR was opened. To minimize interference of the regular workflow, the GP had to start the recruitment module themselves whenever the patient was eligible for recruitment.
The recruitment module required completion of an automated questionnaire based on the inclusion and exclusion criteria that were formulated in the study protocol. Patients who fulfilled the inclusion criteria needed to give written informed consent before the GP could finalize the recruitment step. If the patient asked for time to reflect, the recruitment module enabled the GP to postpone the decision to include the patient in the study and to continue later in time from that point on. The recruitment module also stored an electronic version of the patients' informed consent for the research database. Once a patient was included or excluded, the software removed the reminder from the EPR. The user interface of the recruitment module was the same as the interface of the general practice information system.
Randomization Module
After finalization of the recruitment step, the randomization module allocated the patient to one of the treatment options. The software presented the results of the randomization procedure to the GP and verified whether the randomized treatment was actually prescribed. The recruitment module also ensured equal allocation to the alternative treatments within a practice.26
Follow-up Module
We used the IPCI infrastructure to collect patient data but added a follow-up module that allowed researchers to collect data directly from the patients while maintaining the anonymity of the patients and the GPs. This was achieved by producing a new study number for each individual patient. This number differed from the patient number in the GP information system and the patient identification number in the IPCI database. Researchers used this study number to collect data from patients by means of patient questionnaires to communicate with the GP about the patient and to link the information with the IPCI database. To comply with the GCP documentation requirement, we retained a time-stamped printed version of the EPR and the patient questionnaires as source document.
Application
To test the feasibility and validity of a randomized database study, we compared the gastrointestinal tolerability of celecoxib and diclofenac in patients diagnosed with osteoarthritis. Both celecoxib and diclofenac are nonsteroidal anti-inflammatory drugs (NSAIDs) licensed, marketed, and reimbursed for the treatment of osteoarthritis. Due to preferential prescribing (i.e., channeling) of celecoxib to patients with gastrointestinal and cardiovascular comorbidity in general practice,27 it was considered difficult to assess this study question in an observational study with a general practice research database. All patients diagnosed with osteoarthritis who needed an NSAID for osteoarthritis during routine GP visits were eligible for entry in the study. During the recruitment, patients could be excluded if they were treated with a NSAID in the past three months or if they had any contraindication. After recruitment, patients were automatically randomized to diclofenac or celecoxib, but the GP decided the dose and treatment regimen. In the naturalistic follow-up, we focused on changes in NSAID treatment indicative of gastrointestinal intolerability (e.g., discontinuation of drug, adding gastroprotective agents).
We recruited 42 GPs and implemented the software in their information system. We used the local EPR database in the general practice information system to select patients older than 18 years of age who were diagnosed with osteoarthritis. The selection module selected 7,127 patients who met the selection criteria; for these patients, the selection module generated a reminder in the EPR.
During a median patient recruitment period of 188 days (range, 26–261), the GPs had contact with 4,586 of the 7,127 selected patients. When the GP accessed the EPRs of these patients, the selection module displayed the message reminding the GP that these patients were selected for the randomized database study. The GPs prescribed NSAIDs to 1,245 of the 4,586 patients. However, only 170 received the NSAID directly for osteoarthritis; these patients were potentially eligible patients for the study.
The objective of the recruitment module was to facilitate the recruitment procedure, and it also documented the reasons for noninclusion. Of the 170 potentially eligible patients, 42 (24.7%) patients meet one of the exclusion criteria. Another 12 patients (7.1%) refused to participate. In 55 (32.4%) cases, the GP stated that he or she was not the principal health care provider treating the patient at the moment the patient was eligible for recruitment and therefore could not include the patient. In 30 cases (17.6%), the GP stated he or she was too busy to start the informed consent procedure. Finally, in 11 cases (6.5%), the GP forgot to start the recruitment procedure. The remaining 20 cases (11.8%) were included in the study and randomized to the treatment arms by the randomization module. The naturalistic course of the treatment was monitored by retrieving the EPRs of the included patients. In addition, to study generalizability, the EPRs of the entire selected population were retrieved.
Due to low number of eligible patients, the recruitment was less than our initial expectations and we planned to terminate the study. Events overtook us when other researchers reported an increased risk of cardiovascular adverse events in patients treated with high doses of celecoxib, and we therefore terminated the study at that time.28
Discussion
In this paper, we describe our attempt to build an infrastructure to enable researchers to conduct a randomized database study in the IPCI general practice research database.
Our assessment shows that it is technically possible to conduct a randomized database study in a general practice research database and RCTs in the future. The shortcomings of the existing GP information systems that are the basis for the IPCI database were solved by software modules that corresponded with the essential steps in the conduct of randomized database studies, namely, patient selection, recruitment, randomization, and follow-up. Although the software facilitates the conduct of a randomized database study, some practical and methodological problems remain.
Regarding practical issues, the number of eligible patients was less than expected. Patient recruitment depended on the visit rate of the selected patients and whether they required NSAID treatment for osteoarthritis. In our study, we observed that more than a third of the selected patients did not visit the GP during the patient recruitment period and almost half of the patients did not require NSAID treatment. In addition, many patients received NSAID treatment for indications other than osteoarthritis.
The pressure on daily care in general practice is reflected by the fact that the participating GPs were not always the principal health care providers treating the patient or that they reported to be too busy or they simply forgot to recruit the patient. These general practice–related issues were the leading cause of noninclusion (56.5%). Although we facilitated the recruitment procedure, it remains a disruption of the workflow of normal practice and it does require extra time, which may have limited the performance.29
It is difficult to judge how the performance of the recruitment strategy in our randomized database study (11.8%) compares to that of other studies due to the lack of data about their source populations. In one study comparing multiple patient recruitment approaches in an RCT conducted in primary care, the researchers reported that only 1.4% of all enrolled patients were recruited directly by the physicians.30 In this study, direct physician recruitment was discontinued. Even though we cannot compare the percentages of this study to our data directly, we conclude that our recruitment strategy was effective, but leaves room for improvement; alternative patient recruitment strategies should be considered in future studies.
Several methodological issues remain regarding the implementation of the randomized database study design itself. The purpose of the whole endeavor was to circumvent confounding by indication while keeping the naturalistic follow-up procedure and outcome assessments. The gain obtained with removal of confounding may be at the expense of introducing a form of selection and information bias, which are absent in observational studies in the same research database. For example, due to time constraints, hesitance to recruit, or other issues, GPs may not recruit consecutive patients.31 As a result, only a selected population will enter the study, which may limit the generalizability of the results.
Although we were able to verify patient recruitment (information on eligible but nonincluded patients is available in the IPCI database and in the randomized database study software log), selective recruitment may limit both the sample size and the possibility of extrapolating the results to other populations.32 In addition, a naturalistic randomized database study is an open-labeled study (i.e., the GP and the patients are aware of the study question and the intervention). Due to this feature, information bias may occur if the follow-up and outcomes are recorded differently for the different study drugs.33
Conclusion
In conclusion, we described an infrastructure that facilitates randomized database studies in the IPCI database. Technically, it is feasible to conduct studies in automated general practice according to the randomized database study design. The infrastructure built to conduct randomized database studies in general practice research databases, however, showed some practical difficulties in the conduct of such a study and some issues that could jeopardize the validity of the methodology. Randomized database studies seem to be both promising and potentially feasible for future studies; the methodological issues, however, need to be evaluated in more detail.
Footnotes
Confounding by indication is a term used when a variable is a risk factor for a disease among nonexposed persons and is associated with exposure of interest in the population from which the cases derive, without being an intermediate step in the causal pathway between the exposure and the disease.
References
- 1.van der Lei J, Duisterhout JS, Westerhof HP, van der Does E, Cromme PV, Boon WM, et al. The introduction of computer-based patient records in the Netherlands. Ann Intern Med. 1993;10:1046–8. [DOI] [PubMed] [Google Scholar]
- 2.Knottnerus JA. Role of the electronic patient record in the development of general practice in the Netherlands. Methods Inf Med. 1999;38:350–4. [PubMed] [Google Scholar]
- 3.Walley T, Mantgani A. The UK General Practice Research Database. Lancet. 1997;350:1097–9. [DOI] [PubMed] [Google Scholar]
- 4.Dietlein G, Schroder-Bernhardi D. Use of the Mediplus patient database in healthcare research. Int J Clin Pharmacol Ther. 2002;40:130–3. [DOI] [PubMed] [Google Scholar]
- 5.Vlug AE, van der Lei J, Mosseveld BM, van Wijk MA, van der Linden PD, Sturkenboom MC, et al. Postmarketing surveillance based on electronic patient records: the IPCI project. Methods Inf Med. 1999;38:339–44. [PubMed] [Google Scholar]
- 6.Byard DP. Problems with using observational databases to compare treatments. Stat Med. 1991;10:663–6. [DOI] [PubMed] [Google Scholar]
- 7.Salas M, Hofman A, Stricker BH. Confounding by indication: an example of variation in the use of epidemiologic terminology. Am J Epidemiol. 1999;149:981–3. [DOI] [PubMed] [Google Scholar]
- 8.Altman D, Bland J. Treatment allocation in controlled trials: why randomise? BMJ. 1999;318:1209. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Rovers MM, Zielhuis GA, Bennett K, Haggard M. Generalisability of clinical trials in otitis media with effusion. Int J Pediatr Otorhinolaryngol. 2001;60:29–40. [DOI] [PubMed] [Google Scholar]
- 10.Thomas P. The research needs of primary care. BMJ. 2000;321:2–3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Charlton BG. Understanding randomized clinical trials in general practice: explanatory or pragmatic? Fam Pract. 1994;11:243–4. [DOI] [PubMed] [Google Scholar]
- 12.Garcia Rodriguez LA, Perez Gutthann S. Use of the UK General Practice Research Database for pharmacoepidemiology. Br J Clin Pharmacol. 1998;45:419–25. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Sacristan JA, Soto J, Galende I, Hylan TR. A review of methodologies to assess drug effectiveness and a new proposal: randomized database studies. Clin Ther. 1997;19:1510–7. [DOI] [PubMed] [Google Scholar]
- 14.Dorr DA, Rowan B, Weed M, James B, Clayton P. Physicians' attitudes regarding patient access to electronic medical records. AMIA Annu Symp Proc. 2003:832. [PMC free article] [PubMed]
- 15.van der Lei J, Musen MA, van der Does E, Man in 't Veld AJ, van Bemmel JH. Comparison of computer-aided and human review of general practitioners' management of hypertension. Lancet. 1991;338:1504–8. [DOI] [PubMed] [Google Scholar]
- 16.Gezondheidsraad. Privacy bij Post-marketing Surveillance [Health Council. Privacy on Post-marketing Surveillance]. Health Council's Gravenhage, The Netherlands 1993 [Dutch].
- 17.Voordouw AC, Sturkenboom MC, Dieleman JP, Stijnen T, Smith DJ, van der Lei J, et al. Annual revaccination against influenza and mortality risk in community-dwelling elderly persons. JAMA. 2004;292:2089–95. [DOI] [PubMed] [Google Scholar]
- 18.Straus SM, Bleumink GS, Dieleman JP, van der Lei J, 't Jong GW, Kingma JH, et al. Antipsychotics and the risk of sudden cardiac death. Arch Intern Med. 2004;164:1293–7. [DOI] [PubMed] [Google Scholar]
- 19.Van Wyk JT, Van Wijk MA. Assessment of the possibility to classify patients according to cholesterol guideline screening criteria using routinely recorded electronic patient record data. Stud Health Technol Inform. 2002;93:39–46. [PubMed] [Google Scholar]
- 20.Straus SM, Bleumink GS, Dieleman JP, van der Lei J, Stricker BH, Sturkenboom MC. The incidence of sudden cardiac death in the general population. J Clin Epidemiol. 2004;57:98–102. [DOI] [PubMed] [Google Scholar]
- 21.Verhamme KM, Dieleman JP, Bleumink GS, Bosch JL, Stricker BH, Sturkenboom MC. Treatment strategies, patterns of drug use and treatment discontinuation in men with LUTS suggestive of benign prostatic hyperplasia: the Triumph project. Eur Urol. 2003;44:539–45. [DOI] [PubMed] [Google Scholar]
- 22.van der Linden PD, van de Lei J, Nab HW, Knol A, Stricker BH. Achilles tendinitis associated with fluoroquinolones. Br J Clin Pharmacol. 1999;48:433–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Britton A, McKee M, Black N, McPherson K, Sanderson C, Bain C. Threats to applicability of randomised trials: exclusions and selective participation. J Health Serv Res Policy. 1999;4:112–21. [DOI] [PubMed] [Google Scholar]
- 24.EMEA. ICH Topic 6. Guidance for good clinical practice. Available from: http://www.emea.eu.int/pdfs/human/ich/013595en.pdf. Accessed September 2004.
- 25.EMEA. Detailed guidance on the collection, verification and presentation of adverse reaction reports arising from clinical trials in medicinal products for human use; April 2004. Available from: http://eudract.emea.eu.int/docs/Detailed%20guidance%20collection%20of%20adverse%20events.pdf. Accessed May 2005.
- 26.Altman DG, Bland JM. How to randomise. BMJ. 1999;319:703–4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Schneeweiss S, Glynn RJ, Avorn J, Solomon DH. A Medicare database review found that physician preferences increasingly outweighed patient characteristics as determinants of first-time prescriptions for COX-2 inhibitors. J Clin Epidemiol. 2005;58:98–102. [DOI] [PubMed] [Google Scholar]
- 28.Available from: www.pfizer.com/are/mn_investigators_news.cfm. Accessed February 2005.
- 29.Prout H, Kinnersley P, Robling M, Hood K, Tudor-Jones R. A qualitative evaluation of implementing a randomized controlled trial in general practice. Fam Pract. 2003;20:675–81. [DOI] [PubMed] [Google Scholar]
- 30.Margitic S, Sevick MA, Miller M, Albright C, Banton J, Callahan K, et al. Challenges faced in recruiting patients from primary care practice into physical activity intervention trial. Prev Med. 1999;29:277–86. [DOI] [PubMed] [Google Scholar]
- 31.Tognoni G, Alli C, Avanzini F, Bettelli G, Colombo F, Corso R, et al. Randomised clinical trials in general practice: lessons from a failure. BMJ. 1991;303:969–71. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Kramers SM, Shapiro SH. Scientific Challenges in the application of randomized trials. JAMA. 1984;252:2739–45. [PubMed] [Google Scholar]
- 33.Schulz KF, Grimes DA. Blinding in randomised trials: hiding who got what. Lancet. 2002;359:696–700. [DOI] [PubMed] [Google Scholar]