Practice-Based Research Network Infrastructure Design for Institutional Review Board Risk Assessment and Generalizability of Clinical Results

Abstract

Data from clinical studies generated by Practice Based Research Networks should be generalizable to the profession. For nationally representative data a broad recruitment of practitioners may pose added risks to IRB's. Infrastructure must assure data integrity while minimizing risk to assure that the clinical results are generalizable. The PEARL Network is an interdisciplinary dental/medical PBRN conducting a broad range of clinical studies. The infrastructure is designed to support the principles of Good Clinical Practice (GCP) and create a data audit trail to ensure data integrity for generalizability. As the PBRN concept becomes of greater interest, membership may expand beyond the local community, and the issue of geography versus risk management becomes of concern to the IRB. The PEARL Network describes how it resolves many of the issues related to recruiting on a National basis while maintaining study compliance to ensure patient safety and minimize risk to the IRB.

Introduction

A longstanding issue for PBRNs has been the role of the IRB and how it assesses risk to the patient and practitioner for participation in a study. This manuscript describes PBRN risk assessment and the Institutional Review Board (IRB) for a dental/medical PBRN located at New York University College of Dentistry called PEARL, an acronym for Practitioners Engaged in Applied Research & Learning. The PEARL Network was initially funded by the National Institution of Health (NIH). The PBRN concept has gained some attention with the recent passage by the US Congress of both the Health Care and Education Reconciliation Act of 2010 and the Patient Protection and Affordable Care Act.^{1, 2} The clinical emphasis on these acts are centered on comparative effectiveness research (CER), the hallmark of studies conducted by PBRNs.

In 2005, the PEARL Network was launched as an initiative to close the translational gap between bench research and clinical practice. One of the premises for a PBRN is that practitioners are more likely to change the way they currently practice by participating in a study and incorporating the clinical findings into their practice. To keep practitioners engaged the PBRNs reimbursed practitioners for their time involved in a study only, not for patient care. The PEARL Network (www.pearlnetwork.org) differs in infrastructure in many ways from traditional PBRNs. Although PEARL is centralized at New York University, the data management and statistical analysis are performed by a data coordinating center located in Rockville, Maryland (The EMMES, Corp.). PEARL is an interdisciplinary Network consisting of a dental and a medical component and conducts a range of clinical studies from surveys, retrospective and prospective studies to randomized controlled trials (RCTs). PEARL is a practitioner based and patient centered PBRN functioning on a proprietary and Food and Drug Administration (FDA) compliant (21 CFR) remote data entry system developed and managed by the EMMES Corp. PEARL has also conducted Network to Network studies and has recruited hospitals and community clinics. Although there is no current consensus on a definition of a PBRN PEARL functions under the definition of: A PBRN is a collaboration between an academic health science center and community practitioners conducting primarily clinical studies of mutual interest that would benefit and enhance patient care, delivery, cost and to improve health care policy.³ The code of federal regulations that governs human-subjects research- known as the ‘Common Rule’ – charge institutional review boards (IRBs) with ensuring first that “risk to subjects are minimized”, and second with ensuring that “risks to subjects are reasonable in relation to anticipated benefits...”.⁴ Concerns that clinical research conducted in tertiary care centers was not generalizable to the primary care setting, including community based populations, assisted in the development of practice-based research networks (PBRNs).^{5, 6} Geography of the PBRN practitioners presents further issues to be resolved with the IRB.

PBRN Compliance Issues and the Institutional Review Board (IRB)

The PEARL Network's infrastructure was designed to contain specific safeguards to minimize risk to the patient's regardless of the P-I's level of clinical research experience to ensure data integrity. Figure 1 depicts the organization and relationship of the IRB to the Network. Protocol and IRB compliance is managed by PEARL in terms of risk assessment.

Figure 1.

Relationship between New York University School of Medicine Institutional Review Board to the PEARL Network and collaborative schools.

IRB issues focus on study compliance, patient risk, oversight of the study sites and their ability to fulfill the protocol requirements. These concerns arise from the PBRN concept that clinical research should not be restricted to academic-health science centers as they only represent a small percentage of the population seeking medical care. ⁷ This has been reviewed and the issue still remains.⁸ PBRN practitioner-investigators (P-Is) may lack experience in the principles of clinical research and from a purist point of view the P-Is have not been influenced by study controls. However, the principles of clinical research should not alter the way a practitioner administers patient care as they are completely separate issues. The principles of clinical research as defined under Good Clinical Practice (GCP) provide a solution accompanied with an audit trail to ensure data integrity and reproducibility.

Creating a transparent relationship between the PEARL Network and the NYU School of Medicine (NYU SoM) IRB has provided the opportunity for PEARL to partner with the IRB on the concept, goals and mission of the Network. This transparency allows for a shared governance with the IRB to see beyond the visible aspect of a PBRN and observe the core operations which are formally structured to support clinical research.⁹ The formality of the PEARL Network is shielded under the banner of a PBRN but consists of standard operating procedures, a clinical monitoring staff of clinical research associates, formal procedures on P-I education and training, and study close out, all following the principles of Good Clinical Practice (GCP) to ensure study compliance and patient safety. This infrastructure, designed to minimize patient and investigator risk to the IRB, has led the NYU School of Medicine to be the IRB of record for the PEARL Network. The NYU IRB , is fully accredited by the Association for the Accreditation of Human Research Protection Programs, Inc., (AAHARPP). The novelty of the NYU IRB relationship is that PEARL is no longer a local or regional Network but has evolved into a National Network of P-Is. The structure of a PBRN is centered on multi-site practitioners and one study included a multi-network trans-PBRN study on osteonecrosis of the jaw¹⁰ requiring a certificate of confidentiality.¹¹ PEARL in the course of practitioner recruitment has worked with a large number of institutional IRB's and with this experience concurs with the literature that there is considerable variability in IRB processes even for minimal-risk studies.¹² This collaboration of the PEARL Network and the NYU SoM IRB does not involve shared responsibilities with any external IRB. However, when PEARL was first formed it used a centralized IRB which morphed into a hybrid of using both internal and external IRB's finally ending up with an exclusive arrangement with the SoM. This learning lesson exemplified the issues which were discussed at the National Conference on Alternative IRB Models. ¹³ This environment challenged the PEARL Network to design studies that would define its depth, robustness and its capability to conduct a variety of clinical studies utilizing both simple and complex clinical designs. Over a seven year period, PEARL completed sixteen studies including a number of surveys, retrospective and prospective studies as well as randomized controlled clinical effectiveness studies, best describe as randomized effectiveness studies (RES). The PEARL clinical operations group works closely with the School of Medicine IRB and has in place a number of procedures and documents to assure the IRB that PEARL has diligent oversight of the clinical studies to both assess and manage risk. Briefly, the procedures include practitioner screening, tutorials which include the Collaborative Institutional Training Initiative (CITI) online training as per IRB compliance, signed investigator agreements for study compliance as well as for reimbursement of practitioner study time, ongoing clinical monitoring of a study, electronic data capture for real time quality assurance intervention, study close out and resolution of study queries prior to locking of the data base. PEARL has an SOP for practitioner mis-conduct, and describes the safeguards that PEARL has in place to assure the IRB of practitioner oversight and risk mitigation. Compliance to study protocols, GCP and the awareness of monitoring of the study and audit of the source documents are important attributes that are the hallmark of PEARL. Since some 71% of dentists are solo practitioners the idea of a quality assurance person evaluating their performance on a study could cause an issue if not addressed and presented beforehand.^14,15 The PEARL recruitment cycle (figure 2) classifies practitioners according to their level of interest and participation. Some practitioners choose to do only surveys (tier III) while others are highly involved and engaged in RES (tier I) while the majority participate in a study of their interest (tier II). Surveys are conducted using all tiers for greater representation.

Figure 2.

Progression of practitioner-investigators related to the level of studies.

Early phase studies, (I, II) establishing safety and considered high risk, requires a controlled environment and are not candidates for a PBRN. PBRN studies are primarily “standard of care” using FDA approved medications, devices and/or procedures. The format of these studies varies, for example, a standard of care study could take the form of RES whereby the randomization could be at the practice or patient level. All practitioners of PEARL are credentialed as “Research Associates” of New York University. Studies are conducted using electronic data capture so that distance of the practitioners from NYU is not a factor. The practitioner screening process is designed to meet University and GCP standards for compliance and data integrity. This can also limit the recruitment of practitioners which is another reason for broadening the Network to a National level. It is estimated that by the number of practitioners screened compared to those who are active members approximately five percent of the dental population are interested in the concept. This may increase over time. Retention of practitioners, who satisfy the PEARL criteria, is the most important issue for sustainability of the Network as a high turnover can be costly and limit the progress of ongoing studies. The screening process for PEARL is a distinguishing characteristic for the network to ensure timely study compliance and data integrity. Two important characteristics of PEARL PBRN clinical studies are the clinical outcome parameters which include “patient reported outcomes” making the studies “patient centered” and “benchmarking” which has the investigator receiving metrics on how they performed in the study compared to other investigators (anonymously) making the studies interactive from both the patient and the investigator point of view. The benchmarking component functions as a quality assurance metric to explain data that may be considered as an outlier. This information is unique and adds insight to explain the statistical evaluations of data that may not be consistent with the pharmacology of the drug being tested. This educational continuum allows for practitioner comparison within studies and possible practitioner improvement. This may further assist in closing the translational gap.¹⁶ PBRNs have been identified as the functional entities to conduct point of use clinical studies of medications and/or devices.

For studies involving drug development by pharmaceutical companies the question remains as to how to cast a broader drug safety net during the development process to identify drug side effects which may not be so evident in a controlled clinical study. Conducting patient use studies in a PBRN may provide early detection of just how robust the drug safety profile is at that stage of development. Such studies need to be considered on a case by case basis as one does not know the full side effect potential of the drug in a less controlled environment. However, conducting a PBRN study as a Phase III or IV Trial can add a large drug safety profile to the portfolio. A PBRN can add both a large number of patients to assess drug safety and effectiveness as well as practitioners to assess use in a relatively short time. The pharmaceutical industry normally will compensate the cost of conducting clinical research in the physician offices (site management organizations, SMO), much like PBRNs who are not independently supported by government funding.¹⁷ However the funding, the PBRN remains a concept that can satisfy regulatory compliance for industry. Government sponsored PBRNs have limitations on the studies they conduct.

Risk and Benefits of Studies

PBRN studies can define the treatment that is most effective under the clinical conditions of the study design termed best practice. They can also clarify and have the potential to expand the label claim. Risk management by the PBRN for the provider can be described in three criteria which include: conceptual and operational criteria, an appropriate balance of relevant moral concerns and theoretical unity in scope of applicability.¹⁸ IRB approval can be by a health science center, university IRB or by an approved central IRB. The difference resides in the mission of the PBRN and whether it chooses to offer practitioners educational and academic components for personal career development and/or enhancement. A PBRN not affiliated with a university functions more like a contract management organization or more specifically a SMO. For industry purposes the university affiliation between the practitioner and the sponsor can function as a safeguard to ensure regulatory compliance. The degree of intrusion that the PBRN has on the practitioner and office staff in conducting studies should be minimal. This is a fine line and one that can be debated but the issue still remains as to “data integrity”. If one does not have confidence in the data then the study and its impact is at risk. PBRN clinical findings should be reproducible if it is to be meaningful and contribute to practitioners changing the way patients are treated. An infrastructure with safeguards assures the network of some form of clinical milestones for reproducibility. The safeguards that are used should not be at the point of use as this will alter the intent of a PBRN CER study. These safeguards also assure the IRB that the PBRN Network has oversight of the study and that patients are at minimal risk.

PBRNs can be structured in many different forms and philosophically are the antithesis of the so called ‘gold standard’ of clinical studies, the randomized controlled trial (RCT). The challenge now in health care is directed at novel strategies to expedite clinical studies and some have suggested a moratorium on the traditional RCT pathway.¹⁹ PEARL has only conducted studies using approved drugs or standard of care studies. Thus, an RES becomes a sub-classification of the RCT. The RCT, however, transcends all phases of drug development. The RES is limited to possibly phase III and post approval studies (phase IV). Both types of studies satisfy different clinical end points and serve to increase the clinical knowledge base. However, they function at different ends of the spectrum in regards to risk assessment for the IRB. The RCT is designed to assess safety and efficacy for a drug in the drug development pipeline in a specific population in a controlled environment with a highly trained investigator to control and manage risk to the patient. This controlled environment is needed for early clinical studies to build the knowledge base and safety profile for a new drug. The RCT remains the fundamental basis for early clinical drug development. An RES is consistent with the goals of CER where “participants are patients” of the practice, and an RCT “participants are subjects” and are mainly recruited from outside the practice. Recruiting patients from an investigator's practice mitigates risk as the provider has a history of the patient. Recruiting subjects externally to clinical studies may increase the risk and variability of the data as the investigator lacks the depth of patient history available to the primary care provider.

PBRN Spectrum of Studies

PBRN surveys (Table 1) conducted, in many instances, may have the informed consent streamlined into the survey and implied through participation. However, apart from such implied consent, the level of risk assessment is minimal, if any, only if the type of research and/or data collection is defined as: (1) projects that involve non-invasive procedures; (2) there is no apparent risk to participants above the everyday norm; and (3) participants (and/or organizations) are not identified or have complete anonymity. An example includes the use of questions/surveys that do not include sensitive questions and are sent to adult participants, and returned anonymously so that participants cannot be identified.

Table 1.

Risk assessment assigned to studies categorized as randomized effectiveness research (RES) and randomized controlled trial (RCT).

PEARL Practitioner-Investigator
IRB Risk Assessment by Study Type
Type of Study PEARL Clinical Portfolio	IRB Risk/Benefit	Degree of Intervention/Geographic Dependency	Qualifications for Site Activation
Survey TMJD Perio SBIRT I SBIRT II PEARL/WREN (PBRNs) Analgesic	Risk: -None Known Benefit: -Optimized Clinical Study Design -Future Benefit to patients -Network Practitioner Communication	No/No	Tier: III Tier II Tier I
Comparative Effectiveness Research (CER) – Retrospective ONJ Endo Outcomes Implant Outcomes	Risk: -None Known Benefit: -Optimized Clinical Study Design -Patient receives standard of care treatment -Cost Effectiveness	No/No	Tier II Tier I
Comparative Effectiveness Research (CER) – Prospective POH Deep Caries Analgesic	Risk: -Known/Low Benefit: -Optimized treatment outcomes -Cost Effectiveness	No/No	Tier II Tier I
Standard of Care RES using FDA/ADA approved products Randomized Effectiveness Studies (RES) NCL POH	Risk: -Known/Low Benefit -Optimized Clinical Study Design -Patient receives standard of care treatment -Cost Effectiveness	Yes/No	Tier I
RCT phase IV development Pharmacovigilance Could be RES	Risk: -Low/Moderate Benefit: - Label Change -Safety/ Risk Assessment	Yes/No	Tier II Tier I
RCT phase III development Randomized Clinical Trial To be determined Could be RES	Risk: -Low/Moderate Benefit: -Efficacy -Improved Safety	Yes/Yes	Tier I
RCT phase II development	Risk: -Moderate/High Benefit: -Unknown/Determining Drug Parameters	Yes/Highly Selective Sites	Research Trained Investigators; not yet conducted in a PBRN
RCT phase I development	Risk: - High Benefit: -Unknown/Determining Drug Parameters		Research Trained Investigators; not yet conducted in a PBRN

Glossary of Protocols

TMJD: PRL0910 Advancing Care Management for TMJD Pain: Planning for a Clinical Trial

Perio: PRL0808, A Case Study of Diagnosis, Treatment, and Maintenance/Recall of Periodontal Patients by General Dentists

SBIRT I/II: Screening and Interventions for Tobacco, Alcohol, and other Drug

Use in Dental Settings: Survey to Assess Provider Practices and Interest

PEARL/WREN Analgesic: PRL1014 Oral Assessment of Pain by Dental Practitioners; Wisconsin Research and Education Network.

ONJ: PRL0603 Trans-PBRN Case-Control Study of Osteonecrosis of the Jaws

Endo Outcomes: PRL0705 Outcomes for Endodontic Treatment and Restoration of Teeth in Dental Practice Implant Outcomes: PRL 1012 Outcomes for Single Unit Implant Placement and Restoration in General Dental Practice

POH: PRL0602 Post-Operative Hypersensitivity in Occlusal Restorations

Deep Caries: PRL 0604 Complete VS Partial Removal of Deep Caries: A Comparison Study of Treatment Outcomes

Analgesic: PRL0706 Analgesic Use Effectiveness

NCL: PRL0707 Noncarious Cervical Lesion Treatment Outcomes: Randomized Clinical Trial

POH RCS: PRL1013 Resin Based Composite Occlusal Restoration Postoperative

Hypersensitivity: Randomized Comparative Effectiveness Research Trial.

Tier: III: Credentialed Practitioners

Tier II: Credentialed, Trained Practitioner-Investigator

Tier I: Credentialed, Trained, Experienced Practitioner-Investigators

Standard of Care Studies

Risks associated with Standard of Care Studies are the same risks that exist in everyday practice. Even with minimal intervention, an ethical assessment of risk must also include a voluntary informed consent, whereby potential research participants know if they are part of the placebo (control) or treatment group; assurances that participants’ confidentiality will be maintained in the study, in whatever form such data are captured and stored; recruitment is free from coercion or undue influence, particularly for vulnerable populations; and the investigators and ancillary research staff have no financial or non-financial conflict of interest. If established and effective therapy exists, then it should be used in place of the placebo, much like the European drug approval process. If these qualifications are met, then the risk assessment as shown in the table 1 may be known and/or low.

Randomized Controlled Trials

RCTs require the IRB to oversee a small number of investigators at any given time to conduct a clinical study. Usually the investigators are directly associated with the University and are known entities to the IRB Board. CER uses a large number of clinicians each recruiting a relatively small number of patients as opposed to an RCT which uses a small number of clinicians each recruiting a large number of patients. However, important considerations regarding the ethical assessment of risk still apply to randomization of research participants into treatment versus control group in a standard of care study for an RES or RCT (Phase III or IV). Providing full disclosure including fiduciary obligations informs the patients/subjects of their choice to participate in the study despite the fact they will not know which treatment they will be receiving. If these ethical considerations are met, then the assessment of risk for the standard of care RCS or RCT as shown in table 1 is appropriate. All randomized studies conducted to date by PEARL have the additional oversight, NIH directed, of a Data Safety Monitoring Board.

IRB Safeguards

IRB's assess risk, first and foremost risk to the patient, followed by the institution, investigator and sponsor. Inherent in the risk of conducting a study is the disease state, investigator expertise, environment where the study is being conducted and distance of the investigator from the home institution. We exclude rare diseases from this discussion as they pose their own unique set of circumstances.

The design of the study also dictates the complexity of the informed consent. One may want to limit the geographic distances of the practitioners depending on the complexity of the study on a case-by-case basis.

Electronic data capture (EDC) utilizes real time data compliance as a metric of lowering the risk to the site and eventually to the institutional IRB. Monitoring of a site further ensures site compliance to the principles of GCP. PEARL conducts periodic interim analysis as a form of quality assurance. The operations director or the person assigned to oversee the studies should be knowledgeable in both clinical and regulatory principles. How the PBRN recruits its investigators becomes pivotal in assessing risk to a study both for the patient and the University. PBRNs that have some type of screening as well as a credentialing procedure lower the potential for risk. For a PBRN study, the risk assigned to the protocol should never be with a practitioner unfamiliar with the regulatory and/or clinical process. PEARL infrastructure is designed with an experienced individual to oversee the clinical operations and function as an “operational principal investigator” along with a “network practitioner investigator”. This assures both compliance and network representation and places the burden of the IRB administrative compliance onto the PBRN rather than the practitioner. The “art side” of designing a study for a PBRN is to protect the practitioner's time to conduct the study and not unduly disrupt office routine. Having the PBRN operations group act as an intermediary for regulatory compliance removes much of the anxiety and time commitment from the practitioner and their staff.

Should there be a need to investigate the conduct of a study, investigator and/or the office staff PEARL follows the NYU University policy but has the additional depth of having the IRB conduct an independent review of the findings or to conduct their own investigation should the incidence warrant such action. Either way the quality assurance loop is closed so that the clinical studies have complete oversight.

As the pharmaceutical drug approval process gets more personalized, the dependence on biomarkers and surrogate end points may become more accepted in place of laboratory clinical end-points. The new clinical paradigm requires extreme robustness in the interpretation of the data generated, especially related to how these biomarkers affect patient treatment. This robustness in data, from both the patient and practitioner side may translate into very large numbers of study participants for the interpretation of the approval process.

Conclusion

Having the IRB knowledgeable to the concept of a PBRN allows for a better understanding of assessed risk related to a study. CER studies have the potential to be an important component influencing the future of health care policy. Costs as related to pharmaceuticals, devices, diagnostics, all contribute to the translational gap. Practitioners of a PBRN are tethered to a centralized organization, usually a university based affilitation to maintain investigator compliance, credentialing, and/or indemnification. Investigators that are separated from the university by large distances are held accountable by the PBRN Central Network for IRB approval and participation in a study following the principals of GCP. Clinical safeguards put in place for ethical consideration by the PEARL Network may be beneficial. The PEARL infrastructure has addressed the IRB compliance issues related to conducting clinical studies for both government and industry sponsored research.

Figure 3.

PBRN infrastructure based on principle of GCP and IRB role in drug development