Lessons learned conducting a multi-center trial with a military population: The Tinnitus Retraining Therapy Trial (TRTT)

Abstract

Background

The Tinnitus Retraining Therapy Trial (TRTT), a randomized, placebo-controlled, multi-center trial, evaluated the efficacy of tinnitus retraining therapy and its individual components, tinnitus-specific educational counseling and sound therapy versus the standard of care in military practice to improve study participants’ quality of life. The trial was conducted at six US military hospitals to take advantage of the greater prevalence of tinnitus in the military population.

Methods

During the trial, various challenges arose that were uniquely related to the military setting. To convey these challenges to investigators planning future multi-center trials in military hospitals, we itemized various challenges that arose during the trial, interviewed clinic directors and coordinators to elicit their viewpoints, and then collated and organized their responses, together with those challenges presented while conducting the TRTT.

Results

We encountered challenges in site selection, the approval process, administrative issues, study personnel training and retention, participant recruitment methods and issues, adherence to protocol, reimbursement issues, and military security. Site selection involved visiting twenty military hospitals to identify six sites that enrolled and followed study participants. We found that commitment for the trial must be obtained from the full military chain of command, but with ongoing changes in staff or military priorities initial commitments were insufficient to sustain support throughout the entire trial. More time is required to obtain necessary administrative approvals by various military authorities and institutional review boards than is typically experienced in civilian settings. Recruitment strategies must be flexible due to changing military regulations regarding display of materials. Protracted periods of inactivity were due to sequestration and delays in institutional review board approval of required study personnel or protocol amendments. While mostly adherent to the protocol, study staff had difficulties in integrating study visits into the military clinical schedule. Unexpected study expenses revolved around hiring civilian study staff and obtaining associated security clearance while maintaining a consistent flow of funds to each site. The added expense negated cost savings realized by conducting the NIH-funded trial at federal institutions, whose personnel could not be reimbursed for their efforts. Military security concerns impacted the use of web-based data systems and led to increased time and effort required for site visits.

Conclusions

Overall, U.S. military hospitals provide a unique setting to conduct multi-center trials. Challenges arise mainly due to ever-changing authority personnel and military priorities. Pre-planning and flexibility are keys in overcoming these challenges. Multi-center trials conducted in the military will likely take longer to initiate and complete than those in the civilian sector due to multiple levels of command and administrative approvals.

Background

Subjective tinnitus, the perception of sound in the absence of a corresponding external stimulus, has no known medical etiology and is a severe problem for one to two percent of the U.S. population.¹ Because tinnitus diminishes the ability to lead a normal lifestyle,² current therapy, including tinnitus retraining therapy,³ focuses on improving the patient’s quality of life. The Tinnitus Retraining Therapy Trial (TRTT), a randomized, placebo-controlled, multi-center trial, evaluated the efficacy of tinnitus retraining therapy and its individual components, tinnitus-specific educational counseling and sound therapy versus the standard of care.⁴ Compared with the civilian population, tinnitus is more prevalent in military personnel, who are exposed more frequently to intense noises,⁵ a risk factor for tinnitus. Clinical sites in the TRTT were six U.S. military hospitals, including three Navy, two Air Force, and the integrated service site at Walter Reed National Military Medical Center.

In this article we share the challenges encountered in conducting a multi-center randomized trial in U.S. military hospitals. Trials registered in ClinicalTrials.gov with military participants mostly are single site trials.⁶ Our experience may prove useful to others planning multi-center trials across military branches. Lessons learned that may be useful relate to site selection and the approval process, administrative issues, study personnel training and retention, participant recruitment methods and issues, adherence to protocol, reimbursement issues, and military security.

Methods and Results

Identifying, recruiting, and retaining successful sites

Selecting sites for a multi-center trial usually occurs during the planning phase. In contrast to assessing civilian sites by viewing investigators’ experience and potential study populations, the TRTT chair spent considerable up-front time and effort to identify suitable military facilities. Criteria for site selection were similar and included availability of qualified staff and appropriate audiological equipment and accessibility to potential study participants. This effort began with the chair meeting face-to-face with the existing Army, Navy, and Air Force Military Audiology Consultants, who are responsible for overseeing all audiology and hearing centers across their military branch, including knowledge of potential sites and personnel. Consultants are specialty experts and are appointed by each service Surgeon General. The study chair first contacted the Army Consultant, who he knew personally from previous interactions and who in turn provided introductions to the Navy and Air Force Consultants. Based on the Consultants’ recommendations, the chair arranged site visits, which included meetings with potential investigators, administrative research and compliance officials, and leadership at each center. The visit agenda included a presentation of TRTT objectives and protocol, investigator expectations, participant time and effort, advantages in learning an alternative tinnitus treatment, and potential trial impact on site resources. Together the chair and site leadership assessed facility suitability, personnel, potential time and resource commitment, and enthusiasm for the TRTT.

These early planning site visits identified six military sites, which were named as clinical centers in the first trial proposal submission to the National Institute on Deafness and other Communication Disorders (NIDCD). Subsequent submissions reflected site withdrawal due to loss of interest or personnel, change in command personnel, or other reasons (e.g., extensive facility damage following Hurricane Katrina at Keesler Air Force Base), requiring the chair to visit yet more sites.

During the trial, NIDCD approved adding sites to increase recruitment and the chair visited twelve new sites; four initiated the process of becoming a full clinical center. One site was re-defined as two sub-sites, but over a two-year period, neither completed all steps needed to become a site before trial recruitment ended. Two other sites withdrew after completing all steps, but prior to enrolling any study participants; one because command and personnel changes prevented its participation and one because the trial ended. Of twenty potential sites visited, twelve agreed to participate, eight completed all steps to participate in the TRTT, six enrolled study participants, and five completed the trial.

Commitment initially required face-to-face visits and time to establish relationships with personnel at all levels in the chain of command. In the Army and Navy, the Hospital Commander reports directly to the respective Military Surgeon General, while in the Air Force, the chain goes from Hospital Commander to Base Commander, Major Command Commander, and Surgeon General. Even with initial commitment, support could and did change over time with personnel changes and shifting priorities. Although Consultant support was critical for early success, replacement Consultants were often less enthusiastic for the trial. Hospital Commander support provided the best chance for continuity, but did not ensure commitment longevity. For example, at one site, a Research Office official mandated that personnel could spend only 5% of their time on trial effort, making it impossible for that site to enroll new participants.

The major lesson learned is that for a successful trial at a military hospital, commitment must be marshalled from the top down, starting with the senior leadership down to the site investigators. Optimally, support should be garnered prior to grant submission, with accompanying letters of support from Consultants, Hospital Commanders, and site investigators. Likewise, obtain support of local research officials and compliance officers prior to funding requests. Sustained commitment may be impossible due to changes in staffing, competing time priorities, or change in site and higher command authority structures. Others have described the processes of forming academic-military collaborations with similar themes.^7–9 Continued discussions and interactions at all levels during the trial are essential to ensure support continuity. In retrospect, it might have been more useful to have engaged more fully the Surgeon Generals to ensure support continuity across ever-changing military personnel and priorities.

Administration issues – red, white and blue tape

After funding was received, the first administrative step at each site was formal local approval to conduct the trial by the local Research Office and final sign-off by the Hospital Commander. This approval involved review of the study protocol, projected roles and effort levels of personnel and participants in the study, and use of site facilities. The local Research Office considered the trial impact on usual clinical operations and could request minor protocol changes or restrictions in personnel effort. Each iteration of this process could and did take months. One site began but never completed the process because of protracted reviews and corresponding delays over two years. A final approval was required by the branch-level scientific Research Office before the trial could proceed. This final official sign-off often languished on the commanding officer’s desk for weeks or even months.

After participating sites were selected and funding secured, a signed Cooperative Research and Development Agreement (CRADA) was required. A CRADA is a contractual agreement allowing collaboration between a federal institution and civilian institutions to conduct scientific research. Specifically, it delineates the resources provided and responsibilities of each party, and addresses study rationale, funding, required reports, intellectual property rights, liability, and general provisions. The CRADAs allowed the military hospitals to collaborate with the University of Alabama and Johns Hopkins University to conduct the TRTT. Initially drafted collaboratively by the chair’s institutional and military hospital’s legal representatives, it was distributed to legal representatives of each collaborating institution and respective branch command for approval and signature. No study activities could proceed at a site until the CRADA was reviewed and signed by all parties. For Walter Reed National Medical Military Center, the CRADA was administered by the Henry M. Jackson Foundation for the Advancement of Military Medicine, Inc. This third-party agreement supplanted the fiscal administrative role of the chair’s office and institution so that the Jackson Foundation served as a link between the federal funding institution and the military hospital to provide administrative support.

With single site trials often only one CRADA is required. With a multi-site design, approval of the same protocol at multiple sites results in multiple nuanced CRADAs approvals. The CRADAs typically took about a year to complete (range, 6 to 26 months) from initiation to final signed document. During this time, the Federal government implemented the Base Realignment and Closure program, a congressionally mandated process the Department of Defense used to reorganize its base structure. The resulting integration of Bethesda Naval Hospital and Walter Reed Army Medical Center necessitated a new CRADA for this site, increasing the delay. A related issue was the length of the commitment specified by the CRADA. The original CRADAs were tied to the expected five years of funding. When the trial was extended, it was necessary to update each CRADA, including review and signature of the updated CRADA by all parties. If an updated CRADA was not signed before the original CRADA expired, all study activities were temporarily suspended at that site until the CRADA was re-approved.

Another area often taken for granted in multi-centered trials is attendance at full investigative group meetings by site personnel. Notwithstanding an agreement in the CRADA specifying meeting attendance, the study chair was required to extend proffer letters requesting personnel attendance at study meetings. Even with this request and all travel expenses covered by the chair’s office, attendance was sometimes denied by the site commander. Thus, full attendance by all clinic directors and coordinators was achieved infrequently.

In addition to approval of relevant CRADAs, research studies also require Institutional Review Board (IRB) approval. While these two processes can be initiated simultaneously, submission to IRBs was not allowed until the CRADA was fully signed for some sites. A common Defense Medical Research Network (IRBNet) proved valuable for some military sites to share information (e.g., formatting of the study protocol as required by military IRBs, exchanging health-related or trial information across cooperating sites, and transferring information during transfer of a study participant to a second site). Unfortunately, it did not serve as a single IRB for all sites. Each military hospital had its own IRB or shared IRBs only within the same service branch. Some IRBs met infrequently, so approval sometimes took months. Moreover, IRBs tended to be stringent in their requirements. For example, when a clinic coordinator left a clinic, one IRB suspended all trial activities until a replacement was hired and approved by the IRB, and disallowed existing personnel to assume coordinator tasks in the interim. Amendments to the protocol requiring IRB approval also typically took longer than expected at a civilian site.

The major lesson learned is that local site Research Office and branch command, CRADA, and IRB review and approvals take time. Planners of military trials should expect to spend more than a year in obtaining approvals. Williams et al. reported that continuing IRB approval for studies they conducted took anywhere from seven to twelve months.¹⁰ Green et al. reported that approximately 4,680 hours of staff time over a period of 19 months were required for a single IRB approval.⁸ Such time and effort had to be incorporated into staff routine schedules or performed after regular work hours. These extended times for approvals were not exceptions, but typical. Also, even with a CRADA in place, military priorities take precedence over an approved research study.

Staff training and retention

Clinicians at military sites may have no or limited clinical trial research experience. The TRTT represented the first research involvement for many TRTT team members. Only three sites had directors with trial experience, so training included Good Clinical Practice and research integrity along with study-specific training. Continued Good Clinical Practice was still sometimes a problem (e.g., site personnel not understanding the importance of completing every follow-up visit or within the protocol-specified time window).

Staff replacement due to separation from the military or transfer was a major challenge during the trial. Fifty-four persons were trained and certified (documented and verified role-specific competence) at six participating sites, including 10 directors, 19 coordinators, 29 audiologists, and 16 others, including study physicians. The principal investigator at three TRTT clinics was replaced at least once during the trial due to retirement or transfer, and clinic coordinators a median of 2.5 times (range, 1 to 3). The TRTT required a minimum of two audiologists per site to maintain blinding; one to administer treatment and one to complete outcome measurements. Training and certification of 29 audiologists spanned four years (May 2011 to May 2015), with 23 audiologists certified to perform the tinnitus retraining counseling, 22 to perform the standard of care counseling, three certified only to complete audiological and other outcome measures, and three not completing certification. Training and certification presented a resource challenge to the coordinating center and protocol monitors, who mounted five training sessions - two initial, two additional, and a webinar session. While personnel turnover also happens at civilian clinical sites,¹¹ it is a special problem in military hospitals because of military hiring restrictions, exacerbated by inordinate periods needed to obtain security clearance for newly hired personnel. Also, all trial activities could cease when local IRBs required pre-approval of all required study personnel.

The major lesson is that training must be more comprehensive than in a civilian trial, and must include Good Clinical Practice and research integrity. Ongoing training must be anticipated to continue throughout the trial and associated resources built into the budget. Staff replacements are difficult to achieve in a timely manner, so training of back-up key personnel should be anticipated to provide for continuity of trial activities. The transitory nature of military trial staff has been noted by others.^{8, 9} To the extent possible, transition plans for key personnel should be in place before retirement, transfer, or deployment takes place to allow for the seamless transition of responsibilities from one person to another.

Recruitment and enrollment

Typically during a multi-center randomized trial, the study chair or coordinating center personnel develop recruitment materials, such as posters, newspaper or radio ads, and brochures. These materials require approval by site IRBs. However, rules at local military sites regarding TRTT recruitment materials were neither uniform across sites nor consistent over time at individual sites. Recruitment paper posters were not allowed at some sites, yet encouraged at others. Electronic posters were only allowed to be displayed for short periods of time, typically one week. One site was allowed to ‘advertise’ the trial on the hospital billboard, but only for a limited amount of time. One coordinator wrote a description of the TRTT for the staff newsletter, but it required lengthy IRB review and approval before publication. Also, what was allowed one time at a given site was later denied; one site successfully recruited study candidates with paper posters, but a change in command mandated removal of all paper posters throughout the hospital and resulting decline in recruitment.

A second recruitment issue was the definition of “retired” military personnel. Not all persons who served in the military are eligible to be treated in military hospitals. Over 75 persons, who learned of the trial through the ClinicalTrials.gov website, sent an inquiry to the coordinating center and were referred to the relevant military site, but none were eligible. Conversations with personnel at military hospitals clarified the military parlance; the TRTT’s recruitment criterion of “retired and active military personnel” morphed into “persons who have Tricare insurance”.

Throughout the trial, there were multiple periods when recruitment could not take place. Over the three years of recruitment, one site or another was not allowed to recruit study participants during six separate time periods for a total non-functional period of 18 clinic-months. Reasons recruitment could not take place included delayed CRADA re-approval; prolonged periods needed for identifying, training, and obtaining security clearance of newly hired staff; and extended periods waiting for IRB approval of a protocol amendment. Another obstacle that affected all sites was sequestration in 2013 when personnel were required to take furloughs. Although many TRTT staff were military and required to be ‘on duty’ at all times, audiology staff tended to be civilian and could not work. Clinical responsibilities were prioritized and research activities postponed for months before, during and after sequestration; completing trial activities on a voluntary basis was discouraged with a resultant halt in trial recruitment and follow-up visits delays.

On the positive side, the TRTT ended with a study population that was 30% women, 25% non-white and 10% Hispanic, mirroring the U.S. military population. Military sites provide a recruitment resource to enroll these sub-populations. The diversity of the population did create a challenge in that study sound generators used for sound therapy had to be adapted to match skin tones for Hispanic and African-American participants. Also, because the military setting has a unique ethos, participants were sometimes reluctant to express concerns that their tinnitus might be affecting their work performance and everyday activities.

We learned that recruitment strategies had to be focused on the target population and be readily adapted. Flexibility is critical when previously successful methods are no longer permitted or are restricted to a limited time period by the regulations at a particular base. Green et al. found, based upon trials registered in ClinicalTrials.gov, that accrual is no better or worse in the military compared with civilian trials; roughly half of the studies in both groups achieved at least 85% of expected recruitment goals.¹² Bush et al. also compared recruitment in civilian versus military research studies and identified five major differences. The military population is younger and healthier; has more difficulty with transportation, work hours, and release from duty; are more transient and mobile; are more vulnerable to coercion; and for some conditions, more often experience research “burnout”.¹³

Adherence to protocol

Our initial expectation that military personnel and participants would provide excellent adherence to the protocol was generally upheld. Protocol deviations were usually due to constraints imposed by military rules rather than study personnel not choosing to follow the protocol. The recurring theme was that military responsibilities are the priority at all times at all sites. The most prevalent conflict was scheduling study visits. Scheduled follow-up visits could be cancelled at any time due to an unexpected temporary duty assignment for either participants or staff; timely re-scheduling was a problem because of tight schedules in most clinics. Clinical responsibilities were always given priority; TRTT visits had to be fit into the clinic schedule. One clinic simply devoted one afternoon a month to study visits. For all clinics, the first treatment visit was especially problematic. The protocol originally specified that treatment should be within one month of randomization. Clinics often were booked more than a month in advance resulting in a treatment delay. Accordingly, we amended the protocol to increase the allowable interval to two months, but additional protocol deviations accrued while waiting for IRB approval for this amendment. In other areas, military staff were excellent at following the protocol, with only 25 relatively minor protocol deviations occurring mostly during the first six months of the study.

One problem we had not anticipated was participant retention. Withdrawals were treated casually at some sites. For example, when a study participant was transferred to a non-TRTT military base, study personnel simply withdrew that participant from the study rather than try to continue to accrue data through other means. When two other study participants could no longer be treated at a military hospital because they left the military and were no longer covered under Tricare insurance, we asked the clinics to have the participant complete the primary outcome measure by mail. One clinic complied, but the other stated their IRB did not allow them to send questionnaires by mail, even though this option was clearly stated in the TRTT Manual of Procedures.

We learned that other than scheduling conflicts, protocol deviations are few with military personnel providing excellent protocol adherence. It is important to recognize unique obstacles to scheduling and completing study visits in the military. Other investigators have reported the proportion of cancelled military study visits is as high as 26%, frequently last minute, and often duty related.⁹ When recruiting prospective participants, the researcher must consider the possibility of the participant’s changing status due to deployment, transfer, or discharge and subsequent retention problems.

Payroll strategies and changes in funding climate

Funds for clinic study operations were disbursed on a per capita basis, with reimbursement for enrollment, treatment, and follow-up visits. Travel expenses to training and full investigative group meetings were paid directly. Monies equivalent to payment for four study participants were provided initially for start-up administrative costs, with payment triggered by completion of the CRADA. This payment structure created a problem at two sites: 1) funds supported a coordinator who left before recruitment began, leaving no funds to hire a replacement; and 2) the site withdrew from the trial before enrolling any participants. Both resulted in loss of funds, study equipment, and materials to the trial. Although fiscal problems are not unique to the military, they are exacerbated in the military by the difficulties in replacing personnel and in recruiting viable sites.

As a federal site, neither military nor civilian staff are allowed to accept payment for time spent on a federally funded trial, so the per capita payment was utilized differently across clinics. Most clinics used the funds to pay civilian coordinators; one clinic refused to accept payment because of that site’s military bureaucracy; and one used the funds to augment their general research fund. Because of military and federal hiring restrictions, the chair’s office sometimes hired study personnel directly through a professional service agreement with the chair’s institution. Sometimes this meant re-hiring existing directors, audiologists, or coordinators who had separated from the military, but remained physically located near the site. Although already trained and certified, their costs were incremental to the trial budget, placing a strain on study resources. With the per-capita arrangement, gaps in IRB approval or fewer study visits, especially prevalent near the end of the trial, meant sparse payment for coordinators, so the coordinating center and The Hearing Center of Excellence assisted with data entry.

We found that while the per capita system worked reasonably well in the beginning of the trial, it was difficult to ensure a sufficient payment stream throughout the entire trial. It is not clear what monetary model would have worked better, but clearly thought must be given to the impact of different payment strategies throughout the entire trial. Payment of contractual or non-federal personnel hired specifically for a trial may be essential for maintaining requisite trial personnel and activities. These potential costs must be anticipated when planning trial budgets.

Military security

We did not initially appreciate the significance of the enhanced security at military installations. While we knew that civilian staff required security clearance before hospital entry for site visits, measures differed across hospitals. All sites checked identification before access, ranging from display of an official identification to a stop at the gatehouse for an escort on base. Some sites required pre-registration, with visitor requests submitted at least one week prior to the actual visit. Vehicles were also inspected prior to entry at some sites.

Local firewalls on the facilities’ internet limited access to our web-based data system by preventing data entry or downloading study forms at the clinic, access to sites used for scheduling meetings, and inability to open documents sent by email. Study personnel often conducted these activities in public access areas at the military hospital or off-site. No computers or USB devices were allowed to be brought on any site, so paper, rather than electronic monitoring materials, were required for site visits, although some sites did allow CDs or DVDs as storage units.

It is prudent to check with local authorities about security requirements when assessing sites, although we found the enhanced security was mostly a minor nuisance that only nominally affected trial conduct, while promoting enhanced patient confidentiality at all sites.

Conclusions

The military provides a unique setting and potentially reduced costs for NIH-funded trials compared with civilian settings. Military sites offer staff willing to follow clinical trial protocols and resources to track study participants and gather data. Sufficient time and resources must be anticipated, and accommodations made to address chain of command bureaucracy and transient nature of study participants and staff (see Table 1 for our recommendations). Importantly, the trial goal must harmonize with military priorities. Tinnitus was and continues to be a major problem in the military, allowing us to initiate and successfully complete the TRTT.

Table 1.

Recommendations for conducting a multi-center trial in U.S. military hospitals

Obtain commitment for the trial from site investigators and the full chain of command before requesting funding. Initiate CRADA and IRB approval process early and simultaneously, if allowed. Be prepared for military priorities to take precedence over study protocol, especially last minute cancellations of study visits. Include basic research training for trial staff, and anticipate ongoing training due to transitory staff. Provide for redundancy in key personnel. Plan flexible recruitment strategies. For long term studies, make plans for continued care when study participants separate from the military and can no longer be treated at a military hospital. Consider hiring civilian trial staff, but be prepared for lengthy intervals between hiring and clearance. Be prepared for local firewalls on the military hospitals’ internet that may limit access to web-based data systems. Be prepared for the unexpected – furloughs, base closures, and other events.