Translating research into action: An evaluation of the World Trade Center Health Registry's Treatment Referral Program

Abstract

This manuscript describes the design, implementation and evaluation of the World Trade Center (WTC) Health Registry's Treatment Referral Program (TRP), created to respond to enrollees’ self-reported 9/11-related physical and mental health needs and promote the use of WTC-specific health care. In 2009–2011, the TRP conducted personalized outreach, including an individualized educational mailing and telephone follow-up to 7,518 selected enrollees who resided in New York City, did not participate in rescue/recovery work, and reported symptoms of 9/11-related physical conditions or posttraumatic stress disorder (PTSD) on their most recently completed Registry survey. TRP staff spoke with enrollees to address barriers to care and schedule appointments at the WTC Environmental Health Center for those eligible. We assessed three nested outcomes: TRP participation (e.g., contact with TRP staff), scheduling appointments, and keeping scheduled appointments. A total of 1,232 (16.4%) eligible enrollees participated in the TRP; 32% of them scheduled a first-time appointment. We reached 84% of participants who scheduled appointments; 79.4% reported having kept the appointment. Scheduling an appointment, but not keeping it, was associated with self-reported unmet health care need, PTSD, and poor functioning (≥14 days of poor physical or mental health in the past 30 days) (P < 0.05). Neither scheduling nor keeping an appointment was associated with demographic characteristics. Successful outreach to disaster-exposed populations may require a sustained effort that employs a variety of methods in order to encourage and facilitate use of post-disaster services. Findings from this evaluation can inform outreach to the population exposed to 9/11 being conducted by other organizations.

Abbreviations

WTC

World Trade Center

TRP

Treatment Referral Program

PTSD

posttraumatic stress disorder

EHC

WTC Environmental Health Center

MI

motivational interviewing

NYC

New York City

PCL

PTSD Checklist

Introduction

More than a decade after the terrorist attack on the World Trade Center (WTC) on September 11, 2001 (9/11) in New York City (NYC), a substantial number of individuals continue to be affected by disaster-related physical conditions, including asthma, shortness of breath, persistent cough and wheezing, gastroesophageal reflux disease, and mental health conditions such as posttraumatic stress disorder (PTSD) and depression.^1-8

In the aftermath of a disaster, programs may be created or adapted to address both the immediate and longer-term physical and mental health care needs of those directly impacted.⁹ Several programs were implemented in NYC to address short-term mental health needs. One such program, Project Liberty, was funded by the Federal Emergency Management Agency (FEMA) to provide public education and short-term crisis counseling.^10,11 Project Liberty was expanded in 2002 to include enhanced services, such as more intensive or longer-term counseling, for participants with more substantial functional impairment.¹¹

To provide more sustained screening, monitoring, and treatment of 9/11-related physical and mental health conditions among directly exposed persons in NYC, several medical institutions established monitoring and treatment programs.¹² Since 9/11, three Clinical Centers of Excellence have provided 9/11-related physical and mental health services to over 50,000 patients exposed to the disaster.¹² The NYC Health and Hospital Corporation's WTC Environmental Health Center (EHC) was established in 2005 to provide specialized medical care, at no out-of-pocket cost, to eligible individuals who resided, worked, or were present in lower Manhattan on 9/11, referred to as survivors.¹² Rescue, recovery, and clean-up workers and volunteers, referred to as responders, are eligible for no-cost services at either the General Responder Consortium, established in 2002, or the WTC Medical Monitoring and Treatment Program of the Fire Department of New York, which expanded its existing medical monitoring program to include monitoring and treatment for 9/11-related health conditions shortly after the disaster. As of July 1, 2011, these three programs became known collectively as the WTC Health Program for survivors and responders as part of the federally funded James Zadroga Health and Compensation Act of 2010,¹³ which also includes a nationwide network of providers for affected persons living outside the NYC area.

Despite broad outreach efforts by the EHC, the WTC Health Registry (the Registry), and a coalition of community-based organizations, including subway advertising campaigns, only a relatively small proportion of potentially eligible survivors have utilized services available at the EHC. Studies conducted immediately after 9/11 attributed low utilization of mental health services to barriers to care such as survivors’ inadequate finances or time, beliefs that others are in greater need of services or that individuals can care for themselves, mistrust of mental health professionals, and fear of discussing the attacks.^14,15 In focus groups conducted several years after 9/11, Registry enrollees mentioned numerous programmatic and personal barriers to care, including limited knowledge of 9/11-related health conditions, lack of awareness of 9/11-related health care programs, perceived stigmatization of receiving mental health care, and lack of referrals from primary care providers.¹⁶ Participants also were discouraged from seeking care by perceived complicated intake procedures, long waiting lists, and delays in scheduling an appointment.¹⁶

In 2009, in response to enrollees’ 9/11-related health care needs, the Registry collaborated with the EHC to develop the Registry's Treatment Referral Program (TRP). The purpose of the TRP was to identify and contact enrollees with self-reported 9/11-related physical or mental health conditions, and, using special interviewing techniques, encourage them to seek 9/11 specialty care at the EHC program for survivors. The TRP was modeled on outreach conducted after the 2005 London transit bombings.¹⁷ Following the bombings, Brewin and colleagues¹⁷ developed a “screen-and-treat” approach to identify and screen all trauma-exposed individuals, providing referrals to evidence-based treatment as necessary.

This manuscript describes the design, implementation, and process evaluation of the TRP. We sought to understand the main factors influencing TRP participation, defined as contact with TRP staff, and scheduling and keeping appointments. Specifically, we sought to answer the following questions: (1) Were enrollees with self-reported unmet health care need more likely to schedule and keep an appointment than those without? (2) Were physical or mental health symptoms more closely associated with scheduling and keeping an appointment? (3) What demographic characteristics were associated with scheduling and keeping an appointment? Answering these questions is essential to maintaining and improving the effectiveness of this ongoing program.

Methods

World Trade Center Health Registry

The Registry performs epidemiologic surveillance on a cohort study of more than 71,000 individuals directly exposed to the events of 9/11 in NYC via periodic surveys, but does not provide clinical services. Details on recruitment methods have been published elsewhere.^18,19 The Registry comprises rescue/recovery workers and volunteers, lower Manhattan residents, students and staff of schools located south of Canal Street, as well as area workers and passersby present in the area on 9/11. Recruitment was conducted through outreach to eligible groups and individuals and via local and regional media campaigns from 2003 through 2004.^18,19 The 2003–2004 Wave 1 survey included health and exposure information.¹⁸ The 2006–2007 Wave 2 survey provided an update on the health status of adult enrollees (68% response rate) five to six years after the disaster.¹ The Registry protocol was approved by the institutional review boards of the Centers for Disease Control and Prevention and the NYC Department of Health and Mental Hygiene (DOHMH).

Treatment Referral Program

Planning

The purpose of the TRP was to identify and contact Registry enrollees to assess need and eligibility to facilitate referrals for 9/11-related specialty care at the EHC. We used Registry data to screen for enrollees who reported physical and mental health symptoms shown in the literature to be associated with 9/11 exposure and provided referrals as appropriate. Referral services were also provided to enrollees’ family members, friends, and colleagues.

We were aware that our target population might be difficult to reach, vulnerable, reluctant to seek care, and unlikely to initiate and complete the EHC appointment process on their own. We employed personalized mailings based on previous Registry experience in which personalized mailings yielded a greater response than generalized mailings. Personalized mailings were individually addressed and included a personalized greeting (e.g., “Dear Michael”) rather than a generic one (e.g., “Dear WTC Health Registry Enrollee”). To prepare TRP staff prior to the program's launch, we provided a two-day, scenario-based training in motivational interviewing (MI), a goal-focused counseling and communication style that helps individuals identify and address ambivalence.^20,21 MI techniques useful in TRP encounters included the use of open-ended questions and reflexive listening. Staff practiced MI skills during weekly team debriefings and attended a two-day advanced MI training during year two of the program.

Targeted enrollees

English-speaking adult Registry enrollees who were not rescue/recovery workers were screened for TRP outreach on the basis of their most recently completed survey (Wave 2 or Wave 1 if Wave 2 was not completed). Active outreach in this phase of the TRP did not include responders because the TRP was funded through an agreement with the EHC's program for survivors. Additionally, because 9/11-specific health care services were only available in NYC at the time of the TRP's launch, this phase of TRP outreach was limited to enrollees who resided in NYC.

In consultation with the medical director of the EHC, we defined TRP eligibility based on the presence of mental and/or physical symptoms that were associated with 9/11 exposure and for which treatment was covered at the EHC. The EHC would subsequently determine if their symptoms/conditions were, in fact, related to 9/11 and the enrollee was eligible for continued treatment beyond the initial evaluation. Enrollees who had completed the Wave 2 survey were targeted for TRP outreach if they reported at least one 9/11-related physical symptom (persistent cough, headache, dyspnea, sinus problems, or wheezing) at both waves and/or probable PTSD at Wave 2. Enrollees who completed only the Wave 1 survey were included if they reported at least two new or worsening physical symptoms and/or probable PTSD. Probable PTSD was assessed at Waves 1 and 2 using the PTSD Checklist (PCL-17), a self-reported, 17-item scale corresponding to the symptom criteria in the DSM-IV.²² Enrollees with a cumulative score of 44 or greater on the PCL-17 were considered to have probable PTSD. Enrollees not completing sufficient questions on the PCL-17 to determine PTSD status were considered to have unknown PTSD status. Targeted enrollees were then placed into one of four TRP eligibility categories: those with physical symptoms only; physical symptoms and PTSD; PTSD only; or physical symptoms and unknown PTSD status.

All enrollees younger than 18 y of age at Wave 2 received outreach as detailed above via their parent or guardian whether or not symptoms were reported on their most recent survey; however, children were not included in this evaluation due to small sample size.

Outreach activities

TRP outreach was conducted from November 2009 to August 2011 via staged mailings and phone calls. Targeted enrollees received a personalized mailing (letter, brochure, and postage-paid reply card) to provide them with an opportunity to consider their own 9/11-related health issues and the use of a 9/11 health care program. Next, enrollees were contacted by an external call vendor that made daytime, evening, and weekend calls to confirm receipt of the mailing and screen for interest in assistance with accessing health care services. Interested enrollees were placed on a list for callback by a TRP staff member (either a public health nurse or licensed social worker). Periodically, vendor calls were monitored by a TRP staff person for quality assurance purposes. Enrollees were also able to contact the TRP on their own (self-referral) by phone or by returning the reply card. Although only targeted enrollees received the personalized mailing and telephone call, program brochures were sent in English, Spanish, and Chinese to all Registry enrollees when the TRP was launched. All enrollees who contacted the TRP, including responders, were provided with referral services whether they were targeted for outreach or not.

Upon contact, TRP staff provided enrollees with information on 9/11-related health conditions and services offered at the EHC and other 9/11 programs, screened for EHC eligibility, and completed an EHC intake form as needed. Staff used MI techniques to identify psychosocial and logistical barriers to accessing care and subsequently aided in addressing these concerns. Enrollees were re-contacted by TRP staff to schedule an appointment after the EHC had processed their intake forms.

Follow-up

TRP participants who scheduled appointments at the EHC received a TRP reminder call one to three days before their appointment and a follow-up TRP call one to three days after their scheduled appointment to ask if they had kept their appointment or needed further assistance. Both reminder and follow-up calls were made by a TRP staff member who made an average of four call attempts and left two voice messages for each enrollee.

Sample

Of 66,138 English-speaking adult enrollees, 36,541 were survivors, of whom 9,130 were potential candidates for TRP outreach based on self-reported physical and/or mental health symptoms (Fig. 1). We attempted to contact a total of 8,098 enrollees after excluding 146 who had withdrawn from the Registry or were deceased and 886 who were not contacted because outreach was deferred due to a temporary lack of mental health provider availability in the final three months of this phase of the TRP. Enrollees for whom outreach was deferred were placed on a list for future contact during the second phase of the TRP. Enrollees who were not targeted for personalized outreach, but who received services from the TRP, were not included in this evaluation (e.g., survivors who did not meet TRP eligibility criteria at the time of their last completed survey or responders).

Figure 1.

Treatment Referral Program (TRP) sample. ^aPersons participating in rescue, recovery, and clean-up work between 9/11/01 and 6/30/2002 at a WTC site. ^bPersons who lived, worked, attended school or daycare, or were otherwise present in the lower Manhattan disaster area on 9/11 or for extended periods in the months thereafter. ^cEnglish-speaking adult enrollees, who reside in New York City (NYC), did not participate in rescue/recovery work, and reported probable posttraumatic stress disorder (PTSD) and/or select physical symptoms at the time of their most recently completed survey. ^dEnrollees who had withdrawn from the Registry, were deceased, or were deferred from participation due to a temporary lack of mental health provider availability. ^e64 enrollees without information in Kept Appointment.

Outcomes

Three nested outcomes were assessed in the process evaluation: (1) conversion to TRP participant, (2) appointments scheduled, and (3) appointments kept. Enrollees were considered TRP participants if they were placed on the vendor's list of interested enrollees for callback from TRP staff, responded to the mailing by calling the TRP, or returned the reply card included with the mailing (self-referred). TRP participation represented an opportunity to educate enrollees about 9/11-related health conditions and services as well as maintain long-term engagement in the Registry. TRP participants were considered to have scheduled appointments if they made an appointment at the EHC or another program with the assistance of a TRP staff member. Those who scheduled appointments were categorized as having kept or not kept their appointments based on self-report during post-appointment follow-up calls. We were unable to obtain confirmation of kept appointments from the EHC due to privacy restrictions. Enrollees who were not reached for follow-up were considered missing for this outcome.

Covariates

Demographic variables examined were gender, race/ethnicity, age, 2002 household income, education at Wave 1, and marital status. Enrollees responding yes to the Wave 2 question “Did you not get the health care you needed during the last 12 months?” were considered to have self-reported unmet health care need. Enrollees reporting 14 or more poor mental health and/or poor physical health days during the 30 d preceding Wave 2 were considered to have poor daily functioning.

Analysis

Descriptive statistics were calculated for demographic and health characteristics of the study population according to each of the study outcomes. Chi-square statistics were used to assess the statistical significance of associations. An a priori α level of 0.05 was used for all statistical tests. Analyses were performed using SAS Version 9.2 (SAS Institute Inc., Cary, NC).

Results

Contact Attempt

We attempted to contact 8,098 enrollees, of whom 580 (7.2%) were unreachable (mailing returned without forwarding information and/or unreachable by phone or unable to leave a message), yielding a target population of 7,518 (Fig. 1). Enrollees who were non-White, under 45 y of age, never married, had a household income under $50,000, had less than a college education, and had PTSD were significantly less likely to have had valid contact information when compared with their counterparts.

TRP Participants

Of the 7,518 enrollees with valid contact information, 1,232 (16.4%) participated in the TRP. The main modes of contact with the program were the call vendor's callback list (53.2%) and the reply card (34.0%), followed by self-referral by phone to the TRP (10.7%) and referrals through other Registry activities (2.1%) (e.g., tobacco cessation program or in-depth studies) (Table 1). Among enrollees returning the reply card, 42.5% requested a call from a TRP staff member; 27.0% indicated they were already receiving care at the EHC or from another provider; 14.6% said they would contact the EHC themselves; 9.3% requested a copy of the DOHMH publication “Clinical Guidelines for the Treatment of Adults Exposed to the WTC Disaster”; 1.2% wanted to refer a friend; and 5.5% were not interested. Enrollees who did not contact the TRP directly in response to the mailing were considered TRP nonparticipants (n = 6,286) if they were not reached by the call vendor (68.5%), refused to speak with the call vendor (1.8%), or told the call vendor they did not want to participate in the TRP (29.7%).

Table 1.

Mode of contact with Treatment Referral Program (n = 1,232)

Mode of contact	Number (%)
Call vendora	655 (53.2)
Returned reply card included with personalized mailing	419 (34.0)
Self-referredb	132 (10.7)
Otherc	26 (2.1)

^aParticipant was on a callback list from the call vendor; ^bParticipant initiated phone contact in response to personalized mailing; ^cParticipant contacted the Treatment Referral Program via other Registry activities

Participation in the TRP was higher among targeted enrollees who were Black or Hispanic, aged 45 y or older, had an income less than $25,000, or were divorced/separated or widowed compared with their counterparts (Table 2; P < 0.05). TRP participation was also greater among enrollees with self-reported unmet health care need, PTSD with or without physical symptoms, compared with those with physical symptoms only, and among those who reported poor daily functioning compared with those who did not (Table 3; P < 0.05). The majority of enrollees targeted for outreach had physical symptoms only (55.1%); however, these enrollees were the least likely to participate in the TRP.

Table 2.

Demographic characteristics of enrollees who received outreach for the World Trade Center Health Registry Treatment Referral Program (TRP)

		TRP Participant	Nonparticipant
		(n = 1,232)	(n = 6,286)
	Total	Number (%)	Number (%)	p
Gender
Male	2,468	388 (15.7)	2,080 (84.3)	NS
Female	5,050	844 (16.7)	4,206 (83.3)
Race/Ethnicity
Non-Hispanic White	3,835	597 (15.6)	3,238 (84.4)	<0.05
Non-Hispanic Black	1,619	302 (18.7)	1,317 (81.4)
Hispanic or Latino	1,225	202 (16.5)	1,023 (83.5)
Asian	481	67 (13.9)	414 (86.1)
Multiracial/other	358	64 (17.9)	294 (82.1)
Age at Wave 1
18–24	277	25 (9.0)	252 (91.0)	<0.0001
25–44	2,992	391 (13.1)	2,601 (86.9)
45–64	3,742	721 (19.3)	3,021 (80.7)
65+	450	90 (20.0)	360 (80.0)
Household income (2002)
$24,999 or less	972	195 (20.1)	777 (79.9)	0.0001
$25,000-$49,999	2,164	373 (17.2)	1,791 (82.8)
$50,000-$74,999	1,415	248 (17.5)	1,167 (82.5)
$75,000 or greater	2,168	303 (14.0)	1,865 (86.0)
Marital status at W2
Married/living with partner	3,263	524 (16.1)	2,739 (83.9)	<0.0001
Divorced/separated/widowed	1,362	305 (22.4)	1,057 (77.6)
Never married	1,574	259 (16.5)	1,315 (83.6)

Table 3.

Health characteristics of enrollees who received outreach for the World Trade Center Health Registry Treatment Referral Program (TRP)

		TRP participant (n = 1,232)	Nonparticipant (n = 6,286)
	Total	Number (%)	Number (%)	p
Self-reported unmet health care need at Wave 2
Yes	1,024	256 (25.0)	768 (75.0)	<0.0001
No	5,227	845 (16.2)	4,382 (83.8)
TRP category
Physical symptoms only	4,141	606 (14.6)	3,535 (85.4)	<0.0001
Physical symptoms and posttraumatic stress disorder (PTSD)	2,699	518 (19.2)	2,181 (80.8)
PTSD, no physical symptoms	646	105 (16.3)	541 (83.8)
Physical symptoms, PTSD unknown	32	3 (9.4)	29 (90.6)
Poor daily functioning at Wave 2 (mental health)
Yes	1,627	363 (22.3)	1,264 (77.7)	<0.0001
No	4,524	712 (15.7)	3,812 (84.3)
Poor daily functioning at Wave 2 (physical health)
Yes	1,313	301 (22.9)	1,012 (77.1)	<0.0001
No	4,834	774 (16.0)	4,060 (84.0)

Scheduled appointment

Overall, 399 (32.4%) TRP participants scheduled a first-time appointment at the EHC. The majority (68.9%) of participants who contacted the TRP themselves scheduled an appointment (Table 4). Less than one-third (31.5%) of participants contacted via the call vendor scheduled an appointment; however, this group accounted for the largest percentage (51.6%) of all appointments scheduled. Scheduling an appointment was significantly associated with unmet health care need, PTSD with or without physical symptoms, and poor daily functioning (P < 0.05; Table 5). Participants reporting unmet health care need, PTSD symptoms, or poor daily functioning were significantly more likely to make an appointment compared with their counterparts.

Table 4.

Demographic characteristics of Treatment Referral Program (TRP) participants who did and did not schedule an appointment with the World Trade Center Environmental Health Center

		Scheduled an appointment	Did not schedule an appointment
	Total	(n = 399) Number (%)	(n = 833) Number (%)	p
Source of referral
Self-referred	132	91 (68.9)	41 (31.1)	< 0.0001
Returned reply card	419	92 (22.0)	327 (78.0)
Call vendor	655	206 (31.5)	449 (68.6)
Other	26	10 (38.5)	16 (61.5)
Gender
Male	388	131 (33.8)	257 (66.2)	NS
Female	844	268 (31.8)	576 (68.3)
Race/ethnicity
Non-Hispanic White	597	179 (30.0)	418 (70.0)	NS
Non-Hispanic Black	302	104 (34.4)	198 (65.6)
Hispanic or Latino	202	77 (38.1)	125 (61.9)
Asian	67	23 (34.3)	44 (65.7)
Multiracial/other	64	16 (25.0)	48 (75.0)
Age at Wave 1
18–24	25	8 (32.0)	17 (68.0)	NS
25–44	391	128 (32.7)	263 (67.3)
45–64	721	240 (33.3)	481 (66.7)
65+	90	21 (23.3)	69 (76.7)
Household income (2002)
$24,999 or less	195	73 (37.4)	122 (62.6)	NS
$25,000-$49,999	373	130 (34.9)	243 (65.2)
$50,000-$74,999	248	70 (28.2)	178 (71.8)
$75,000 or greater	303	90 (29.7)	213 (70.3)
Marital status at Wave 2
Married/living with partner	524	162 (30.9)	362 (69.1)	NS
Divorced/separated/widowed	305	98 (32.1)	207 (67.9)
Never married	259	90 (34.8)	169 (65.3)

Table 5.

Health characteristics of Treatment Referral Program (TRP) participants who did and did not schedule an appointment with the World Trade Center Environmental Health Center

		Scheduled an appointment	Did not schedule an appointment
	Total	(n = 399) Number (%)	(n = 833) Number (%)	p
Self-reported unmet health care need at Wave 2
Yes	256	101 (39.5)	155 (60.6)	<0.01
No	845	254 (30.1)	591 (69.9)
TRP category
Physical symptoms only	606	162 (26.7)	444 (73.3)	<0.01
Physical symptoms and posttraumatic stress disorder (PTSD)	518	197 (38.0)	321 (62.0)
PTSD, no physical symptoms	105	39 (37.1)	66 (62.9)
Physical symptoms, PTSD unknown	3	1 (33.1)	2 (66.7)
Poor daily functioning at Wave 2 (mental health)
Yes	363	134 (36.9)	229 (63.1)	<0.05
No	712	215 (30.2)	497 (69.8)
Poor daily functioning at Wave 2 (physical health)
Yes	301	116 (38.5)	185 (61.5)	<0.01
No	774	234 (30.2)	540 (69.8)

Kept appointment

We reached most (84%) participants within one to three days of their EHC appointment date for follow-up. There were no significant differences in demographic or health characteristics between those reached for follow-up and those not reached. Of 335 participants reached, 266 (79.4%) reported having kept their appointment, which is consistent with feedback we received from the EHC as to whether or not enrollees who scheduled their appointment via TRP kept them. Although participants without PTSD were the least likely to schedule an appointment, they were the most likely to have kept their appointment among participants reached for follow-up (P = 0.05; data not shown). No other demographic or health characteristics were significantly associated with keeping an appointment.

Discussion

This evaluation was conducted to understand the main factors associated with TRP participation as well as scheduling and keeping an appointment at the EHC. We found that participants with PTSD were the most likely to schedule an appointment; however those without PTSD were more likely to have kept their appointments. This suggests that while persons with PTSD may have recognized their need for treatment or may have not already been in treatment, symptoms of avoidance and other factors may have prevented them from accessing care. As such, the persons most traumatized and likely most in need of care may, in fact, be the most difficult to engage in clinical services. It is our recommendation that future plans targeting disaster exposed populations plan for those with PTSD by including intensive outreach and supportive services.

Additionally, those with unmet health care need were more likely to make an appointment than those without. Assessing unmet need may be a useful factor to consider when planning outreach activities. Similarly, the likelihood of scheduling an appointment was also influenced by impaired functioning, as those reporting poor physical or mental daily functioning were more likely to schedule an appointment. Individuals who contacted the TRP on their own were more likely to schedule and keep their appointments compared with individuals contacting the TRP through the call vendor, reply card, or other Registry activities. It is likely that enrollees taking steps to seek care after receiving the mailing were highly motivated and proactive about completing the process.

Based on reporting from the EHC, an additional 496 enrollees who were part of the TRP target population contacted the EHC without further assistance from the TRP, completed an intake, scheduled an appointment, and cited the TRP as their referral source. Privacy restrictions prohibited us from identifying these persons for inclusion in this analysis; however, the timing of the appointments coincided with the staged TRP mailings, leading us to assume that they might be recipients of the personalized TRP mailing. Because the TRP was integrated into daily Registry activities, including communications with individual enrollees, an additional 202 enrollees not originally TRP eligible because they did not meet symptom or residency criteria at the time of their last survey were referred to the TRP and scheduled an appointment with the help of a TRP staff member. As such, the TRP was the largest source of referrals to the EHC from October 2009 through September 2011. During this time, 2,395 individuals completed an EHC intake and were eligible to schedule a first-time appointment, of whom, 1,097 were Registry enrollees.

Based on discussions from our weekly debriefings, many enrollees who had not sought care at a 9/11 program prior to TRP contact were unaware of such programs or their eligibility for these services. Enrollees also reported personal and logistical barriers to seeking care, as expected based on previous studies of health care seeking among persons exposed to the 9/11 disaster.^14-16 Similarly, Brewin et al.²³ found that individuals exposed to the London transit bombings were reluctant to refer themselves for treatment, and very few London general practitioners referred persons to their post-disaster program, despite being encouraged to do so. Additionally, seeking mental health treatment after 9/11 was particularly rare among those without prior experience using mental health services,¹⁴ and, in general, only a small percentage of disaster survivors seek out treatment themselves.²⁴ The use of MI techniques by TRP staff may have been crucial to overcoming some of these obstacles by enabling frank discussions with enrollees about their 9/11 experiences, health concerns, fears, and barriers to care, and allowing enrollees to make better informed decisions about service utilization.

Although both our program and the London program had a similar goal of connecting those in need to available post-disaster clinical services, several differences exist. The sole purpose of the TRP was to provide referrals to clinical services, whereas the purpose of the London group was not only to conduct outreach but to provide clinical services as well. As a result, unlike TRP, Brewin and colleagues were able to include diagnoses, quality of care, treatment modalities and outcomes in addition to processes in their evaluation. This indicates an area of future research for the 9/11 clinical programs. Moreover, the London program took place within the context of the National Health Service, in which the treatment services provided were essentially an extension of existing services. Service utilization is likely to be vastly different in this context, as affected persons were able to seek free care immediately following the bombings at institutions close to their homes or workplaces and could continue to receive free care from their customary provider. In contrast, with regards to 9/11, free clinical services were limited immediately after the disaster and new programs had to be created. As a result, knowledge of programs was and continues to be low among both affected persons as well as general practitioners. Furthermore, in order to receive services at no cost, affected persons have to prove their eligibility, which in many cases, is a major barrier to accessing care. Finally, the newly established programs are often located far from their homes or workplaces.

Strengths and limitations

Study limitations include the self-reported nature of the data and the lack of access to medical records to confirm missed and kept appointments, diagnoses, and treatments received. Although we know that a portion of enrollees reported unmet needs, we do not know the insurance status of enrollees and subsequently do not know the role this might have played in the utilization of 9/11 specialty care. We could not reach a significant number of potentially eligible enrollees due to a lack of valid contact information. To minimize losses to follow-up, the Registry continually conducts activities to maintain updated contact information, including surveys, annual holiday cards, and periodic mailings requesting updated information.

This evaluation also has several strengths. It was designed from the outset as part of the TRP and was embedded in TRP activities. In addition, Registry data allowed for the examination of predictors of program participation, including demographic characteristics as well as self-reported physical and mental health.

Conclusion

This evaluation underscores the public health importance of conducting targeted, personalized outreach to exposed populations many years post-disaster. The TRP was a novel undertaking for an observational cohort study, highlighting the Registry's commitment to promoting the health and well-being of its enrollees and demonstrating how a post-disaster registry can implement outreach and education for health promotion. Although resource intensive, the TRP promoted long-term engagement with the Registry and helped hundreds of enrollees access 9/11-related care. The TRP is a core Registry function and has expanded its scope to include responders and enrollees residing outside the NYC area.

Our close collaboration with the EHC and constant monitoring of the TRP process enabled us to make adjustments to our approach and calibrate the volume of outreach with EHC's capacity as needed. We encourage those considering implementing similar outreach programs to maintain strong relationships with their clinical partners. Furthermore, the program was designed to include the use of personalized mailings and MI to help ensure that staff encounters with enrollees were maximally effective. We suggest that those conducting outreach to similar populations considering including MI as part of their staff training and using personalized mailings.

Although we successfully connected hundreds of enrollees to care, large numbers of targeted enrollees with self-reported health problems and unmet health care need did not participate in the TRP or were not reached. This includes responders living in the NYC area, as well as responders and survivors living outside NYC for whom services are now available through the federal WTC Health Program. The Registry is conducting a second phase of the TRP that includes outreach to responders and persons who live outside NYC, as well as survivors in NYC, to encourage them to apply for care with the federal WTC Health Program created by the 2011 Zadroga Act. Findings from this evaluation are being used to plan the next phase of the TRP and can inform outreach to the population exposed to 9/11 being conducted by other organizations.

Disclosure of Potential Conflicts of Interest

No potential conflicts of interest were disclosed.

Acknowledgments

The authors wish to thank the following people for their invaluable support and assistance with the Treatment Referral Program: Terry Miles, Scott Penn, Ruchel Ramos, Joan Reibman and the staff of the World Trade Center Environmental Health Center; Joann Fields, Daniel Wallingford, Odeliya Harel, Lennon Turner and Sara Miller Archie; Jeffrey Hon, Sharon Perlman and Lorna Thorpe; Britt Power, Dawn Hoffman, Chris Stella and the staff of Global Strategy Group; Jonathan Fader; and Adam Karpati, Trish Marsik, Monika Eros-Sarnyai, Pablo Sadler, Gail Wolsk, and Gerald Cohen.

We are grateful for the helpful advice and comments from Carolyn Greene, Steven Stellman, Deborah Dowell, Christina Norman, Margaret Millstone, and Rhoda Schlamm on this manuscript.

Funding

This study was supported by Cooperative Agreement Numbers 1E11OH009630 and 5U50/OH009739 from the National Institute for Occupational Safety and Health (NIOSH) of the Centers for Disease Control and Prevention (CDC); U50/ATU272750 from the Agency for Toxic Substances and Disease Registry (ATSDR), CDC, which included support from the National Center for Environmental Health, CDC; and by the New York City Department of Health and Mental Hygiene (NYC DOHMH). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of CDC.