Using Electronic Health Record Data to Study Latino Immigrant Populations in Health Services Research

Abstract

The study of healthcare disparities in Latino immigrants is underdeveloped and limited by risk to participants. To validate an electronic health record (EHR)-based algorithm that could serve as a safe proxy for self-reported immigration status for health services researchers. Primary collection/analysis of interview data and secondary analysis of electronic health record data. We developed an EHR algorithm to classify a population of patients as likely undocumented or recent Latino immigrants and validated this algorithm by conducting semi-structured interviews of staff whose main role entails asking about immigration status. We presented them with a list of patients (masked to the interviewer) with whom they had worked, and asked them to indicate patient’s immigration status, if they recalled it. We analyzed the correspondence between staff knowledge and our EHR algorithm. Staff described routine conversations with patients about immigration status. The EHR algorithm had fair agreement (66.2%, 95% CI 57.3–74.2) with staff knowledge. When the staff were more confident of their assessment, agreement increased (77.6%, 95% CI 63.4–88.2). The EHR has potential for studying immigration status in health services research, although more study is needed to determine the accuracy and utility of EHRs for this purpose.

Introduction

Latino immigrants, a rapidly growing part of the US population, experience important health and healthcare disparities [1–13]. Healthcare insurance facilitates healthcare utilization, however, undocumented and many recently “documented” immigrants (within 5years of arrival) are barred from federal insurance programs, such as Medicare and Medicaid [14]. While undocumented and recent immigrants may be particularly at-risk for health disparities, measuring the extent of this problem is difficult. Relative to other disadvantaged populations, there are few well-established and reliable methods for accurately identifying immigration status among the Latino immigrant population, and this is especially true for those who are undocumented. Lacking this information hampers the development of targeted initiatives that could mitigate health inequities.

Survey and interview methods are commonly used to determine immigration status when studying health outcomes and healthcare utilization. This includes methods that use large, random telephone surveys [1, 2] and in-person neighborhood-based surveys [15–20] to gather these data. Yet, these methods have a number of limitations: they are labor intensive, risk social desirability bias, and may feel unsafe for participants, which diminishes the credibility of the data. For example, federal [21–23] and state [24–26] policies have created an environment where immigrants may alter their use of vital public services and may perceive disclosing their immigration status as risky [27]. This has borne out in smaller qualitative studies which have directly examined immigrant’s avoidance of health care and other services and found reticence/fear in disclosing immigration information. Cleveland et al. found that many immigrants would only speak with bilingual, bicultural interviewers in public places, thereby not revealing where they lived [19]. Other similar works have demonstrated that many Latino immigrants forego contact with routine services for themselves or their children because of fear of immigration authorities [16, 17, 19, 28]. This suggests that direct questioning of patients/subjects about immigration status is not likely a safe or accurate method for eliciting information. In addition, these methods are difficult to scale up to the level necessary for population-level health services research.

Although immigrants may forgo contact with certain providers, there is evidence to suggest that undocumented and recent immigrants frequently utilize care at federally qualified health centers, (#41 and #42 will go here eventually) providing a possible access point for health services research in this population. Federally qualified health centers care provide healthcare to millions of US people, regardless of patients’ ability to pay and a majority of patients are low-income and ethnically diverse (Cite NACHC chartbook).

Additionally, electronic health records (EHRs) may be an alternative data source for studying disparities among undocumented Latino immigrants, one which overcomes the challenges of scale and self-reporting bias. EHRs are increasingly common, often contain demographic data, such as race, ethnicity, and income level, and these data are commonly utilized in health services research [29–32], including studies that likely include high proportions of Latino immigrants [33]. Demographic data in the EHR are collected in the process of routine health care delivery, and because they are not directly related to immigration status, disclosure of these demographics may be less risky. The direct recording of immigration status in the medical record is generally not done for a variety of ethical reasons [34]. In this proof of concept study, we developed and validated an EHR based algorithm, using demographic data, to identify Latino patients with a high probability of being a recent/undocumented immigrant that could serve as a proxy for immigration status for health services research. We hypothesized that staff employed by Federally Qualified Health Centers (FQHCs) and working with patients on gaining access to financial programs (e.g., health insurance) could serve as a potential gold-standard to assess the validity of the EHR-based algorithm we developed, and that our algorithm would agree with their assessment.

Methods

Immigration Status Algorithm

We combined a set of EHR-based demographic variables to serve as a proxy for undocumented and recently “documented” (within 5years of arrival) immigrant status among populations served by Federally Qualified Health Centers (FQHCs). Based on a knowledge of Oregon Medicaid policy, we hypothesized that patients meeting each of the following five criteria during our study period (November 1, 2014–February 28, 2015) have a high probability of being a recent or undocumented immigrant:

1. Oregon FQHC patient.

2. Household income under 100% of the Federal Poverty Level before 1/1/2014 and always under 138% on or after 1/1/2014 (income level that would correspond to Medicaid eligibility in Oregon [36]) in all instances when such data was recorded in the EHR.

3. Hispanic/Latino or primary language Spanish (both as self-identified to clinic).

4. Age > 18 (Medicaid eligibility for children may differ from adults).

5. Uninsured at all visits/encounters available in the record.

The demographic data required for the above algorithm are typically collected by FQHCs, easily abstracted from the OCHIN EHR system, and are required for purposes of their federal funding [35].

Setting

OCHIN FQHCs were the setting for this study. OCHIN (previously the Oregon Community Health Information Network, shortened to OCHIN as other states joined) is a non-profit health information company that hosts a linked EHR for community health centers and other similar safety net clinics serving ~ 2 million patients in 23 states.

Sample/Participants

“Eligibility specialists” are individuals with expertise in patient relations who meet with patients to evaluate eligibility for public assistance programs, and help them apply for these programs. We recruited a convenience sample of twelve eligibility specialists from eight OCHIN member FQHCs in Oregon. We interviewed eligibility specialists to determine if performing their tasks involved asking patients about citizenship and residency status, and to assess how confident eligibility specialists were in the information they received from patients.

Data Collection and Analysis

Interviews were conducted in person (by JH). Interviews were conducted with the eligibility specialists between 1/1/2015 and 5/31/2015, followed a guide (see Appendix A) and were designed to (1) understand the job and tasks in which eligibility specialists engaged, (2) identify their experiences caring for immigrant clients and (3) ascertain whether or not the eligibility specialists had knowledge of patients’ immigration status and their confidence in this information.

In addition, eligibility specialists were given a list of adult Latino patients whose charts they had viewed in the last 3months. Participants had a list with patient names, but the interviewer was blinded to this information to maintain confidentiality. Eligibility specialists were asked to look through the list and identify names they recognized. Among those that were recognized, the eligibility specialists then answered questions about each patient: whether or not this patient was a recent or undocumented immigrant and their level of confidence in that assessment.

Interviews were recorded and professionally transcribed and entered into Atlas.ti for data management and analysis. Responses to questions about patient immigration status were securely linked to the patient’s electronic health record data by a unique identifier (not known to the primary investigators). These responses, along with deidentified EHR data, were securely transmitted to the primary study institution for analysis.

Interview data was analyzed by a multidisciplinary team using an immersion-crystallization approach. This team started by listening to and analyzing interviews together to identify themes related to eligibility specialists’ roles and experiences working with Latino patients. Once a code book was developed, data were analyzed independently. In the second cycle, the team made comparisons across eligibility specialists to understand their approach to working with undocumented immigrants, the relationships and trust these clinical staff developed with patients, and how confident they were about the immigration status of the patients they worked with.

For the survey and EHR data, we described characteristics of the patients recalled by the eligibility specialists and those not recalled. In patients that the eligibility specialists assessed immigration status, we applied the EHR algorithm to produce an EHR assessment of immigration status. Utilizing the eligibility specialist assessment as the gold standard, we then calculated statistics of correspondence (i.e. agreement, sensitivity, specificity and prevalence-adjusted bias-adjusted kappa) between the EHR algorithm and the eligibility specialist assessment, overall and stratified by eligibility specialist confidence level of assessment. Statistical analyses were done using R version 3.4.0. This study was approved by the Institutional Review Board (IRB) of Oregon Health and Science University. Informed consent was obtained from the staff interviewees. Given that patient data was de-identified, a waiver of consent was obtained as part of IRB approval.

Results

Eligibility specialists were employed in clinics serving underserved patients in urban, rural and semi-rural areas in Oregon. Ten eligibility specialists were female, two were male, 10 were fluent in Spanish and English, and two were not Spanish speaking.

Eligibility Specialists Ask About Immigration Status

Eligibility specialists’ do not document immigration status in the EHR, but it is their job to help patients complete the documents necessary for enrolling in insurance programs and applying for publicly available services. As part of this work, eligibility specialists described routine, universal conversations about immigration status with patients. For instance, “I don’t assume anything when I see somebody. So, I’ll ask anybody, basically who’s going to be filling out an application, whether they’re a US citizen. Yes or no? Then, no. Then I ask if they are a legal permanent resident. Then we go from there…”(Clinic 12) Most eligibility specialists described direct, sequential questioning couched in careful explanations of why information is being requested.

Eligibility Specialists Confident in Collection of Immigration Information

Eligibility specialists expressed a high degree of confidence that they collected correct immigration information about their patients. Multiple eligibility specialists reported that they could not remember a time when they did not successfully elicit necessary immigration information: “Every now and then, I’ll come across a family that is hesitant. And so, I repeat over and over the fact that it is all confidential. This information is only for this purpose and so on. And I always seem to get the answer.” (Clinic 1) For this reason, we believe that when an eligibility specialist remembered a patient by name that his/her assessment of a patient’s immigration status would be a reasonable gold standard for testing the validity of EHR-based results.

Measures of Correspondence Between EHR Data and Eligibility Specialist

We identified 441 adult Latino patients whose chart had been accessed by the eligibility specialists in the 3months prior to interviews. Of these patients, the eligibility specialists recognized 130 (31%) that they recalled sufficiently to answer the follow-up survey questions about immigration status and level of confidence. Table 1 describes the demographic characteristics of patients selected for eligibility specialist review. Of note, aside from insurance status, patient characteristics did not differ between patients recalled by the specialists and patients not recalled to memory.

Table 1.

Demographic characteristics of patients by eligibility specialist’s ability to recall from a list of patients whose chart they had viewed in the last 3 months

Characteristic	Recalled	Not recalled
N (row %)	130 (29.5)	311 (70.5)
Age based on EHR, N (col %)
18–30	36 (27.7)	93 (30.0)
31–10	32 (24.6)	67 (20.0)
41–50	33 (25.4)	89 (30.0)
51–60	19 (14.6)	36 (11.8)
61–84	10 (7.7)	26 (8.2)
Spanish preferred language, N (col %)
Yes	112 (86.2)	281 (90.5)
No	18 (13.8)	30 (9.5)
Hispanic/Latino, N (col %)
Yes	128 (98.5)	310 (99.7)
No	2 (2.5)	1 (0.3)
Uninsured, N (col %)
Yes	73 (56.2)	213 (68.0)
No	57 (43.8)	98 (32.0)

Patient characteristics were derived from the OCHIN electronic health record system

EHR electronic health records

Among patients that the eligibility specialists were able to recall (N = 130), 62% were denoted as an undocumented/recent immigrant by the specialist (Table 2). These patients were older, had higher percentage of Spanish as the preferred language, and had a higher prevalence of being uninsured compared to those that the eligibility specialist denoted as not being an undocumented/recent immigrant.

Table 2.

Among those that the eligibility specialist recalled (N = 130), patient demographic characteristics by eligibility specialist’s determination of undocumented/recent immigrant status

Characteristic	Undocumented/recent immigrant status as denoted by the eligibility specialist
	Yes	No
N (row %)	81 (62.3)	49 (37.7)
Age based on EHR, N (col %)
18–30	13 (16.0)	23 (46.9)
31–10	23 (28.4)	9 (18.4)
41–50	23 (28.4)	10 (20.4)
51–60	13 (16.0)	6 (12.2)
61–84	9 (11.1)	1 (2.0)
Spanish preferred language, N (col %)
Yes	75 (92.6)	37 (75.5)
No	6 (7.4)	12 (24.5)
Hispanic/Latino, N (col %)
Yes	79 (97.5)	49 (100.0)
No	2 (2.5)	0 (0.0)
Uninsured, N (col %)
Yes	55 (67.9)	18 (36.7)
No	26 (32.1)	31 (63.3)

Patient characteristics were derived from the OCHIN electronic health record system

EHR electronic health records

Statistics of correspondence between our EHR algorithm and eligibility specialist determination (our gold standard) are reported in Fig. 1 and the cross-tabulation of responses between EHR and Eligibility Specialists is found in Appendix Table 3. Overall, the EHR algorithm had a fair agreement (66.2%, 95% CI 57.3–74.2) with the 130 patients the eligibility specialist responded to (i.e., our presumed gold standard). Sensitivity (67.9%, 95% CI 56.6–77.9) and specificity (63.3%, 95% CI 48.3–76.6) were similar overall.

Fig. 1.

Correspondence between EHR algorithm and eligibility specialist assessment, overall and by self-rated confidence in assessment

When the eligibility specialist was either moderately confident of their assessment or very confident of their assessment, that agreement increased (77.6%, 95% CI 63.4–88.2) as did sensitivity (81.1%, 95% CI 64.8–92.0) while specificity remained constant (66.7%, 95% CI 34.9–90.1). Prevalence-adjusted bias-adjusted kappa (PABAK) statistics demonstrated fair to moderate strength of agreement between the EHR algorithm and the eligibility specialist. When eligibility specialists were moderately or very confident in their assessment, the agreement was moderate (PABAK = 0.55, 95% CI 0.27–0.76). However, when they were not at all or somewhat confident, we observed slight agreement (PABAK = 0.19, 95% CI 0.00–0.40).

Discussion

This study describes the development and validation of an EHR-based approach to study safety-net immigrant populations in health services research. In our mixed methods evaluation, eligibility specialists reported consistent and routine conversations about immigration status with every patient, along with significant confidence that they gathered accurate information. Using the eligibility specialist knowledge of patient immigration status as the gold standard, our proposed EHR algorithm produced fair to good correspondence.

There are several reasons why our EHR algorithm may have not produced even better correspondence. First, our gold standard may be imperfect. While qualitative interviews revealed that eligibility specialists were confident that they knew patients’ immigration status, we were unable to match this confidence against another standard. Eligibility specialists only remembered approximately 30% of the patients they had contact with in the prior 3months based on name recall alone (we had no secure way of showing patient pictures). The combination of heavy caseload and time elapsed between client interaction and our interview may have significantly affected recall; the job of eligibility specialists is to assess patients’ needs for resources and to connect patients with those resources, not to remember their immigration information. Second, the EHR-based algorithm did not account for individuals without Medicaid for reason other than immigration status (e.g., patient failed to register for Medicaid on single or multiple occasions for other reasons not measured in the study). Our algorithm could identify these people as ineligible for federal programs because of their citizenship status, when in fact they are eligible for insurance. Future work can explore other reasons for persistent uninsurance in this demographic other than immigration.

Researchers have employed other methods to ascertain immigration status: hospital billing records [36, 37], including those without valid social security numbers [38], notes of hospital social workers [39], and in depth, mixed method approaches that include interview methods [40]. These have not resulted in a validated, safe, and reproducible method for identifying immigration status at the scale necessary for health services research in Latino immigrant populations. Assurances of anonymity and confidentiality by faceless phone interviewers, or by research staff (even native-speaking staff with community connections) may not be enough to overcome larger societal forces, and researchers may need to be skeptical about using interviews to assess documentation or immigration status. Although there is some evidence that Latino immigrants experience fear in health care settings [15, 20], the clinicians and clinic team members at community health centers who ask patients to share this type of information are viewed, by low-income Latinos as trustworthy; clinics are perceived to be a safe place to share information about one’s social and economic background [41, 42]. Levison et al. performed similar work in a disease-specific population [43], but to our knowledge, this work expands on that work, as no studies have attempted the validation of an EHR-based approach to study immigration in a broad primary care population as described here. While our EHR–based approach to assessing immigration status had only moderate agreement, our high confidence subset suggests that this approach may still have potential.

Limitations

This proof-of-concept study was limited by an overall small sample size in a narrow demographic in a single network in a single region. It was further limited by eligibility specialist name recall. Future efforts can explore ways to improve staff recall (such as showing patient pictures), test their knowledge in real-time (eliminating the need for recall), or more safely inquire with patients. In addition, data items such as household FPL and ethnicity are still self-reported by the patient to the clinic, introducing the potential for reporting bias or recording error. In the future, the use of natural language processing could also aid in the gathering of chart data about this population for research, as this has been used to gather demographic and clinical data from electronic health records [44–46]; it is uncertain though, whether this would be robust source of reliable nativity data. While the EHR algorithm produced fair to good correspondence, this approach gives up precision in estimating individual-level immigration status for patient safety by constructing a group-level proxy for immigration status. This level of correspondence may provide a good enough proxy to measure immigration status given these safety concerns. Future health services research—where immigration status is the main independent variable of interest- can couple this with statistical methods to address measurement error in binary regressors [47–49]. Additionally, involving other practitioners, with even more established relationships with patients, could further verify or refute our findings, and would be an important next step.

Conclusion

Measurement challenges are not a new phenomenon in health disparities research. Assessing immigration/documentation status to study the impact of immigration on healthcare utilization/outcomes is one of those challenges. FQHCs employ eligibility specialists who have regular conversations about immigration and citizenship with their patients. The electronic health record may be a potential means for studying immigration status in health services research on a larger scale, although more study is needed to determine the accuracy and utility of EHRs for this purpose. Validated proxies for immigration status (in lieu of actual immigration status), obtained in a safe manner for the population, are crucial in order to understand and eliminate health disparities in this population.

Appendix A: Interview Guide

1. Describe your role here at the clinic.

2. How does that role involve working with Latino immigrants?

3. Tell me how you learn about patients’ immigration status?

4. How do you decide how to handle or discuss patient’s immigration status?

5. If one wanted to know the immigration status of a group of patients, what would be the best way to do that?

[Explanation of EHR algorithm: Our research study is investigating whether or not we can use electronic health records to study the health care if undocumented or recent Latino immigrants. We think using electronic health records may help avoid having to ask patients sensitive questions about their immigration status, but still enable us to study how recent or undocumented immigrants use healthcare. To study immigrants in this way, we use several pieces of information to identify a population whom we think are likely recent or undocumented Latino immigrants: adults who are Hispanic or Latino or Spanish speaking, whose income is 100% FPL or lower, and who are uninsured]

What did you think about this EHR method for studying Latino immigrants?

Appendix B

See Table 3.

Table 3.

Cross-tabulation of overall correspondence between EHR algorithm and Eligibility Specialist Assessment, N(%)

		Undocumented/recent immigrant status as denoted by the eligibility specialist
		Yes	No
Undocumented/recent immigrant status as denoted by the EHR-based algorithm	Yes	55 (42.3)	18 (13.8)	73 (56.1)
	No	26 (20.0)	31 (23.9)	57 (43.9)
		81 (62.3)	49 (37.7)	130 (100.0)