Multidimensional measure of immigrant integration

Significance

While successful integration of immigrants and refugees is a goal for many host countries, scholarly assessment of progress toward that goal is hampered by the lack of an accepted measure of integration success. This study proposes a pragmatic, survey-based measure that identifies six dimensions of integration and then, through four surveys, examines the construct validity of the composite measure. The measure has the potential to advance scientific progress in the study of immigrant integration.

Abstract

The successful integration of immigrants into a host country’s society, economy, and polity has become a major issue for policymakers in recent decades. Scientific progress in the study of immigrant integration has been hampered by the lack of a common measure of integration, which would allow for the accumulation of knowledge through comparison across studies, countries, and time. To address this fundamental problem, we propose the Immigration Policy Lab (IPL) Integration Index as a pragmatic and multidimensional measure of immigrant integration. The measure, both in the 12-item short form (IPL-12) and the 24-item long form (IPL-24), captures six dimensions of integration: psychological, economic, political, social, linguistic, and navigational. The measure can be used across countries, over time, and across different immigrant groups and can be administered through short questionnaires available in different modes. We report on four surveys we conducted to evaluate the empirical performance of our measure. The tests reveal that the measure distinguishes among immigrant groups with different expected levels of integration and also correlates with well-established predictors of integration.

Many countries have experienced high levels of immigration in recent decades. Successful integration of immigrants into a host country’s society, economy, and polity has therefore become a major focus for policymakers and scholars. In the policy world there are heated debates about which policies most effectively facilitate immigrant integration, and in academia there is a vigorous discourse about why some immigrant groups integrate while others do not (1).

In this study we address one of the fundamental obstacles to scientific progress in this field: the lack of a common empirical measure of immigrant integration. To date, research on immigrant integration has proceeded such that each study relies on its own measures of what constitutes successful integration. This heterogeneity substantially reduces the possibility of informative comparison across studies, across countries, and over time and has hampered the accumulation of scientific knowledge.

Justifications for the current heterogeneity of definitions and proxies are usually based on the recognition that integration as a concept is “essentially contested” (2) or too complex to be captured by a single metric (3, 4). This, however, is equally true of other important and complex concepts, such as a country’s level of wealth, where the literature has successfully coordinated on commonly used measures such as gross domestic product or the human development index. Other examples include the K10/K6 scale, which is widely used in public health as a measure of mental health (5), or the Rosenberg scale, which is extensively used in cross-cultural studies to measure self-esteem (6). While these scales are arguably far from perfect measures of complex concepts, scholars tend to agree that they provide sufficient construct validity to permit well-conceived scientific analyses. This agreement facilitates the accumulation of knowledge by allowing for comparisons across studies, populations, and time.

In this study we propose the Immigration Policy Lab (IPL) Integration Index as a pragmatic, survey-based measure of immigrant integration. We developed this measure to provide scholars with a short instrument that can be implemented across survey modes, applies to different groups of immigrants (e.g., new citizens, refugees, undocumented immigrants), allows for comparisons across countries and over time, and provides construct validity in capturing the multidimensional nature of integration. The IPL Integration Index is available in two forms, the short-form IPL-12 and long-form IPL-24. Both scales capture six dimensions of integration—psychological, economic, political, social, linguistic, and navigational—each associated with two or four survey items, respectively.

The IPL Integration Index is versatile and allows scholars to pursue different goals. The measure can be used for descriptive analyses to map out the integration levels of different groups or generations or as an outcome measure for causal analyses evaluating the effect of a program, event, or policy intervention on integration success. Scholars looking for a short but comprehensive overall measure of integration can use the IPL-12 scale. If more precision is required and space on the questionnaire is available, scholars may prefer the overall IPL-24 scale. Other scholars who focus on particular dimensions, say political or economic integration, might use only the four-item subscales that capture integration on those dimensions. The organization in six distinct dimensions also allows researchers to characterize immigrant populations by the way the individual dimensions correlate.

It is important to emphasize that our measure does not claim to be the only, best, or perfect measure of integration. The purpose of our measure is to strike a pragmatic compromise and help generate cumulative knowledge. Therefore, we designed the measure to capture key aspects of integration with a small number of widely applicable questions so that it can be used at low cost and facilitate comparability. It is important to recognize that our index prioritizes wide applicability to facilitate comparisons across studies with different immigrant populations. For example, instead of developing the most precise measure of integration that captures the specific situation of, say, refugees or unauthorized immigrants, our measures are designed to be general enough to allow for meaningful comparisons across all immigrant groups without resorting to questions that would apply only to one group.

In developing our measure, we defined integration as the degree to which immigrants have the knowledge and capacity to build a successful, fulfilling life in the host society (7, 8). This definition recognizes the dual importance of knowledge and capacity. Knowledge entails aspects such as fluency in the national language and ability to navigate the host country’s labor market, political system, and social institutions. Capacity refers to the mental, social, and economic resources immigrants have to invest in their futures. Knowledge and capacity jointly enable individuals to realize their potential and achieve their vision and life goals in the host society.

Our definition distinguishes integration from assimilation, the latter of which requires immigrants to shed their home country’s culture in favor of adopting the cultural practices of the host country’s dominant group (9). In our view, immigrants need not shed their own culture to live successful and fulfilling lives in the host country. Therefore, our measure focuses exclusively on capturing the degree to which immigrants have acquired the knowledge and capacity to build successful lives rather than the degree to which they have shed their cultural heritage. This is the reason we do not use the scores of native-born respondents as a benchmark for our measure. While our measure is solely focused on measuring integration, we do not deny some overlap between the two concepts as they are not mutually exclusive. For example, to capture linguistic integration we measure only whether immigrants have acquired skills in the host country’s or region’s dominant language, but we are agnostic as to whether immigrants still use their home country’s language. In contrast, a measure of assimilation would by definition take both aspects into account.

Theory and Methodology

In developing our measure we built on two interlinked literatures. First, we consulted theoretical research that clarifies the core concepts of integration, incorporation, and assimilation (1, 4, 10). Second, we consulted an extensive set of surveys, most of them collected throughout Europe and North America, that seek to measure the degree to which immigrant populations are integrating into host country societies (for example refs. 1 and 11–13). In SI Appendix we list the datasets and studies we consulted.

Given our goal of developing a short yet comprehensive scale, we first reduced the multiple domains discussed in previous research to six dimensions of integration: psychological, economic, political, social, linguistic, and navigational. To develop the questions within each dimension, we then devised a set of criteria that each question needed to fulfill.

First, a question should reflect construct validity. Second, a question should have clear directionality, such that higher values refer to higher levels of integration. Third, given our focus on integration, as opposed to assimilation, a question should not presuppose that immigrants shed cultural repertoires of their home country. The native population is not a point of reference for respondents; rather it is success in the host society. Fourth, a question should translate well into different national and local environments. (Note that some of our questions apply only to democratic states. Small modifications will be necessary to use our measure in nondemocracies.) Fifth, a question should be answerable by all adult immigrant groups, all adult immigrants within a group, and host country natives. This ruled out questions that would apply only to a subset of respondents (e.g., those who are refugees or those who have a job). Sixth, a question should be adjustable to different survey modes, including phone, face-to-face, and online surveys. Seventh, a question should yield variation across responses. The more a question can discern different levels of integration, the more useful it is for statistical analysis.

Based on these theoretical criteria, we developed the questionnaire through an iterative process of question writing, empirical testing, and refinement. Overall, this process involved six rounds of major revisions based on workshops with experts and eight pilot surveys of various immigrant samples administered online, by mail, and in face-to-face surveys. Additionally, we conducted qualitative, think-aloud interviews with immigrants to examine their subjective understanding of all questions. During the entire development process, we tested over 200 questions and conducted 3,954 interviews (see SI Appendix for details; this does not include the four validation surveys we use in this study).

The final products of this process are the short-form IPL-12 and long-form IPL-24 scales, which capture each dimension of integration with two or four questions, respectively. Below, we briefly summarize the core concepts of integration success that inform our questions. SI Appendix provides the full questionnaire and also details the development process that led to the final questions.

For psychological integration, our measure captures respondents’ feeling of connection with the host country, their wish to continue living there, and their sense of belonging. For economic integration, our measure captures income, employment, satisfaction with employment situation, and the ability to meet different levels of unexpected expenses. For political integration, our measure captures understanding of the important political issues facing the host country and the degree to which respondents engage in discussion and political action. We also include questions that assess respondents’ political knowledge. For social integration, our measure captures social ties and interactions with natives in the host country, as well as bridging social capital as evidenced by participation in organizations with natives. For linguistic integration, our measure captures respondents’ assessment of their ability to read, speak, write, and understand the dominant language of their host country or region. For navigational integration, our measure captures their ability to manage basic needs in the host country, such as seeing a doctor, addressing legal problems, and searching for jobs. The measure also tests knowledge of basic conventions in the host country: the typical way to pay income taxes, rules for driving, how to put an address on a letter, and how to appropriately seek medical help.

We also developed a scoring rule: A score between 1 and 5 points is computed for each question such that there is a maximum score of 60 across all six dimensions for the IPL-12 and 120 for the IPL-24. The measure is then rescaled to range from 0 to 1 in increasing levels of integration (see SI Appendix for the detailed scoring rules). Of course, in their own research projects, researchers can recode particular questions according to their interests, while in addition reporting the standardized IPL Integration Index score.

We pursued two strategies to examine the performance of the IPL Integration Index. First, we applied a “contrasted groups approach” (14) to test whether the measure successfully distinguishes between groups that are expected to have different levels of the characteristic being measured. To apply this approach, we administered our survey to four samples of immigrants who we expected to differ in terms of their average levels of integration. The samples, listed in order of decreasing expected levels of integration, included a stratified sample of high-income, white immigrants in the United States (sample A); a stratified sample of immigrants in Germany (sample B); a sample of registrants for a program in New York that assisted low-income immigrants who are eligible for naturalization (sample C); and a sample of mostly recent immigrants enrolled in English language classes in San Jose, CA (sample D). The surveys of samples A–C were administered through an online survey platform, while the survey for sample D was administered with paper questionnaires. Questionnaires were administered in English for sample A, in German for B, and in English and Spanish for C and D (see SI Appendix for more details on survey samples and expectations).

Our second strategy was to check whether, across the four surveys, the IPL Integration Index correlates in the expected direction with important predictors of integration that are used in the literature. First, we expect immigrants with more years of residency in the host country to have higher levels of integration (15–17) and therefore higher IPL Integration Index scores on average. Second, we expect immigrants with more secure legal status, such as permanent residents and naturalized citizens, to have higher levels of integration (17–20). Third, we expect immigrants with higher levels of education to have higher levels of integration (15, 21–23).

Results

Construct Validity Through “Contrasted Groups.”

Fig. 1 shows the distribution of residency in the host country (Fig. 1, Left) and the distribution of IPL-12 scores (Fig. 1, Right) for each of the four contrasted samples. The average length of residency varies from about 4 y in sample D, the recent immigrants enrolled in English language classes in San Jose, CA, to about 35 y for sample A, the high-income immigrants in the United States. Sample B, the immigrant sample from Germany, and sample C, the registrants for the naturalization program in New York, fall in between with 20 y and 12 y of average residency, respectively. Given that residency has been identified as one of the most important correlates of integration, these differences suggest that the expected level of integration is highest in sample A and lowest in sample D, with samples B and C falling in between. If our IPL-12 scale measures integration, we would expect the average level of measured integration to vary from highest to lowest in samples A–D, respectively.

Fig. 1.

Box plots show for four immigrant samples A–D the distributions of residency (Left) and IPL-12 Integration Index scores (Right). The samples are ordered by decreasing expected levels of integration. The measured integration levels based on the IPL-12 Integration Index reproduce the ordering of the samples from highest to lowest expected levels of integration. Box width is proportional to sample size.

The results in Fig. 1, Right indicate that the IPL-12 measure does successfully distinguish among the four samples in terms of their measured integration levels. The average IPL-12 scores are 0.8 in sample A, 0.69 in sample B, 0.55 in sample C, and 0.46 in sample D, and the box plots show that the distributions of scores are well separated across the four samples. The results are very similar when we consider the long-form IPL-24 scores instead (see SI Appendix for details). These findings provide evidence that the IPL Integration Index is able to discriminate among groups that are expected to vary in their integration levels and thus speak to the construct validity of the measure.

Construct Validity Through Correlation with Predictors of Integration.

If the IPL-12 score measures integration, we would expect it to correlate with well-established predictors of integration from the literature. To test this, we pooled the data from all four samples and regressed the IPL-12 scores on the following predictors: residency, education, immigration status, and an indicator for shared language, as well as controls for age and gender and sample fixed effects. Fig. 2, Left shows the estimated marginal effects from this regression. We find that the IPL-12 scores are conditionally correlated with all five predictors in the expected direction. For example, a 1-SD increase in residency (19 y) is associated with a 0.03-point increase in the IPL-12 score (SD = 0.16), controlling for the other variables (P value < 0.0001). Similarly, a 1-SD increase in years of education (5 y) is associated with a 0.01-point increase in the IPL-12 score (P value = 0.0007). We also find that compared with immigrants with temporary visas (the reference category), immigrants who are permanent residents or naturalized citizens of the host country have IPL-12 scores that are 0.06 points (P value = 0.0003 and P value < 0.0001, respectively) higher on average. Finally, immigrants from countries with the same dominant language as that of the host country have 0.02-point higher scores on average than immigrants where home and host languages are different (P value = 0.09). We find no significant differences in IPL-12 scores based on age and gender. Taken together, these results speak to the construct validity of the IPL Integration Index as a measure of integration.

Fig. 2.

The IPL-12 Integration Index scores correlate with established predictors of integration. (Left) Marginal effects from a regression of IPL-12 Integration Index scores on predictors of integration. Circles indicate point estimates and lines 90% and 95% confidence intervals. (Right) A scatter plot between the IPL-12 Integration Index scores and years of residency. Lines show how the percentiles of the IPL score distribution change with residency. Lines are drawn for the 5th, 10th, 25th, 50th (orange), 75th, 90th, and 95th percentiles.

Fig. 2, Right plots the IPL-12 score against years of residency, arguably the most reliable predictor of integration. The lines show how the percentiles of the conditional distribution of IPL-12 scores change with increased residency; the solid orange line indicates the median, and the dashed blue lines indicate the fifth, 10th, 25th, 75th, 90th, and 95th percentiles. We find that IPL-12 scores increase with longer residency across all percentiles. In addition, the increase is more pronounced at the lower percentiles, indicating that there is some degree of convergence toward higher IPL-12 scores at higher levels of residency.

We also see that for most percentiles there are decreasing marginal returns such that the rate of growth in IPL-12 scores becomes flatter with longer residency. For example, at the median, the IPL-12 scores increase from 0.5 to 0.73 when moving from 1 y to 20 y of residency, but then increase only by 0.09 additional points when moving from 20 y to 40 y of residency. This nonlinear shape speaks to the construct validity of the IPL-12 scale, because it is consistent with our intuition that the marginal integration returns of longer residency are reduced the longer immigrants live in the host country.

Correlations Between and Within Integration Dimensions.

Beyond the aggregate integration score, one advantage of our IPL Integration Index is that it also allows researchers to focus on any specific dimension of integration as well as the interplay among them. For each dimension there are two or four questions, which can be aggregated into a single dimension-specific scale. Here we illustrate these dimension-specific scales by focusing on the long-form IPL-24, which captures each dimension with four questions, respectively. Fig. 3 shows the pairwise relationships and correlations between the six dimension-specific integration scales in our pooled sample.

Fig. 3.

Scatter-plot matrix for the six dimensions of integration as measured by the IPL-24 Integration Index (pooled sample, N = 787): economic (Econ.), linguistic (Ling.), navigational (Nav.), political (Pol.), psychological (Psy.), and social (Soc.) integration. Panels in Middle diagonal show the histograms of the marginal distributions, panels in Upper Right diagonal show the bivariate correlation coefficients, and panels in Lower Left diagonal show the scatter plots with Loess lines (black).

The marginal distributions show that our sample exhibits wide variation in terms of the levels of psychological, social, economic, political, and navigational integration. The one exception is the linguistic integration dimension, where the distribution is skewed toward the top of the scale, as expected, given that our four validation surveys were mostly administered in the host country’s dominant language and the average residency is 25.7 y in the host country. The skew toward higher language ability explains why our validation sample does not include many immigrants with extremely low values on the IPL Integration Index. The fact that our scale leaves room for lower values is a desired feature. We expect the lower part of the scale to be populated in samples of less integrated immigrants who do not speak the host country language. As we detail in SI Appendix, we did find much larger variation in linguistic integration in two pilot surveys that we administered in New York and Switzerland to immigrants who chose to take the survey in their birth country language (SI Appendix).

Moreover, we find that in our sample the six dimensions tend to be mostly positively correlated, indicating that immigrants who score high on one dimension of integration also tend to score high on the other dimensions. That said, we also see that some of the relationships are rather weak, as we might expect given the sample composition. For example, we find that psychological integration and economic integration are only weakly correlated. This is partly driven by the sample of immigrants in the language classes in San Jose, which included a large number of spouses from high-income households who had recently arrived. This is consistent with models of segmented assimilation (24, 25) that reject approaches envisioning a straight line over time toward adoption of host country culture, with the possibility of intergenerational shifts in some domains toward the host country culture and in others a return to the cultural repertoires of their parents’ home country. In line with these findings, the possibility of differential progress (or regression) across our six dimensions of integration opens the possibility of measuring more precisely (across generations and over time in response to policy changes) differential patterns of integration across groups and countries.

In SI Appendix we provide tests that indicate that within each dimension the items are highly correlated. For example, the standardized Cronbach’s alphas in the pooled sample for the six IPL-24 dimensions are 0.96 for linguistic, 0.78 for political, 0.6 for social, 0.62 for economic, 0.81 for psychological, and 0.76 for navigational integration. For samples A and B we were also able to collect response times, and the median time to complete the IPL-12 was 2–3 min and that for the IPL-24 was 6–8 min. Note that samples A and B were online surveys and that response times might be different in other survey samples.

Conclusion

Immigrant integration has become a major policy issue in many host countries, and the academic community has generated much research on the subject. However, scientific progress has been hampered by the lack of a common measure of integration, which would allow for cumulative knowledge. In this study we propose the short-form IPL-12 and long-form IPL-24 as pragmatic measures of immigrant integration, namely the degree to which immigrants have the knowledge and capacity to achieve success in their host society. Our measure captures six dimensions of integration—psychological, economic, political, social, linguistic, and navigational—and each is measured with a set of two or four survey questions. We do not claim that this is the only or best possible measure of integration, but our goal was to strike a pragmatic compromise between construct validity, ease of use, and wide applicability. The measure is short but comprehensive and designed such that it can be applied across countries, immigrant groups, time, and survey modes. It allows researchers to focus on overall levels of integration or study the interplay between specific dimensions of integration.

We examined the construct validity of the measure using four original surveys of different immigrant samples, which we expected to vary in terms of their integration success. The IPL Integration Index successfully distinguished among the four samples in the expected order. We also found that the IPL Integration Index correlates in the expected direction with several well-established predictors of integration, such as length of residency, education, and legal status. Finally, we illustrated how the measure can be used to study the interplay between different dimensions of integration.

Overall, we foresee substantial payoffs for the study of immigrant integration if the scientific community were to coordinate on the use of a single common measure like our IPL Integration Index. Our hope is that scholars will take up this proposal and put the measure to good use so that it can be further refined as more data are accumulated across multiple studies and contexts.

Materials and Methods

Instrument.

The questionnaire for the IPL Integration Index was developed based on a systematic review of existing survey instruments and measures in the literature. Six criteria guided the development and selection of questions: construct validity and clear directionality, measurement of integration rather than assimilation, applicability across national and local environments, applicability across immigrant groups and subsets of respondents, and yield of variation in responses. The wording of the questions is provided in SI Appendix. To make our measure accessible to scholars and practitioners, we provide our survey instrument in different languages as well as guidelines for implementation online (www.integrationindex.org).

Data.

To examine the construct validity of the measure, we administered the IPL Integration Index to four samples of immigrants as described in Theory and Methodology. Each survey was approved by Stanford University’s Institutional Review Board (protocol ID: 35163). We obtained informed consent from survey participants.

Statistical Methods.

To compare the distributions of IPL-12 scores across the four samples we used box and whisker plots (Fig. 1). To examine the correlation between the measure of integration and predictors of integration, we used linear regression analysis. In particular, we regressed the IPL-12 score on age, gender education, shared language, residency, and immigration status (temporary visa/permanent resident/naturalized). The model also includes sample fixed effects. We then computed 95% confidence intervals for the regression coefficients based on robust standard errors (Fig. 2, Left). To examine the relationship between the IPL-12 scores and residency across different quantiles, we used a quantile regression where the IPL-12 score is regressed on a third-order polynomial of residency (Fig. 2, Right). To examine the correlations between the different dimensions of integration, we first aggregated the four IPL-24 questions in each dimension to construct scales. We then constructed a scatter-plot matrix to summarize the pairwise relationship between the scales using correlation coefficients and Loess smoothers. Replication materials have been deposited in the Harvard dataverse, at https://doi.org/10.7910/DVN/MF2Q7U.