Abstract
The lack of recommended design for Roma health-monitoring hinders the interventions to improve the health status of this ethnic minority. We aim to describe the riskiness of Roma ethnicity using census-derived data and to demonstrate the value of census for monitoring the Roma to non-Roma gap. This study investigated the self-declared occurrence of at least one chronic disease and the existence of activity limitations among subjects with chronic disease by the database of the 2011 Hungarian Census. Risks were assessed by odds ratios (OR) and 95% confidence intervals (95% CI) from logistic regression analyses controlled for sociodemographic factors. Roma ethnicity is a risk factor for chronic diseases (OR = 1.17; 95% CI: 1.16–1.18) and for activity limitation in everyday life activities (OR = 1.20; 95% CI: 1.17–1.23), learning-working (OR = 1.24; 95% CI: 1.21–1.27), family life (OR = 1.22; 95% CI: 1.16–1.28), and transport (OR = 1.03; 95% CI: 1.01–1.06). The population-level impact of Roma ethnicity was 0.39% (95% CI: 0.37–0.41) for chronic diseases and varied between 0 and 1.19% for activity limitations. Our investigations demonstrated that (1) the Roma ethnicity is a distinct risk factor with significant population level impact for chronic disease occurrence accompanied with prognosis worsening influence, and that (2) the census can improve the Roma health-monitoring system, primarily by assessing the population level impact.
Keywords: Roma minority, health status, census, risk assessment, impact assessment, monitoring
1. Introduction
The Roma population is the largest ethnic minority group in Europe. The common opinion is that the health status of this population is poor compared with that of the non-Roma population. Recently, this rather anecdotal belief has been convincingly supported by a growing number of quantitative investigations. The current urgency for intervention is hardly debatable [1], but effective interventions are hindered by our limited knowledge regarding the details of the mechanisms that have led to the substantial population-level health deficit among the Roma population [2,3,4].
It has been thoroughly demonstrated that Roma people are overrepresented in the marginalized communities, living in relatively unhealthy environments, and have an unhealthier lifestyle than the non-Roma population [5,6,7,8,9,10]. In addition, the access of Roma people to care is impaired, leading to high levels of unmet health needs [4,11,12,13,14,15,16,17,18]. It is yet to be clarified whether the Roma people themselves are a distinct intervention target or whether the unhealthy environment, lifestyle, and restricted availability of health services, to which many non-Roma people are exposed, should be targeted by organized interventions. In brief, is the Roma ethnicity an important and distinct risk factor to be targeted by population-level interventions?
Furthermore, it is obvious that the Roma people are not a homogeneous social stratum. Our knowledge of the role of this heterogeneity in determining the health status of this population is quite limited. Moreover, we do not know whether it is justified at all to use this dichotomization (distinguishing Roma and non-Roma persons) in epidemiological investigations of the risk associated with Roma ethnicity [19,20,21].
It seems that higher quality publications on the health impact of Roma ethnicity use more narrow case definitions or more specific settings [22,23,24,25,26]. This approach results in knowledge that relies mainly on subgroup specific findings [7,10,12,25], observations that cannot be extrapolated to the whole Roma population [10,27] and limited knowledge of the public health importance of Roma ethnicity as a risk factor.
All of these problems are reflected in the questionable effectiveness of Roma-targeted health interventions [3,4,28,29], showing that program elaboration, monitoring, and evaluation require the development of currently applied epidemiological methods. At present, responsible policy-makers face a scarcity of useful data on the population-level health impact of Roma ethnicity.
Realizing the weight of this challenge, the European Union accepted “An EU Framework for National Roma Integration Strategies up to 2020” (NRIS) in 2011. It commits Member States to integrate policies that focus on Roma minorities in a “clear and specific” way and address the requirements of Roma minority with “explicit actions” in order to reduce the health gap between Roma and non-Roma populations. [30] Furthermore, the NRIS implementation-monitoring states that achieving the goals of the national adaptation of NRIS [31] without defining measurable indicators and operating a robust monitoring system is doubtful [32,33].
Unfortunately, there is no recommended, detailed design for health-monitoring of the Roma population in NRIS that could solve the two interrelated issues of epidemiological monitoring: identification of Roma individuals and definition of sampling frame. Although, there are initiatives to overcome these methodological shortcomings [34,35]. In practice, the combined efforts by subgroup investigations related to the mechanisms connected to Roma ethnicity, which are completed with self-declared ethnicity-based population-level investigations, are applied to obtain useful Roma health-specific data. Taking into account that surveys are inevitably subject to sampling bias because of the nonexistence of a Roma sampling frame (e.g., the recently conducted Second European Union Minorities and Discrimination Survey study coordinated by the European Union Agency for Fundamental Rights has representativity limitations arising from the sampling frame) [36], a census that would provide data representative of the population could be a useful part of monitoring.
Self-declared ethnicity-related data based on national censuses have been available for decades in many countries [37,38,39,40]. To avoid the well-known under-registration of Roma ethnicity, multiple questions used to be applied in census questionnaires. Governments ensure the regularity and the feasibility of censuses required for monitoring. If a census collects data on health status and ethnicity along with the most important sociodemographic risk factors of health impairments, then it can establish a basis for comparative investigations of Roma and non-Roma investigations regularly and in a feasible manner.
Our aim is (1) to compare the occurrence of chronic diseases and activity limitations among the Roma and non-Roma populations in order to describe the riskiness and impact of Roma ethnicity independently from sociodemographic factors using census-derived data and (2) to describe the added value of a census-based, obviously not perfect methodology applying in itself, to the monitoring of the health gap between Roma and non-Roma populations.
2. Materials and Methods
2.1. Setting
This study investigated an anonymized database of the last Hungarian national census conducted in 2011. The cross-sectional survey covered the whole Hungarian population. The data collection was carried out from 1 October 2011 to 31 October 2011. The study population included all Hungarian citizens living in the country or staying temporarily abroad for a period of less than 12 months as well as all foreign citizens and stateless persons living in Hungary for a period of more than 3 months.
The availability of the database was provided by the Hungarian Central Statistical Office (HCSO). The study was approved and supervised by the institutional board of the HCSO, which is responsible for both the utilization of the census database and the protection of human rights in handling sensitive personal data (KSH/ADKI/1156/2014).
2.2. Explanatory Variables and Outcome Measures
The first part of the census questionnaire [41] was focused on housing conditions (type of walls and presence of utilities, such as bathroom, flush toilet, electricity, water and hot water supply, and heating system). Because the response to these questions was compulsory, apart from homeless people and those who are living in institutions, all subjects’ living conditions were characterized by these parameters. All factors were used as proxy indicators of deprivation in the analysis.
The second part of the census questionnaire was focused on personal characteristics [42]. The responses were compulsory in this part of the questionnaire as well, apart from voluntary questions related to ethnicity, the presence of chronic disease, and activity limitation. Subjects were classified according to covariate variables, such as sex, age, marital status, highest level of education, and employment status. The following age groups were used: 0–5, 6–17, 18–34, 35–59, 60–64, and 65+ years. The highest level of education was categorized as less than primary, primary, vocational, high school, and tertiary school. Those individuals less than 6 years old or attending an educational institution were classified as not having completed their education. Employment status was described as working, unemployed, retired, receiving social benefits, dependent, or student. Single, coupled, divorced, and widowed were distinguished with respect to marital status.
Four questions referred to self-declared ethnicity. These questions included primary self-declared ethnicity (“Which ethnicity do you feel you belong to?”), secondary self-declared ethnicity (“Do you think you belong to another ethnicity too?”), first language (“What is your first language?”), and secondary language (“In what language do you usually speak with family members or friends?”). Those who responded with “Roma” to any of these questions were regarded as Roma people. All others were considered to be non-Roma people.
The primary outcome variable was the self-declared occurrence of at least one chronic disease. The type of disease was not requested. The existence of activity limitations among subjects with chronic disease was investigated as a secondary outcome. The activity limitations were sub-grouped according to the affected function: self-sufficiency, everyday life, learning-working, family life, transport, communication and community life.
2.3. Statistical Analysis
First, crude prevalence of having at least one chronic disease and activity limitations among those with chronic disease were calculated for the Roma and non-Roma subpopulations. The association with the Roma ethnicity was evaluated by χ2-test.
Then, internal indirect standardization was used to control for the confounding effect of age, sex, and level of education. The age- and sex-standardized prevalence ratios (SPR) and age-, sex-, and education-standardized prevalence ratios (SPRe) were calculated both for the Roma and non-Roma populations. Next, we assessed the risk associated with Roma ethnicity by the ratio of Roma-specific and non-Roma-specific standardized prevalence ratios (RR = SPRRoma/SPRnon-Roma; RRe = SPRe,Roma/SPRe,non-Roma). The 95% confidence intervals (95% CI) of the calculated measures were used in the statistical evaluation.
Then, we used multiple logistic regression modeling to investigate the influence of Roma ethnicity, independent of housing conditions (wall quality, public utilities for bathroom, flush toilet, electricity, water and hot water supply, and heating system) and personal characteristics (sex, age, level of education, employment, and marital status). Associations were quantified by odds ratios (OR) and corresponding 95% CIs.
Finally, point estimates and 95% CIs of the relative risks estimated by standardization or logistic modeling were used to describe the impact of Roma ethnicity on chronic disease and the occurrence of activity limitation. Attributable risk fractions among Roma (ARRoma) and the whole population (ARPopulation) were computed with corresponding 95% CIs.
Statistical computation was performed with STATA 14.0 (Stata Corporation, College Station, TX) software.
3. Results
There were 9,937,628 subjects who participated in the Hungarian census in 2011. The restriction of this population to the Hungarian citizens living in Hungary resulted in a database of 9,794,318 persons. Because the response rate for ethnicity and chronic disease-related questions was 78.36%, the database that could be used to analyze the determinants of chronic disease contained 7,674,607 records. In the case of logistic regression modeling, 166,366 records were excluded because the housing conditions data was not available. This exclusion reduced the response rate to 76.66%. On the basis of self-declarations, 3.83% (294,189 persons) were classified as Roma individuals, and 21.19% (1,626,447 persons) had at least one chronic disease in the study population.
The response rate for the question about activity limitation among those with chronic disease was 87.09%. This resulted in a database of 1,416,424 persons with chronic disease and who reported activity limitations (in the multivariate modeling, 40,763 whose housing conditions data was not available were also excluded, resulting in an 84.58% response rate for that analysis). There were 38,800 (2.74%) Roma persons in this database (Figure 1).
Figure 1.
Studied groups’ selection from Hungarian census 2011 population.
The sex representations among the Roma and non-Roma people were significantly different because of the higher proportion of males among the Roma population. The Roma age structure deviated significantly from the non-Roma age composition. The Roma were overrepresented in younger age groups and underrepresented among the older age groups. The level of education was much lower among Roma than among non-Roma. The percentage of individuals who had primary or less than primary education was 53.31% among the Roma population and 23.84% in the non-Roma population (Table 1). All of the other personal and housing-related deprivation indicators applied in the multiple logistic regression models indicated the poorer status of the Roma population in comparison to the non-Roma population (Table A1).
Table 1.
Demographic characteristics of the study population and the differences between the Roma and non-Roma population evaluated by χ2-test.
| Total (%) | Ethnicity | p-Value | ||
|---|---|---|---|---|
| Roma (%) | Non-Roma (%) | |||
| Sex | ||||
| Male | 3,634,387 (47.36%) | 148,889 (50.61%) | 3,485,498 (47.23%) | <0.001 |
| Female | 4,040,220 (52.64%) | 145,300 (49.39%) | 3,894,920 (52.77%) | |
| Age group (years) | ||||
| 0–5 | 451,388 (5.88%) | 37,801 (12.85%) | 413,587 (5.6%) | <0.001 |
| 6–17 | 959,523 (12.5%) | 77,720 (26.42%) | 881,803 (11.95%) | |
| 18–34 | 1,733,206 (22.58%) | 84,204 (28.62%) | 1,649,002 (22.34%) | |
| 35–59 | 2,728,387 (35.55%) | 81,286 (27.63%) | 2,647,101 (35.87%) | |
| 60–64 | 495,728 (6.46%) | 6160 (2.09%) | 489,568 (6.63%) | |
| 65+ | 1,306,375 (17.02%) | 7018 (2.39%) | 1,299,357 (17.61%) | |
| Education | ||||
| Not-completed-education | 1,705,508 (22.22%) | 102,090 (34.70%) | 1,603,418 (21.73%) | <0.001 |
| Less than primary | 499,959 (6.51%) | 60,581 (20.59%) | 439,378 (5.95%) | |
| Primary | 1,416,351 (18.46%) | 96,261 (32.72%) | 1,320,090 (17.89%) | |
| Vocational | 1,353,740 (17.64%) | 25,160 (8.55%) | 1,328,580 (18.00%) | |
| Highschool | 2,244,151 (29.24%) | 9221 (3.13%) | 2,234,930 (30.28%) | |
| Tertiary | 454,898 (5.93%) | 876 (0.30%) | 454,022 (6.15%) | |
| Total | 7,674,607 (100%) | 294,189 (100%) | 7,380,418 (100%) | - |
3.1. Chronic Disease Occurrence
The crude prevalence of having at least one chronic disease was significantly lower in the Roma population (14.75%) than in the non-Roma population (21.45%) (Table 2). However, significant risk elevation was observed among the Roma population after adjusting for age and sex (RR = 1.41) and after adjusting for age, sex, and education (RRe = 1.11). We found that the Roma ethnicity was a significant risk factor for having at least one chronic disease in the multivariate logistic model that controlled for deprivation indices (OR = 1.17) (Table 3).
Table 2.
Crude prevalence of having at least one chronic disease and the prevalence of activity limitations among subjects with chronic disease by functions in Hungary, according to the 2011 census, evaluated by χ2-test (more detailed descriptive statistics are summarized in Table A2).
| Total (%) | Ethnicity | Roma / Non-Roma (%/%) | p-Value | ||
|---|---|---|---|---|---|
| Roma (%) | Non-Roma (%) | ||||
| Chronic diseases subjects in study population | 1,626,447 (21.19%) | 43,382 (14.75%) | 1,583,065 (21.45%) | 0.68 | <0.001 |
| Activity limitation among chronic diseased subjects in: | |||||
| Self-sufficiency | 139,933 (9.88%) | 4193 (10.81%) | 135,740 (9.85%) | 1.09 | <0.001 |
| Everyday life | 467,747 (33.02%) | 16,351 (42.14%) | 451,396 (32.78%) | 1.28 | <0.001 |
| Learning working | 218,145 (15.40%) | 13,360 (34.43%) | 204,785 (14.87%) | 2.31 | <0.001 |
| Family life | 42,022 (2.97%) | 2064 (5.32%) | 39,958 (2.90%) | 1.83 | <0.001 |
| Transport | 361,757 (25.54%) | 10,279 (26.49%) | 351,478 (25.51%) | 1.03 | <0.001 |
| Communication | 39,339 (2.78%) | 1537 (3.96%) | 37,802 (2.74%) | 1.44 | <0.001 |
| Community life | 81,101 (5.73%) | 2897 (7.47%) | 78,204 (5.68%) | 1.31 | <0.001 |
Table 3.
Roma ethnicity’s association with the occurrence of having at least one chronic disease and activity limitations among those with chronic disease, according to age- and sex-standardized risk ratios (RR); age-, sex-, and education-standardized risk ratios (RRe); and multivariate logistic regression analyses (OR).
| RR [95% CI] | RRe [95% CI] | OR [95% CI] | |
|---|---|---|---|
| Chronic disease | 1.41 [1.40–1.43] | 1.11 [1.10–1.12] | 1.17 [1.16–1.18] |
| Activity limitations in: | |||
| Self-sufficiency | 1.75 [1.70–1.80] | 1.01 [0.97–1.04] | 0.96 [0.93–1.00] |
| Everyday life | 1.64 [1.61–1.66] | 1.18 [1.16–1.20] | 1.20 [1.17–1.23] |
| Learning-working | 1.59 [1.56–1.62] | 1.19 [1.17–1.22] | 1.24 [1.21–1.27] |
| Family life | 1.60 [1.53–1.67] | 1.22 [1.17–1.27] | 1.22 [1.16–1.28] |
| Transport | 1.54 [1.51–1.57] | 1.16 [1.14–1.19] | 1.03 [1.01–1.06] |
| Communication | 1.44 [1.36–1.51] | 0.86 [0.82–0.91] | 0.97 [0.92–1.03] |
| Community life | 1.14 [1.10–1.18] | 1.03 [0.99–1.07] | 1.02 [0.98–1.07] |
[95% CI]: 95% confidence interval; OR: odds ratio adjusted for housing conditions (walls quality, public utilities as bathroom, flush toilet, electricity, water and hot water supply, and heating system), and personal characteristics (sex, age, marital status, education, employment).
Each of the age- and sex-adjusted; age-, sex-, and education-adjusted; and multiply adjusted estimations showed a significant impact of Roma ethnicity on chronic disease occurrence, both within the Roma population and in the whole population. (Table 4)
Table 4.
Roma ethnicity’s association with the occurrence of having at least one chronic disease and activity limitations among those with chronic disease, according to age- and sex-standardized risk ratios (RR); age-, sex-, and education-standardized risk ratios (RRe); and multivariate logistic regression analyses (OR).
| Attributable Number of Cases among Roma [95% CI] | Attributable Percentage of Cases among Roma [95% CI] | Attributable Percentage of Cases in the Population [95% CI] | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Age-Sex Adjusted * | Age-Sex-Education Adjusted * | Multiple Adjusted ** | Age-Sex Adjusted * | Age-Sex-Education Adjusted * | Multiple Adjusted ** | Age-Sex Adjusted * | Age-Sex-Education Adjusted * | Multiple Adjusted ** | |
| Chronic diseased subjects in study population | 12,615 [12,395; 13,045] |
4299 [3944; 4648] |
6303 [5984; 6618] |
29.08% [28.57; 30.07] |
9.91% [9.09; 10.71] |
14.53% [13.79; 15.25] |
0.78% [0.76; 0.80] |
0.26% [0.24; 0.29] |
0.39% [0.37; 0.41] |
| Activity limitation among subjects with chronic disease in: | |||||||||
| Self-sufficiency | 1797 [1727; 1864] |
42 [−130; 161] |
−175 [−316; 0] |
42.86% [41.18; 44.44] |
0.99% [-3.09; 3.85] |
−4.17% [−7.53; 0] |
1.28% [1.23; 1.33] |
0.03% [−0.09; 0.12] |
−0.12% [−0.23; 0] |
| Everyday life | 6381 [6195; 6501] |
2494 [2255; 2725] |
2725 [2376; 3058] |
39.02% [37.89; 39.76] |
15.25% [13.79; 16.67] |
16.67% [14.53; 18.70] |
1.36% [1.32; 1.39] |
0.53% [0.48; 0.58] |
0.58% [0.51; 0.65] |
| Learning-working | 4957 [4796; 5113] |
2133 [1941; 2409] |
2586 [2319; 2840] |
37.11% [35.90; 38.27] |
15.97% [14.53; 18.03] |
19.35% [17.36; 21.26] |
2.27% [2.20; 2.34] |
0.98% [0.89; 1.10] |
1.19% [1.06; 1.30] |
| Family life | 774 [715; 828] |
372 [300; 439] |
372 [285; 452] |
37.50% [34.64; 40.12] |
18.03% [14.53; 21.26] |
18.03% [13.79; 21.88] |
1.84% [1.70; 1.97] |
0.89% [0.71; 1.04] |
0.89% [0.68; 1.07] |
| Transport | 3604 [3472; 3732] |
1418 [1262; 1641] |
299 [102; 582] |
35.06% [33.77; 36.31] |
13.79% [12.28; 15.97] |
2.91% [0.99; 5.66] |
1% [0.96; 1.03] |
0.39% [0.35; 0.45] |
0.08% [0.03; 0.16] |
| Communication | 470 [407; 519] |
−250 [−337; −152] |
−48 [−134; 45] |
30.56% [26.47; 33.77] |
−16.28% [−21.95; −9.89] |
−3.09% [−8.7; 2.91] |
1.19% [1.03; 1.32] |
−0.64%[−0.86; −0.39] | −0.12% [−0.34; 0.11] |
| Community life | 356 [263; 442] |
84 [−29; 190] |
57 [−59; 190] |
12.28% [9.09; 15.25] |
2.91% [−1.01; 6.54] |
1.96% [−2.04; 6.54] |
0.44% [0.32; 0.54] |
0.10% [−0.04; 0.23] |
0.07% [−0.07; 0.23] |
* by standardization; ** by logistic regression model.
3.2. Activity Limitation among Subjects with Chronic Disease
Table 2 summarizes the unadjusted frequencies of activity limitations among subjects with chronic disease by the studied functions. Each outcome was significantly more frequent in the Roma population compared to the non-Roma population.
Age- and sex-standardized measures showed a similar pattern. The self-sufficiency limitation was more frequent in the Roma population (RR = 1.75). The activity limitation in everyday life showed significantly higher frequency in Roma (RR = 1.64). The Roma ethnicity was also associated with increased risk of limitation among the Roma population in the fields of learning-working (RR = 1.59), family life (RR = 1.60), transport (RR = 1.54), communication (RR = 1.44), and community life (RR = 1.14). Each age- and sex-adjusted risk ratio and attributable fraction was highly significant. (Table 3 and Table 4)
According to the SPRe for activity limitations, there was no evidence of differences between the Roma and non-Roma populations with respect to self-sufficiency or community life-related activities. Significant risk elevation was observed in the fields of everyday life activities (RRe = 1.18, ARRoma = 15.25; ARPopulation = 0.53), learning-working (RRe = 1.19, ARRoma = 16.97; ARPopulation = 0.98), family life (RRe = 1.22, ARRoma = 18.03; ARPopulation = 0.89), and transport (RRe = 1.16, ARRoma = 13.79; ARPopulation = 0.39) among Roma with chronic disease. The activity limitation in communication was significantly less frequent among Roma people (RRe = 0.86, ARRoma = −16.28; ARPopulation = −0.64) (Table 3 and Table 4).
In the multiple logistic regression analyses, Roma ethnicity was associated with higher risk of activity limitation in the fields of everyday life activities (OR = 1.20, ARRoma = 16.67; ARPopulation = 0.58), learning-working (OR = 1.24, ARRoma = 19.35; ARPopulation = 1.19), family life (OR = 1.22, ARRoma = 18.03; ARPopulation = 0.89), and transport (OR = 1.03, ARRoma = 2.91; ARPopulation = 0.08). There was no significant difference in the activity limitation between the Roma and the non-Roma populations in the fields of self-sufficiency, communication, and community life (Table 3 and Table 4).
4. Discussion
4.1. Main Findings
The crude prevalence of chronic diseases among the Roma population was remarkably lower compared to that among non-Roma. The opposite difference was observed for each studied activity limitation, which can be attributed, theoretically, to the slower disease progression, resulting in fewer complications or to faster prognosis, resulting in early death. In light of the published results from many settings on the worse disease prognosis for Roma populations [14,17] and some data on the elevated mortality risk among Roma populations [16,17], the latter explanation seems to be more likely.
The analyses controlled for the younger age structure of the Roma population, with age and sex adjustment, demonstrating that Roma people have an elevated risk both for chronic disease occurrence and for each studied activity limitation. These results suggest that both disease development and prognosis among those with disease is faster among the Roma population.
After standardization by level of education, the risk of chronic disease was significantly mitigated but remained significant. This result shows that a considerable part of the age- and sex-adjusted excess risk was partly attributable to the poor education status of the Roma people. This result is in accordance with the findings of higher disease risk among the Roma populations [14,18,24,43,44]. The elevated age-, sex-, and education-corrected risk of activity limitations proved to be significant in the field of everyday life, learning-working, family life, and transport. While in the field of communication, this risk proved to be reduced among Roma people. This result was also consistent with previously published results [45,46]. Self-sufficiency and community life showed no difference among Roma and non-Roma patients with chronic disease.
The more extensive adjustment by multiple regression modeling confirmed the findings achieved by age, sex, and education adjustment. The only qualitative difference was that the activity limitation in communication was not significant in the regression modeling. Two quantitative modifications were also observed. The risk estimation was significantly higher for chronic disease occurrence and was significantly decreased for activity limitations in transport in the regression models.
On the basis of the multiple regression modeling, the impact of Roma ethnicity on chronic disease occurrence was 14.53%, showing that the Roma ethnicity is a distinct and important risk factor. On the other hand, the Roma ethnicity did not prove to be a risk factor among Roma people with chronic disease with regard to each of the studied activity limitations. In different fields, the impact varied between 0 and 19.35%.
The estimated population-level impact of 0.39% for chronic diseases and between 0 and 1.19% for activity limitations is modest at first glance. Taking into consideration that the Roma population is considerably underestimated by the census, for the actual size of the Roma population may be 870,000 as estimated by a Hungarian study focused on the estimation of the real population size using external Roma identifiers based on active contributions of the local governments [47], the corrected ARPopulation for chronic disease risk is 1.16% (95% CI: 1.10–1.22). Although this measure is not directly comparable to the percentage of total loss of disability-adjusted life years (DALY), since the consequences of chronic diseases are variable, leading to premature death and functional impairments, this ARPopulation is in the same range as the percentage of total DALY from road injuries (1.52%), breast cancer (1.30%), cardiomyopathy and myocarditis (1.24%), and alcohol use disorders (1.04%).
4.2. Strengths and Limitations
The main strength of this census-based investigation is that it covered the whole country, thus avoiding selection bias via preparation of a sampling frame. On the other hand, the census database comprised of nearly 10 million respondents, ensuring high statistical power for the analyses.
The achieved response rates (78.36% for chronic disease, 87.09% for activity limitation among persons with chronic disease) were relatively high compared with those observed in the Hungarian implementations of the European Health Interview Surveys (72% in 2009 and 62% 2014), thus resulting in smaller selection bias in the census-based evaluation than in the surveys.
However, this investigation could not avoid self-report-derived ethnicity misclassification and the remarkable underestimation of the Roma population’s size. Although this validity issue has to be acknowledged [48,49,50], the results from parallel applications of self-report and interviewer classifications in health surveys suggest that this validity issue has minor importance in the investigation of population-level health status differences between Roma and non-Roma people. It is a likely consequence of the fact that the self-declared Roma subjects tended to live in more segregated circumstances, have worse health status, and be less educated than those that did not declare Roma identity [21,51].
There was a misclassification with respect to the studied outcomes because of self-declaration. Because health status was not described in detail, the importance of this potential bias may be modest. On the other hand, low specificity of the outcome classification restricts the usefulness of census-derived data in elaborating and monitoring interventions, of course.
Our efforts to control for confounding effects was far from complete in this investigation. There were factors (e.g., the fact that income is difficult to capture in census settings) that should have been considered in the adjustment for sociodemographic status. Further bias was elicited by the lack of data on health services provided for participants. The variable availability and quality of care leads to variable perceptions of disease and activity limitations.
Because of the cross-sectional nature of the census, the elevated risks of activity limitations among persons with chronic disease cannot be interpreted as evidence for worse prognoses associated with chronic disease among Roma.
4.3. Implications
Taking into account that (1) the census can estimate the health impact of the Roma ethnicity among the Roma population and within the whole population, (2) it is enough to use the census-collected data to obtain reliable sociodemographic factors to adjust the estimations of Roma ethnicity-related risks, (3) the self-report-derived misclassification, which needs further investigations, seems not to be strong enough to prevent the application of census-derived findings in intervention planning and evaluation, the addition of the census to public health monitoring of the health of the Roma population seems to be justified.
However, the only available census-based evaluations are not able to meet the criteria of public health monitoring, mainly because the health status assessment is not detailed enough to provide decision-makers with an identifiable intervention target. Only the magnitude of Roma ethnicity-related health problems and the potential health gain that can be achieved by effective interventions among Roma can be identified by the census. While the details of such interventions can be ensured by population surveys and setting specific epidemiological investigations.
The proposal for the more intensive use of census data in health monitoring of the Roma population can be supported by the success of policies to reduce ethnic/racial inequalities in the United States and in the United Kingdom, which use census-derived ethnicity- and race-specific data in establishing and evaluating interventions [37,38,40].
5. Conclusions
Our investigations demonstrated that the Roma ethnicity is a distinct risk factor independent of sociodemographic status. It suggests significant impact among the Roma population for chronic disease occurrence accompanied with prognosis worsening influence on activity limitation in everyday life, learning-working, family life, and transport but not in self-sufficiency, communication, and community life. It is estimated that 1.16% of chronic diseases can be attributed to the Roma ethnicity in the whole population. Although, census data cannot determine in detail the targets for interventions, they can be used to estimate the general importance of the Roma ethnicity-related problems and the urgency of intervention, which is not achievable with population-level surveys and setting specific epidemiological investigations, according to previous experience. Census based impact assessment of Roma ethnicity on chronic disease occurrence of 1.16% was not described formerly neither by analytic investigations nor households surveys. Since sociodemographic risk adjustment can be carried out using census data, the census can provide useful data in spite of the uncertainties on account of self-declared ethnicity assessments. Altogether, the census, in addition to population-level surveys and subgroup-specific epidemiologic investigations, can significantly improve the health-monitoring system of the Roma population.
Appendix A
Table A1.
Housing conditions and personal characteristics of the study population and the differences between the Roma and non-Roma populations, evaluated by χ2-test.
| Total (%) | Ethnicity | p-Value | ||
|---|---|---|---|---|
| Roma (%) | Non-Roma (%) | |||
| Walls quality | ||||
| Brick or stone | 4,955,727 (65.99%) | 191,170 (67.08%) | 4,764,557 (65.95%) | <0.001 |
| Cast concrete or panel | 1,301,076 (17.33%) | 16,649 (5.84%) | 1,284,427 (17.78%) | |
| Adobe | 1,101,897 (14.67%) | 72,531 (25.45%) | 1,029,366 (14.25%) | |
| Wood | 45,261 (0.60%) | 1,055 (0.37%) | 44,206 (0.61%) | |
| Other | 105,707 (1.41%) | 3,587 (1.26%) | 102,120 (1.41%) | |
| Flush toilet | ||||
| Yes | 7,123,041 (94.87%) | 195,604 (68.70%) | 6,927,437 (95.90%) | <0.001 |
| No | 385,254 (5.13%) | 89,131 (31.30%) | 296,123 (4.10%) | |
| Public utilities as bathroom | ||||
| Yes | 7,174,205 (95.55%) | 203,279 (71.39%) | 6,970,926 (96.50%) | <0.001 |
| No | 334,135 (4.45%) | 81,456 (28.61%) | 252,679 (3.50%) | |
| Electricity | ||||
| Yes | 7,492,989 (99.78%) | 203,279 (71.39%) | 6,970,926 (96.50%) | <0.001 |
| No | 16,259 (0.22%) | 81,456 (28.61%) | 252,679 (3.50%) | |
| Water | ||||
| From pipeline | 7,346,624 (97.83%) | 233,224 (81.83%) | 7,113,400 (98.46%) | <0.001 |
| No piped water | 163,143 (2.17%) | 51,769 (18.17%) | 111,374 (1.54%) | |
| Hot water supply | ||||
| From pipeline or another way | 7,122,446 (94.85%) | 186,011 (65.27%) | 6,936,435 (96.02%) | <0.001 |
| No hot running water | 386,645 (5.15%) | 98,968 (34.73%) | 287,677 (3.98%) | |
| Heating system | ||||
| Heating | 7,504,952 (99.94%) | 284,405 (99.80%) | 7,220,547 (99.95%) | <0.001 |
| No heating | 4,399 (0.06%) | 576 (0.20%) | 3,823 (0.05%) | |
| Marital status | ||||
| Never married | 3,253,724 (42.40%) | 203,618 (69.21%) | 3,050,106 (41.33%) | <0.001 |
| Married | 2,913,135 (37.96%) | 62,367 (21.20%) | 2,850,768 (38.63%) | |
| Widowed | 765,382 (9.97%) | 10,728 (3.65%) | 754,654 (10.23%) | |
| Divorced | 742,366 (9.67%) | 17,476 (5.94%) | 724,890 (9.82%) | |
| Employment | ||||
| Working | 3,042,628 (39.65%) | 48,078 (16.34%) | 2,994,550 (40.57%) | <0.001 |
| Unemployed | 435,976 (5.68%) | 38,265 (13.01%) | 397,711 (5.39%) | |
| Retired | 1,928,973 (25.13%) | 22,740 (7.73%) | 1,906,233 (25.83%) | |
| Learning in education institution | 1,535,091 (20.00%) | 99,092 (33.68%) | 1,435,999 (19.46%) | |
| Dependent | 397,933 (5.19%) | 41,816 (14.21%) | 356,117 (4.83%) | |
| Receiving social benefits | 334,006 (4.35%) | 44,198 (15.02%) | 289,808 (3.93%) | |
| Total | 7,674,607 (100%) | 294,189 (100%) | 7,380,418 (100%) | |
Table A2.
Chronic disease prevalence of the study population and the differences between the Roma and non-Roma populations, evaluated by χ2-test.
| Total (%) | Ethnicity | p-Value | ||
|---|---|---|---|---|
| Roma (%) | Non-Roma (%) | |||
| Sex | ||||
| Male | 18.76% | 13.70% | 18.96% | <0.001 |
| Female | 23.38% | 15.82% | 23.38% | <0.001 |
| Age group (years) | ||||
| <35 | 5.32% | 5.98% | 5.28% | <0.001 |
| 35–59 | 19.68% | 29.05% | 19.39% | <0.001 |
| 60+ | 70.60% | 59.45% | 70.38% | <0.001 |
| Education | ||||
| Not-completed-education | 5.85% | 5.94% | 5.85% | <0.001 |
| Less than primary | 33.26% | 23.42% | 34.61% | <0.001 |
| Primary | 39.06% | 18.07% | 40.59% | <0.001 |
| Vocational | 20.27% | 16.34% | 20.35% | <0.001 |
| Highschool | 20.22% | 15.97% | 20.23% | <0.001 |
| Tertiary | 17.36% | 16.55% | 17.36% | <0.001 |
| Total | 21.19% | 14.75% | 21.45% | <0.001 |
Author Contributions
Conceptualization J.S.; methodology, J.S.; formal analysis, F.V., E.A.J., A.F., A.P., and V.S.; data curation, F.V.; interpretation, F.V., E.A.J., A.F., A.P., V.S. and J.S.; writing, F.V. and J.S.; supervision, R.Á.
Funding
The reported study was carried out in the framework of the “Public Health Focused Model Programme for Organising Primary Care Services Backed by a Virtual Care Service Centre” (SH/8/1). The Model Programme is funded by the Swiss Government via the Swiss Contribution Programme (SH/8/1) in agreement with the Government of Hungary. Additional source of funding was from the grant of Hungarian Academy of Sciences (MTA 11003, 2006TKI227), and from GINOP-2.3.2-15-2016-00005 project which was co-financed by the European Union and the European Regional Development Fund.
Conflicts of Interest
The authors declare no conflict of interest.
References
- 1.European Commission . Roma Health Report—Health Status of the Roma Population. European Commission; Brussels, Belgium: 2014. [Google Scholar]
- 2.Marmot M. Health Inequalities in the EU. European Commission Directorate-General for Health and Consumers: European Union; Brussels, Belgium: 2013. Final Report of a Consortium. [Google Scholar]
- 3.Van Dijk J.P. Roma health: Do we know enough? Int. J. Public Health. 2019;64:647–648. doi: 10.1007/s00038-019-01247-8. [DOI] [PubMed] [Google Scholar]
- 4.Foldes M.E., Covaci A. Research on Roma health and access to healthcare: State of the art and future challenges. Int. J. Public Health. 2012;57:37–39. doi: 10.1007/s00038-011-0312-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Hajioff S., McKee M. The health of the Roma people: A review of the published literature. J. Epidemiol. Community Health. 2000;54:864–869. doi: 10.1136/jech.54.11.864. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Koupilova I., Epstein H., Holcik J., Hajioff S., McKee M. Health needs of the Roma population in the Czech and Slovak Republics. Soc. Sci. Med. 2001;53:1191–1204. doi: 10.1016/S0277-9536(00)00419-6. [DOI] [PubMed] [Google Scholar]
- 7.Kosa Z., Szeles G., Kardos L., Kosa K., Nemeth R., Orszagh S., Fesus G., McKee M., Adany R., Voko Z. A comparative health survey of the inhabitants of Roma settlements in Hungary. Am. J. Public Health. 2007;97:853–859. doi: 10.2105/AJPH.2005.072173. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Gerevich J., Bacskai E., Czobor P., Szabo J. Substance use in Roma and non-Roma adolescents. J. Nerv. Ment. Dis. 2010;198:432–436. doi: 10.1097/NMD.0b013e3181e07d51. [DOI] [PubMed] [Google Scholar]
- 9.Toth M.D., Adam S., Zonda T., Birkas E., Purebl G. Risk factors for multiple suicide attempts among Roma in Hungary. Transcult. Psychiatry. 2018;55:55–72. doi: 10.1177/1363461517731703. [DOI] [PubMed] [Google Scholar]
- 10.Paulik E., Nagymajtenyi L., Easterling D., Rogers T. Smoking behaviour and attitudes of Hungarian Roma and non-Roma population towards tobacco control policies. Int. J. Public Health. 2011;56:485–491. doi: 10.1007/s00038-011-0252-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Arora V.S., Kuhlbrandt C., McKee M. An examination of unmet health needs as perceived by Roma in Central and Eastern Europe. Eur. J. Public Health. 2016;26:737–742. doi: 10.1093/eurpub/ckw004. [DOI] [PubMed] [Google Scholar]
- 12.Balazs P., Rakoczi I., Grenczer A., Foley K.L. Birth-weight differences of Roma and non-Roma neonates—Public health implications from a population-based study in Hungary. Cent. Eur. J. Public Health. 2014;22:24–28. doi: 10.21101/cejph.a3841. [DOI] [PubMed] [Google Scholar]
- 13.Sandor J., Nagy A., Foldvari A., Szabo E., Csenteri O., Vincze F., Sipos V., Kovacs N., Palinkas A., Papp M., et al. Delivery of cardio-metabolic preventive services to Hungarian Roma of different socio-economic strata. Fam. Pract. 2017;34:83–89. doi: 10.1093/fampra/cmw102. [DOI] [PubMed] [Google Scholar]
- 14.Sudzinova A., Nagyova I., Studencan M., Rosenberger J., Skodova Z., Vargova H., Middel B., Reijneveld S.A., van Dijk J.P. Roma coronary heart disease patients have more medical risk factors and greater severity of coronary heart disease than non-Roma. Int. J. Public Health. 2013;58:409–415. doi: 10.1007/s00038-013-0462-5. [DOI] [PubMed] [Google Scholar]
- 15.Silarova B., van Dijk J.P., Nagyova I., Rosenberger J., Reijneveld S.A. Differences in health-related quality of life between Roma and non-Roma coronary heart disease patients: The role of hostility. Int. J. Public Health. 2014;59:271–277. doi: 10.1007/s00038-013-0490-1. [DOI] [PubMed] [Google Scholar]
- 16.Kolvek G., Rosicova K., Rosenberger J., Podracka L., Stewart R.E., Nagyova I., Reijneveld S.A., van Dijk J.P. End-stage renal disease among Roma and non-Roma: Roma are at risk. Int. J. Public Health. 2012;57:751–754. doi: 10.1007/s00038-012-0365-x. [DOI] [PubMed] [Google Scholar]
- 17.Sudzinova A., Nagyova I., Rosenberger J., Studencan M., Vargova H., Middel B., van Dijk J.P., Reijneveld S.A. Seven years’ mortality in Roma and non-Roma patients after coronary angiography. Eur. J. Public Health. 2015;25:765–769. doi: 10.1093/eurpub/ckv057. [DOI] [PubMed] [Google Scholar]
- 18.Kosa Z., Moravcsik-Kornyicki A., Dioszegi J., Roberts B., Szabo Z., Sandor J., Adany R. Prevalence of metabolic syndrome among Roma: A comparative health examination survey in Hungary. Eur. J. Public Health. 2015;25:299–304. doi: 10.1093/eurpub/cku157. [DOI] [PubMed] [Google Scholar]
- 19.Phinney J.S., Ong A.D. Conceptualization and measurement of ethnic identity: Current status and future directions. J. Couns. Psychol. 2007;54:271–281. doi: 10.1037/0022-0167.54.3.271. [DOI] [Google Scholar]
- 20.European Commission . Communication from the Commission to the European Parliament and the Council, Report on the Evaluation of the EU Framework for National Roma Integration Strategies up to 2020, Report on the Evaluation of the EU Framework for National Roma Integration Strategies up to 2020 COM (2018) European Commission; Brussels, Belgium: 2018. [Google Scholar]
- 21.Ladányi J., Szelényi I. The social construction of Roma ethnicity in Bulgaria, Romania and Hungary during market transition. Rev. Sociol. 2001;7:79–89. doi: 10.1556/RevSoc.7.2001.2.5. [DOI] [Google Scholar]
- 22.Sepkowitz K.A. Health of the world’s Roma population. Lancet. 2006;367:1707–1708. doi: 10.1016/S0140-6736(06)68746-1. [DOI] [PubMed] [Google Scholar]
- 23.Fónai M., Fábián G., Filepné Nagy É., Pénzes M. Poverty, health and ethnicity: The empirical experiences of researches in Northeast-Hungary. Rev. Sociol. 2008;14:63–91. doi: 10.1556/RevSoc.14.2008.2.3. [DOI] [Google Scholar]
- 24.Masseria C., Mladovsky P., Hernandez-Quevedo C. The socio-economic determinants of the health status of Roma in comparison with non-Roma in Bulgaria, Hungary and Romania. Eur. J. Public Health. 2010;20:549–554. doi: 10.1093/eurpub/ckq102. [DOI] [PubMed] [Google Scholar]
- 25.Geckova A.M., Babinska I., Bobakova D., Veselska Z.D., Bosakova L., Kolarcik P., Jarcuska P., Pella D., Halanova M. Socioeconomic characteristics of the population living in Roma settlements and their association with health and health-related behaviour. Cent. Eur. J. Public Health. 2014;22:57–64. doi: 10.21101/cejph.a3903. [DOI] [PubMed] [Google Scholar]
- 26.Sandor J., Palinkas A., Vincze F., Kovacs N., Sipos V., Korosi L., Falusi Z., Pal L., Furjes G., Papp M., et al. Healthcare Utilization and All-Cause Premature Mortality in Hungarian Segregated Roma Settlements: Evaluation of Specific Indicators in a Cross-Sectional Study. Int. J. Environ. Res. Public Health. 2018;15:1835. doi: 10.3390/ijerph15091835. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Pappa E., Chatzikonstantinidou S., Chalkiopoulos G., Papadopoulos A., Niakas D. Health-Related Quality of Life of the Roma in Greece: The Role of Socio-Economic Characteristics and Housing Conditions. Int. J. Environ. Res. Public Health. 2015;12:6669–6681. doi: 10.3390/ijerph120606669. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Fernandez-Feito A., Pesquera-Cabezas R., Gonzalez-Cobo C., Prieto-Salceda M.D. What do we know about the health of Spanish Roma people and what has been done to improve it? A scoping review. Ethn. Health. 2019;24:224–243. doi: 10.1080/13557858.2017.1315373. [DOI] [PubMed] [Google Scholar]
- 29.Filakovska Bobakova D. Challenges for research, policy and practice in the field of Roma health. Int. J. Public Health. 2019;64:645–646. doi: 10.1007/s00038-019-01254-9. [DOI] [PubMed] [Google Scholar]
- 30.European Commission . Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions. An EU Framework for National Roma Integration Strategies up to 2020, COM (2011) 173/4. European Commission; Brussels, Belgium: 2011. [Google Scholar]
- 31.Ministry of Public Administration and Justice State Secretariat for Social Inclusion National Social Inclusion Strategy—Extreme Poverty, Child Poverty, the Roma (2011–2020) Ministry of Public Administration and Justice State Secretariat; Budapest, Hungary: 2011. [Google Scholar]
- 32.European Commission The Commission’s assessment of Hungary’s National Strategy. [(accessed on 1 August 2019)]; Available online: https://ec.europa.eu/info/sites/info/files/assessment_hungary_national_strategy_2012_en.pdf.
- 33.European Commission The Commission’s Assessment of the Implementation of Hungary’s National Strategy. [(accessed on 1 August 2019)]; Available online: https://ec.europa.eu/info/sites/info/files/assessment_implementation_hungary_national_strategy_2014_en.pdf.
- 34.Farkas L. Analysis and Comparative Review of Equality Data Collection Practices in the European Union Data Collection in the Field of Ethnicity. European Commission; Brussels, Belgium: 2017. [Google Scholar]
- 35.Hannigan A., Basogomba A., LeMaster J., Nurse D., O’Reilly F., Roura M., Villarroel N., MacFarlane A. Ethnic Minority Health in Ireland-Co-creating knowledge (EMH-IC): A participatory health research protocol. BMJ Open. 2018;8:e026335. doi: 10.1136/bmjopen-2018-026335. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.European Union Agency for Fundamental Rights Second European Union Minorities and Discrimination Survey Roma—Selected Findings. European Union Agency; Luxembourg: 2018. [Google Scholar]
- 37.Dallo F.J., Al Snih S., Ajrouch K.J. Prevalence of disability among US- and foreign-born Arab Americans: Results from the 2000 US Census. Gerontology. 2009;55:153–161. doi: 10.1159/000151538. [DOI] [PubMed] [Google Scholar]
- 38.Wilding S., Martin D., Moon G. The impact of limiting long term illness on internal migration in England and Wales: New evidence from census microdata. Soc. Sci. Med. 2016;167:107–115. doi: 10.1016/j.socscimed.2016.08.046. [DOI] [PubMed] [Google Scholar]
- 39.Vandenheede H., Willaert D., De Grande H., Simoens S., Vanroelen C. Mortality in adult immigrants in the 2000s in Belgium: A test of the ‘healthy-migrant’ and the ‘migration-as-rapid-health-transition’ hypotheses. Trop. Med. Int. Health. 2015;20:1832–1845. doi: 10.1111/tmi.12610. [DOI] [PubMed] [Google Scholar]
- 40.Allik M., Brown D., Dundas R., Leyland A.H. Differences in ill health and in socioeconomic inequalities in health by ethnic groups: A cross-sectional study using 2011 Scottish census. Ethn. Health. 2019;18:1–19. doi: 10.1080/13557858.2019.1643009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Hungarian Central Statistical Office Dwelling Questionnaire. [(accessed on 1 August 2019)]; Available online: http://www.ksh.hu/nepszamlalas/docs/kerdoivek/lakas_angol.pdf.
- 42.Hungarian Central Statistical Office Personal Questionnaire. [(accessed on 1 August 2019)]; Available online: http://www.ksh.hu/nepszamlalas/docs/kerdoivek/szemely_angol.pdf.
- 43.Voko Z., Csepe P., Nemeth R., Kosa K., Kosa Z., Szeles G., Adany R. Does socioeconomic status fully mediate the effect of ethnicity on the health of Roma people in Hungary? J. Epidemiol. Community Health. 2009;63:455–460. doi: 10.1136/jech.2008.079715. [DOI] [PubMed] [Google Scholar]
- 44.Kolarcik P., Geckova A.M., Orosova O., van Dijk J.P., Reijneveld S.A. To what extent does socioeconomic status explain differences in health between Roma and non-Roma adolescents in Slovakia? Soc. Sci. Med. 2009;68:1279–1284. doi: 10.1016/j.socscimed.2008.12.044. [DOI] [PubMed] [Google Scholar]
- 45.Latorre-Arteaga S., Gil-Gonzalez D., Vives-Cases C., La Parra Casado D. Vision and Hearing Health Inequities in the Roma population: A National Cross-Sectional Study in Spain. J. Immigr. Minor. Health. 2017;19:1304–1314. doi: 10.1007/s10903-016-0489-9. [DOI] [PubMed] [Google Scholar]
- 46.La Parra Casado D., Gil Gonzalez D., de la Torre Esteve M. The social class gradient in health in Spain and the health status of the Spanish Roma. Ethn. Health. 2016;21:468–479. doi: 10.1080/13557858.2015.1093096. [DOI] [PubMed] [Google Scholar]
- 47.Pásztor I.Z., Pénzes J., Pálóczi Á. The number and spatial distribution of the Roma population in Hungary—In the light of different approaches. Folia Geogr. 2016;58:5–21. [Google Scholar]
- 48.Hahn R.A., Truman B.I., Barker N.D. Identifying ancestry: The reliability of ancestral identification in the United States by self, proxy, interviewer, and funeral director. Epidemiology. 1996;7:75–80. doi: 10.1097/00001648-199601000-00013. [DOI] [PubMed] [Google Scholar]
- 49.Telles E.E., Lim N. Does it matter who answers the race question? Racial classification and income inequality in Brazil. Demography. 1998;35:465–474. doi: 10.2307/3004014. [DOI] [PubMed] [Google Scholar]
- 50.Ivanov A., Kling J., Kagin J. Roma Inclusion Working Papers. United Nations Development Programme; Bratislava, Slovakia: 2012. Integrated household surveys among Roma populations: One possible approach to sampling used in the UNDP-World Bank-EC Regional Roma Survey 2011. [Google Scholar]
- 51.Janka E.A., Vincze F., Adany R., Sandor J. Is the Definition of Roma an Important Matter? The Parallel Application of Self and External Classification of Ethnicity in a Population-Based Health Interview Survey. Int. J. Environ. Res. Public Health. 2018;15:353. doi: 10.3390/ijerph15020353. [DOI] [PMC free article] [PubMed] [Google Scholar]