Placing Families in Context: Challenges for Cross-National Family Research

Abstract

Cross-national comparisons constitute a valuable strategy to assess how broader cultural, political, and institutional contexts shape family outcomes. One typical approach of cross-national family research is to use comparable data from a limited number of countries, fit similar regression models for each country, and compare results across country-specific models. Increasingly, researchers are adopting a second approach, which requires merging data from many more societies and testing multilevel models using the pooled sample. Although the second approach has the advantage of allowing direct estimates of the effects of nation-level characteristics, it is more likely to suffer from the problems of omitted-variable bias, influential cases, and measurement and construct nonequivalence. I discuss ways to improve the first approach's ability to infer macrolevel influences, as well as how to deal with challenges associated with the second one. I also suggest choosing analytical strategies according to whether the data meet multilevel models’ assumptions.

Social scientists have long viewed cross-national comparisons as a valuable method to both assess the generalizability of findings derived from single-country studies and demonstrate the influences of broader contexts (Elder, 1976; Kohn, 1987). Because welfare policies, cultural norms, and social expectations and support for different members of each society are all likely to shape family processes and outcomes (Cooke & Baxter, 2010; Gornick, Meyers, & Ross, 1998; Heuveline & Timberlake, 2004; Ono, 2003; Park, 2007; Pong, Dronkers, & Hampden-Thompson, 2003; Villarreal & Shin, 2008), a cross-national comparative framework is especially useful for family research. With the rising availability of comparable survey data across countries and statistical methods for analyzing such data, studies of family issues are increasingly extending their scope to multinational settings and focusing on macrolevel influences (e.g., Breen & Cooke, 2005; Craig & Mullan, 2011; Kalmijn, 2013; Sullivan, Coltrane, McAnnally, & Altintas, 2009; Treas, Lippe, & ChloeTai, 2011; Yodanis & Lauer, 2007; Yu, 2005).

Despite improvements of both data and methods, cross-national family research continues to face challenges, partly because the macrolevel forces used to explain cross-national differences in certain outcomes are typically difficult to measure, and partly because there often are many other between-country differences that could affect the interpretations of findings. In some ways, the challenges for cross-national family research are only becoming greater, as researchers are less and less likely to confine their interest to systematically describing a particular set of countries. Instead, cross-national family research increasingly aims to develop general arguments about how society-level characteristics condition family processes, treating the specific countries examined merely as contexts or units of its analyses (e.g., Kalmijn, 2013; Yodanis & Lauer, 2007). When attempting to establish macro-micro relationships that are generalizable to a wide range of countries, it is even more important to ensure that each measured family condition has the same meaning across countries and that explanations for observed macro-micro relationships are not confounded by other factors that differ across societies, such as the potentially differing processes through which people of different countries sort themselves into various types of families.

In this paper, I discuss the challenges that cross-national family researchers frequently encounter and propose ways of dealing with these challenges. I limit the discussion to research that utilizes data from multiple countries and features explicit and systematic comparisons between countries. Although many studies based on a single country are implicitly comparative, as they frequently relate their findings to ones from other countries (e.g., Villarreal & Shin, 2008; Yu & Su, 2006), I exclude such studies because the methodological requirements for the comparisons in which they engage are considerably fewer than those for explicit cross-national research. Moreover, to enable a more focused discussion, I mainly address issues related to quantitative cross-national research. Needless to say, qualitative research comparing multiple countries also faces methodological challenges, given that researchers must have a comprehensive understanding of the languages and cultural differences involved. The challenges, however, tend to differ from those pertaining to quantitative cross-national analyses.

ADDRESSING FAMILY ISSUES WITH CROSS-NATIONAL COMPARISONS

Before considering the methodological challenges for cross-national family research, it is important to address how and why family studies utilize international comparisons as an analytical strategy. One main reason family scholars employ cross-national comparisons is to uncover how macrolevel social forces shape family processes. Much previous research suggests that nation-level factors play crucial roles in explaining variations in family-related outcomes (Cooke & Baxter, 2010). With regards to the likelihood to cohabit, for instance, the differences between subgroups within any given country in Europe are small compared to cross-national differences (Heuveline & Timberlake, 2004; Kiernan, 2000). Without examining the reasons for cross-national differences, our knowledge of why individuals vary in their tendencies of cohabiting, or in other family processes, is ultimately limited.

Generally speaking, cross-national comparisons are especially suitable for answering two types of explanatory questions. The first concerns the effects of public policies on family-related outcomes. Because variations in family and welfare policies are most likely to be observed at the national level, researchers interested in policy effects are almost required to utilize international comparisons. A large proportion of existing cross-national family research indeed centers on the consequences of social policies (Cooke & Baxter, 2010; Pettit & Hook, 2009). Specifically, many scholars inquire about the various outcomes resulting from cross-national differences in policies supporting families of young children. They have shown that greater state support through parental-leave provisions and tax and childcare schemes decrease mothers’ employment discontinuity (Gornick, Meyers, & Ross, 1997), wage penalties on mothers’ part-time or intermittent employment (Stier, Lewin-Epstein, & Braun, 2001), both men's and women's sense of work-family conflict (Stier, Lewin-Epstein, & Braun, 2012), less-educated fathers’ difficulty in finding time for children (Sayer, Gauthier, & Furstenberg, 2004), the gap in children's academic performance between single- and two-parent families (Pong et al., 2003), the likelihood of living in poverty for families headed by single mothers (Misra, Moller, & Budig, 2007), and full-time working wives’ disadvantage in happiness relative to homemakers (Treas et al., 2011). Social policies that help ease women's difficulty in combining jobs and child rearing are also argued to give rise to public approval of mothers’ participation in the labor force (Charles & Cech, 2010; Sjöberg, 2004). Other researchers rely on cross-national comparisons to determine whether and which policies contribute to a rise in fertility (Gauthier, 2007). Their research reveals, for example, the positive influences of policies that offer cash benefits to families and increase the accessibility of formal childcare and parental leaves (Castles, 2003; Gauthier & Hatzius, 1997; Rønsen, 2004). Other policies of interest to family scholars are those related to taxes and welfare transfers, as they can potentially reduce family disparities within countries. Using an international comparative analysis, Heuveline and Weinshenker (2008), for instance, demonstrate that the different tax and transfer schemes in the United States largely account for its exceptionally high child poverty rate, compared to European countries.

The second type of questions for which family scholars must use cross-national research designs to address concerns the issue of how the social or cultural context moderates individual- or family-level associations, such as the relationship between education and the likelihood of marriage and cohabitation, the patterns of domestic division of labor, the extent of intergenerational transmission of divorce, the marriage dissolution rate of couples with prior cohabitation experience, and the influence of single-parent, female-headed, or three-generational households on children's welfare. Because societies differ in their ideologies, institutions, and political environments, a given individual or household characteristic, even when it is defined in the same way, can have different effects across countries (Cooke & Baxter, 2010; Kohn, 1987). One example of a nation-level factor that has been shown to moderate family dynamics is gender relations in society. Not only does women's relative economic status in the country affect the overall equality in household division of labor (Breen & Cooke, 2005), but macrolevel gender equality also conditions the effects of husbands’ and wives’ time availability, relative income, and gender ideology on women's share of domestic work (Fuwa, 2004; Knudsen & Waerness, 2008). Likewise, the society's level of gender equity shapes the gap in household-work equality between couples that cohabited before marriage and those that did not (Batalova & Cohen, 2002). Moreover, previous research has found that in countries with high levels of gender-role differentiation, women with higher economic standings are less likely to marry, whereas women's higher economic status is conducive to marriage in national contexts with relatively low levels of gender role differentiation (Kalmijn, 2013; Ono, 2003).

In addition to the gender-related context, the extent to which divorce, cohabitation, or nontraditional family formation is prevalent and institutionalized in society has also been shown to condition various family outcomes. For instance, high acceptance of divorce at the national level is associated with more equal distribution of work among married couples and greater intergenerational transmission of divorce (Dronkers & Härkönen, 2008; Yodanis, 2005). Similarly, the national cohabitation rate conditions the effects of cohabitation on individual wellbeing and union stability (Liefbroer & Dourleijn, 2006; Soons & Kalmijn, 2009). Other national characteristics that contour family relations in society are equally likely to matter. The prevalence of extended family support in society, for example, is thought to reduce the relative disadvantage of growing up with single parents (Park, 2007).

Needless to say, not all cross-national family research aims to establish generalizable links between macrolevel forces—whether they be family policies or cultural norms—and microlevel outcomes. Some researchers have used multinational data for more descriptive purposes, such as mapping out how nations cluster together in family dynamics or developing ideal types in family processes among countries (e.g., Heuveline & Timberlake, 2004; Treas & Widmer, 2000). Others utilize cross-national analyses to assess the universality of a given argument. Using data from 48 countries, for instance, Hill and colleagues (2004) showed that a work-family interface model that links job flexibility to enhanced work-family fit and increased job satisfaction has cross-cultural applicability. In contrast, a study of 18 countries indicated that the U.S. finding that couples with daughters have higher divorce rates can hardly be replicated internationally (Diekmann & Schmidheiny, 2004).

Although the cross-national research design can be well suited for questions that are both explanatory and descriptive in nature, the methodological challenges are generally greater when researchers aim to explain cross-national differences, or at least link certain macrolevel forces to cross-national variations in family issues. Unlike merely testing the applicability of a theoretical framework in different cultures or developing ideal types across regimes, the attempt to identify impacts of certain policies or the social context's moderating effects requires family scholars to establish some sort of causal relationship. Making causal claims always implies methodological challenges (Gangl, 2010; Moffitt, 2005). The challenges might be even greater when the cause pertains to nation-level conditions, as it tends to be difficult to ensure measurement accuracy across cultures, or to take into account all other potentially relevant macrolevel factors. My discussion of the challenges facing cross-national family research, as well as my recommendations in response to these challenges, is largely based on the assumption that researchers’ intention, at least in part, is to explain cross-national differences.

Before moving to the discussion about the challenges for cross-national family research, I should also note that while international comparisons can be utilized to answer different types of explanatory questions, the actual analytical approach researchers adopt often depends on the number of countries included in the comparison, rather than the type of questions addressed. This is because having a large set of countries in a comparative study enables researchers to directly measure and model macrolevel forces, whereas having relatively few countries precludes such an analysis. In some ways, because of the possibility of directly testing the effect of a given policy, cross-national family research with a large set of nations is especially suitable for addressing the question about policy influences (e.g., Hook, 2006, 2010; Treas et al., 2011), though some researchers still infer policy influences using comparisons of relatively few countries (e.g., Rosenfeld & Kalleberg, 1990; Sayer et al., 2004). By contrast, the question of the broader context's moderating effects may very well be addressed with different analytical approaches, depending on the number of countries included in the study (e.g., Park, 2007; Yodanis, 2005). In any case, given that the number of countries included in a cross-national study is often critical to researchers’ options of analytical strategies, and the different options imply different challenges, I provide separate discussions of the related challenges based on the number of nations involved, rather than the specific research questions to be answered. Although the challenges for the two empirical approaches, employing relatively many and few nations, respectively, are not mutually exclusive, their relevance to each approach differs.

CROSS-NATIONAL RESEARCH WITH A SMALL SET OF NATIONS

A typical approach of cross-national research on family issues employs comparable survey data from a small number of countries, usually two to six of them (e.g., Craig & Mullan, 2011; Heimdal & Houseknecht, 2003; Ono, 2003; Park, 2007; Sayer et al., 2004; Sullivan et al., 2009; Yu, 2005). Based on existing literature or other sources of evidence on cross-national differences in cultural norms, social practices, institutional arrangements, or family and welfare policies, researchers develop hypotheses about how given individual-level predictors will yield different results in different countries. With the proposed hypotheses in mind, researchers then fit identical statistical models for each country. In this approach, a side-by-side comparison of coefficients across countries is used to support or reject hypotheses concerning macrolevel factors. Although I refer to this approach as a small-country-sample approach, as most research comparing a small set of countries relies on it, a few studies have used as many as 14 nations adopting a similar analytical strategy (e.g., Gornick et al., 1997). Strictly speaking, what makes this approach different from the next approach I discuss, one that involves a large set of countries, is not the number of countries examined in the study, but that it draws conclusions mainly from comparisons of microlevel results between country-specific models. Because of the reliance on comparing microlevel results across nations, many researchers use this approach to address how the cultural or social context moderates individual- or household-level associations (e.g., Heimdal & Houseknecht, 2003; Ono, 2003; Park, 2007), rather than directly measuring effects of family policies. It is nevertheless common for researchers employing side-by-side comparisons among a small number of countries to cite policy differences as one of the main reasons for observed differences in the microlevel associations (Craig & Mullan, 2011; Lewis, Campbell, & Huerta, 2008; Sayer et al., 2004). In this sense, family researchers addressing the social context's moderating effect and a policy's influence both may use the small-country-set comparative approach referred here.

The primary challenge for those using the approach just described has to do with the tendency to infer macrolevel influences from comparisons of microlevel results. When researchers employ only a handful of countries—in many cases just two or three—in their studies, the number of degrees of freedom is insufficient for modeling country-level differences thought to be influential, even if they pool the data from all the countries together. Because the nation-level force presumed to be responsible for the different microlevel results between countries is ultimately unmeasured and left out of the statistical models, the provided explanations in this type of research could seem post hoc and be challenged by alternative explanations.

To illustrate the challenges facing comparative family research on a relatively small number of countries, let's consider studies examining the consequences of single parenthood. Family researchers have long been interested in the disparities in educational attainment, health, and economic well-being between children from two- and single-parent families (Biblarz & Gottainer, 2000; Downey, Ainsworth-Darnell, & Dufur, 1998; Hampden-Thompson, 2013; Hao, 1996; MacLanahan & Sandefur, 1997; Pong, 1998). In the case of educational achievement, although U.S. children from single-parent families are generally disadvantaged, cross-national research has shown that the effect of single parenthood varies considerably by national context (Pong et al., 2003). In a few Asian countries, for instance, the disadvantage is virtually nonexistent (Park, 2007). Although, with separate models for each country, researchers can easily test cross-national differences in the coefficients for single- versus two-parent families, to explain such differences is no simple task. Different countries tend to have different family and welfare policies that may offset the economic hardship and time squeeze faced by single parents to different extents. The national cultural norms that lead to differing levels of support from extended family members, or different degrees of discrimination against single mothers, may also affect the well-being of children from one-parent households. To make matters more complicated, countries in which policies provide more support for single parents may also have greater cultural acceptance of single parenthood, less discrimination against single mothers and their children, and generally more social support, from extended families or other ties, for single parents. Without specifically modeling nation-level differences in policies, cultural norms, social expectations, or public opinions, it is difficult to ascertain the factors accounting for the differing effects of single-parenthood on children's academic achievement across countries.

The difficulty of explaining different effects of single parenthood across countries also arises from the fact that the selection processes into single parenthood tend to vary by national context. In some societies, single parenthood is predominately a result of widowhood, whereas in others it has more to do with divorce or nonmarital childbearing. Because single parenthood resulting from widowhood, divorce, and foregone marriage may receive different amounts of social support from the absent parent's extended kin and be perceived differently by other people, the children may fare differently. Moreover, in societies where widowhood is the primary cause of single parenthood, one-parent families may be less selective than their counterparts in societies where most single parents were divorced or never married, making the children's educational outcomes in the former societies more similar to those from two-parent families. Even among countries in which divorce is the primary cause of single parenthood, different types of people might be more likely to experience divorce, depending on the social acceptability of divorce and the level of economic penalty of marital disruption on women in the society. Rather than variations in family policies, cultural norms, or social discrimination, the relative educational disadvantages for children from single-parent families may differ by country because, for instance, mothers with greater long-term income potential are more likely to choose divorce in some societies, whereas those with lower economic prospects are more likely to raise children alone in others; and mothers’ financial security ultimately accounts for children's educational outcomes. Although cross-national analyses with separate models for each country generally include controls for a wide range of individual- or family-level characteristics, unmeasured differences in the characteristics of single-parent households may remain between countries, making it difficult for researchers to rule out the selection process as the explanation for cross-national differences in the effects of single parenthood.

Even if researchers can take into account all the factors that are simultaneously associated with individuals’ entry into single parenthood and their children's academic performance, we still may not know whether cross-national differences in the educational disadvantages of children from single-parent families result mainly from differences in the compositions of single-parent families or in the returns to the characteristics of such families. For example, single- and two-parent families may differ in both the mother's education and the effects of mother's schooling on children's academic performance. Because single mothers may experience greater time constraints than mothers in two-parent households, the former's time spent with children may not vary as much according to their schooling as the latter's. Mother's education therefore may have a weaker effect on children's academic achievement in one-parent households. The gap in the returns to mother's education, however, is likely to differ by national context, as different societies provide differing levels of support to alleviate the time squeeze for single mothers. In this sense, cross-national variations in children's educational disadvantages could result from the differing distributions of characteristics between single- and two-parent households or differential gaps in the returns to family characteristics between the two types of households, or, as often is the case, both.

Although my illustration above centers on single-parent families and their children's outcomes, other topics of cross-national family research are just as likely to suffer from the same issues. For example, studies of class or racial/ethnic differences in marriage and divorce rates, or in children's health and education, across a small number of countries would face difficulty in disentangling many macro-level cross-national differences that could simultaneously affect the association between class or race/ethnicity and the outcome of interest. Research on how immigrant or cohabited households fare differently in different countries would encounter the problem that the selection process for immigrants or cohabiters varies by country. Similarly, cross-national research examining intergenerational transmission of divorce must consider that international variations both in the distribution of characteristics between those with and without divorced parents and in the gap of the influence of each characteristic may contribute to the inter-country difference in the transmission rate. How to address these potential problems, therefore, is likely to concern family scholars with a wide range of research questions.

Building a Convincing Case

Researchers comparing a small number of countries cannot alter their inability to statistically examine country-level factors that might explain cross-national differences in family dynamics and disparities. Nevertheless, following certain practices may help researchers build a convincing case. To begin, the more evidence researchers can provide to document the national differences that may lead to different patterns at the individual or family level between countries, the more convincing their macrolevel explanations will be. In addition to previous research findings for each of the countries examined, including aggregate statistics from other sources or ethnographic data may help further substantiate the claim that there are indeed cross-national differences in cultural, social, or other aspects that may account for variations in family outcomes. Given that two countries often differ simultaneously in a variety of public policies, cultural norms, and social institutions, to be able to conclude which factor accounts more for the family-level differences of interest, researchers may want to consider alternative inter-country differences that might yield similar micro-level results. Ideally, researchers should provide evidence that, say, the cultural and institutional differences between the two countries examined are not sufficiently significant to produce the observed family-level difference, whereas the policy gap is consistent with that difference. Such an approach is line with John Stuart Mill's method of difference. With Mill's methods in mind, choosing countries that share considerable similarities, with only a few exceptions in a particular area, or countries that differ in cultural, political, and social dimensions in ways that would lead to different predictions in family-level differences, should undoubtedly be researchers’ first step toward conducting convincing cross-national family research with a small country sample.

Next, we ought to keep in mind that any cultural, institutional, or policy difference that could lead to a cross-national difference in a certain family outcome is also likely to affect individual- or family-level results in other regards. It therefore would be more convincing to develop a series of hypotheses that are consistent with the macrolevel difference argued to be influential, rather than focusing on just one individual- or family-level difference. With respect to how growing up with a single parent affects children's educational achievement, for example, if public policies that provide single parents more childcare support are expected to reduce the negative effect of single parenthood by alleviating their time squeeze, the gap in the returns to mother's education between single- and two-parent families could also be expected to be smaller in countries with such policies. Likewise, although having an additional sibling could be more detrimental to children's education in single-parent than two-parent families in all countries, public policies argued to offset the detrimental effect of single parenthood should also lead to a comparatively small gap in the effects of sibship size between single- and two-parent households. By linking macrolevel variations to multiple microlevel differences, researchers can further strengthen their arguments.

Because one major challenge for cross-national family research has to do with how different national contexts facilitate different selection processes, another way for those examining a small set of countries to build a convincing case is to take advantage of the various remedies for the selectivity problem that single-country studies have adopted. As discussed earlier, researchers who study children from single-parent families often face difficulty ascertaining the causal effect of single parenthood, because there are likely unobserved characteristics that both lead individuals to select into single parenthood and shape their children's outcomes. Other frequently studied topics in family research, such as the effect of cohabitation on future marital dissolution or the couple's well-being, also encounter a similar selectivity problem. In the context of cross-national comparisons, selectivity is even more of a concern, because the selection mechanisms are likely to differ between countries.

One common approach to address the selectivity or endogeneity bias is to use instrumental variables that affect the endogenous variable but are not directly associated with the outcome of interest (Moffitt, 2005; Winship & Morgan, 1999). Researchers interested in cross-national differences can certainly adopt the same strategy, estimating two-stage least-squares regressions by country, with instrumental variables, and then compare the effects of the endogenous variable between countries. Because finding a convincing instrumental variable, especially one that seems plausible across different national contexts, is rather difficult, an alternative is to use propensity-score matching methods (Gangl, 2010; Rosenbaum & Rubin, 1983). Although propensity-score matching methods are becoming increasingly popular in single-country research assessing causal effects, we rarely see them applied in cross-national research. Comparative family research, however, would benefit greatly from incorporating the methods that researchers in other areas have adopted to address selectivity bias, as cross-national differences in selection processes could easily confound the impacts of social, institutional, or cultural forces on family outcomes. Such incorporation is particularly feasible in cross-national research with a small number of nations, as this research generally relies on side-by-side comparisons of country-specific results. At minimum, to make conclusions about the sources of cross-national differences in family dynamics or consequences, researchers must perform sufficient diagnostic tests to demonstrate that the observed effects for different countries are not biased by endogeneity and that there are indeed statistically significant differences in the effects of interest between countries.

Making Use of Decomposition Analyses

Assuming that researchers could include enough controls in the models to fully account for selectivity bias, the characteristics of people who select into a given family structure are still likely to differ across countries. To better understand the sources of cross-national differences in family disparities, we need to disentangle cross-national variations resulting from compositional differences from those caused by differences in the returns to certain characteristics between different subgroups. One way to achieve this goal is for cross-national family researchers to make more use of decomposition analyses, which have long been utilized in studies of cross-national differences in gender pay gaps (Blau & Kahn, 1996; Chang & England, 2011; Rosenfeld & Kalleberg, 1990). In the latter type of research, a decomposition analysis generally indicates whether cross-national differences in the gender wage gap result from differences in the gender gaps in worker characteristics (e.g., human capital) or differences in the gender gaps in the returns to the same characteristics. In so doing, the analysis enables a more definitive explanation for the cross-national variation.

Although few cross-national studies of family disparities have used full decomposition analyses, a few existing studies have addressed the issue of compositional differences. Most have applied coefficients estimated for one country to the composition of a different country to determine how the predicted discrepancies between countries change (e.g., Geist, 2005; Yu, 2005). Classic decomposition techniques proposed by Kitagawa (1955) and Oaxaca (1973), as well as their variations (e.g., Brusentsev, 2002; Heuveline & Weinshenker, 2008), however, allow researchers to make more precise estimations. Specifically, we can estimate the percentage of the inter-country difference in the outcome of interest resulting from the countries’ different population characteristics, the percentage resulting from the same subgroup's different probabilities of experiencing the outcome between the countries, and each individual characteristic's contribution to the overall cross-national difference. Even when researchers are interested primarily in how a given subgroup fares differently between countries, as in the example of the differential effect of single parenthood on children's educational achievement across countries, or the differential association of premarital cohabitation with marriage dissolution rates, a decomposition analysis can still better address the sources of the differential effect. With sufficient observations, researchers could fit separate models for the subgroups under comparison, whether they be two- versus single-parent families, couples with and without premarital cohabitation experience, immigrant versus native households, or families of different class or racial/ethnic backgrounds. Researchers would then perform a decomposition analysis for each country to distinguish the types of national contexts in which compositional differences between different types of households contribute to more of the gap in the outcome of interest from those in which the differences in the returns to family characteristics are primarily responsible for the relative advantages or disadvantages of a given type of households (for an analogous analysis, see Chang & England [2011] and Rosenfeld & Kalleberg [1990]).

Decomposition methods that have been used to study how inter-country differences in wage structures affect gender wage gaps could also be applied to cross-national research on disparities among different types of families. In the example of single parenthood and children's school performance, we can imagine that the extent to which a country's overall distribution of academic performance is unequal—which is perhaps a result of unmeasured characteristics of the country's school system—will shape the magnitude of difference in educational achievement between children from single- and two-parent families. The same could be said if the research question concerns class or racial/ethnic inequalities in children's outcomes instead. In their international comparisons of gender wage gaps, Blau and Khan (1996) decomposed the difference in the gender pay gap between two countries into four components, which enabled them to separately estimate the extents to which cross-national variations in gender-specific treatment and wage structures contribute to that difference. Their decomposition methods also made it possible to demonstrate how the cross-national differences in the gender pay gap would change if both countries are assumed to have the same wage distribution. Using the same methods, family researchers could address, for instance, whether single-parent or other types of families (e.g., work-class families, racial/ethnic minority households) would maintain their comparatively favorable position in certain countries after adjusting those countries’ distributions in the outcome of interest.

Because a decomposition analysis can inform us about how inter-country differences in returns to given characteristics, as well as how the nation-specific distribution of the outcome of interest, contribute to cross-national variation in household dynamics, it is naturally useful for addressing the question of how social contexts moderate family processes. Some researchers also have used this analysis to help assess the effects of social policies on families. Heuveline and Weinshenker (2008) provide an excellent example of addressing both types of questions with decomposition techniques. Specifically, they estimated the child poverty rate for each of the 14 countries in their sample if they were to have several characteristics that matched those of the United States: the same distribution of earnings among married-couple households, the same associations of different household types with market earnings, the same distribution of children across household types, the same income redistribution through taxes and transfers among married-couple households, and an identical household-type gradient in income redistribution. Whereas adjusting cross-national differences in how household types are associated with market earnings makes it possible to explore national labor markets’ moderating roles, utilizing decomposition techniques to take into account international variation in income redistribution directly answers the question of the effects of taxes and welfare transfers on child poverty rates.

In short, cross-national family research, particularly that concerning family disparities, would benefit from utilizing decomposition analyses more often. Because identifying the actual source of cross-national variation can greatly enhance the credibility of the explanation when direct measures of nation-level influences are unavailable, and because decomposition analyses are quite manageable when the number of countries under comparison is small, researchers comparing family dynamics in relatively few countries should especially consider utilizing decomposition methods.

CROSS-NATIONAL RESEARCH WITH A LARGE SET OF NATIONS

As high-quality data from multi-country collaborations become increasingly available, cross-national family researchers are more often examining a relatively large number of countries simultaneously. By “a relatively large number,” I mean more than 15—in most cases somewhere between 20 and 40. With a few exceptions (e.g., Heuveline & Timberlake, 2004; Liefbroer & Dourleijn, 2006), cross-national research with 15 or more countries departs greatly from research comparing fewer countries in their analytical strategies. Namely, in the former type of research we rarely see separate analyses by country and comparisons across different countries’ models. Instead, such research typically relies on pooled data from all nations and contains measures of nation-level factors that are hypothesized to shape microlevel outcomes. Such research also increasingly employs multilevel models (Dronkers & Härkönen, 2008; Fuwa, 2004; Greenstein & Teachman, 2009; Hook, 2010; Kalmijn, 2013; Knudsen & Waerness, 2008; Soons & Kalmijn, 2009; Treas et al., 2011; Yodanis, 2005; Yodanis & Lauer, 2007), which enables researchers to take into account the fact that individuals from the same country may be under similar unobserved influences (Raudenbush & Bryk, 2002).

Compared to cross-national family research with a small number of countries, the ability to model inter-country variations of studies involving many more countries is a major advantage. Nevertheless, even when researchers are able to gather data from as many as 35 or 40 countries, the number of degrees of freedom at the country level is still very small. Hence it is difficult to include sufficient controls for many country-level differences that may also affect the outcome of interest in the models. Because the number of countries contained in a cross-national analysis is ultimately small, a couple of exceptional countries can strongly influence the results involving country-level predictors. In addition, although measurement comparability is not a challenge encountered only by research with a relatively large number of countries, the use of pooled data and uniform measures across a wide range of countries generally imposes more assumptions than fitting identical models for data from two or three countries. Thus, comparative research with a large country sample faces its own share of problems. In the following, I discuss the problems more fully and suggest ways of handling them.

Problem with Insufficient Nation-Level Controls

Even with a relatively large set of countries, cross-national family research generally does not have a sufficient number of cases at the macro level to control for more than a handful of national characteristics. Therefore, there is always a possibility that another macrolevel variable that is not controlled for explains the observed effect of a given nation-level characteristic on individuals. At minimum, the observed effect is likely to suffer from omitted-variable bias. In addition, family studies focusing on the effects of social policies may face the problem of insufficient nation-level controls, because a country can implement multiple, related policies, the effects of which might cancel each other out (Lewis et al., 2008:25). For instance, among Western industrialized countries, those offering extensive parental leaves often have generous public childcare provisions as well. Whereas the former, according to Pettit and Hook (2009), promote a gendered division of labor within the home, the latter reduce women's domestic responsibilities by directly facilitating their employment. Thus, without simultaneously controlling for all related policies, researchers are likely to have difficulty telling the true effect of a given policy.

With a typical analytical sample of 20 to 30 countries, most existing cross-national family research has barely enough statistical power to include three nation-level predictors simultaneously. Facing this limitation, some researchers opt for the strategy of examining only one nation-level predictor in each model, rather than including all those factors that may confound each other in the same model (e.g., Fuwa, 2004; Treas et al., 2011). As they do so, they are also likely to take the association between each nation-level predictor and outcome as more descriptive than explanatory. Although being modest about the causal claim helps compensate for the insufficient nation-level controls, it is still beneficial to control for even just one or two additional country-level predictors that may account for the association between the national characteristic of concern and the outcome. In the case that there are too many potentially confounding macrolevel factors to be considered, researchers could at least build separate models that include one or two different country-level controls each time. Based on the separate models, they could eliminate controls without statistical significance and, as a final step, fit a reduced model with only those found to be relevant.

An even better way to cope with insufficient nation-level controls is to use nation-year data and fixed-effect models. As several major cross-national surveys now include data from various years for the same country, family researchers increasingly introduce nation-year data in comparative studies. Existing research, however, typically relies on multilevel models and simply utilizes the multiple-year data from the same country as additional data points at the national level (Hook, 2006, 2010). When a sufficient number of countries has been observed more than once, researchers should nevertheless consider using country fixed effect models instead. As country fixed effect model models capture all factors that are unchanged across years within countries, they enable researchers to make a much more convincing case about the macro-level predictor of concern. Because the fixed-effect model approach is more appropriate when the macrolevel predictors of interest do change in most countries under observation, it is most suitable for studying, for example, how the national divorce or cohabitation rate conditions the impacts of divorce or cohabitation on individuals. The approach can be equally useful when the main nation-level predictor is female labor-force participation rate, average marriage age, overall fertility rate, prevalence of single-parent households, or level of gender equality. It can also be employed to study the effect of a certain policy if it is newly introduced in a number of countries, or if the specific provisions of the policy tend to change over time (e.g., tax schemes or the state's transfers to the poor). By taking into account all the other time-constant policies that could potentially offset the effect of the policy of concern, the approach suggested here would be especially helpful in uncovering a policy's true impact.

To my knowledge, no cross-national research on family or other issues has yet utilized the fixed-effects model approach just described, perhaps because of the high data requirement. A recent study with cross-state comparisons within the United States (Brady, Baker, & Finnigan, 2013), however, provides an example of how the fixed-effects model approach can be similarly applied as nation-year data become increasingly available. Even when researchers are restricted to using cross-national data from a single year, it may still be worthwhile to consider country fixed effects models, instead of the typical, multilevel modeling approach. To elaborate this point, let's consider a study that investigates the extent to which the effect of wives’ relative earnings on their share of household work depends on their nation's rate of divorce. With the multilevel modeling approach, a researcher would include in the model the national divorce rate, as a macrolevel predictor, and the cross-level interaction between women's relative earnings and the national divorce rate, along with other macro- and individual-level controls. In that case of using country fixed effects models as an alternative, one would instead specify the model as:

$$Y_{ij}={\beta }_0+{\beta }_{1}R{E}_{ij}+{\beta }_2(R{E}_{ij}\times N{D}_j)+\Sigma {\alpha }_{kij}{X}_{kij}+\Sigma {\delta }_{n-1}{C}_{jn-1}+r_{ij}$$

where Y_ij is the share of household work for woman i in country j (j = 1, 2, 3, ...N); β₀ is the intercept; β₁ is the coefficient for the woman's earnings relative to her spouse (RE); β₂ indicates the extent to which the effect of the relative earnings varies with the national divorce rate (ND); Σα_kij denote the effects of k individual attributes; Σδ_{n − 1} C_{jn − 1} represent fixed effects for the N-1 countries in the data set; and r_ij is the error term.

Compared to the multilevel model described earlier, the fixed effects model just specified lacks information on the main effect of the national divorce rate, or the effects of other national predictors researchers may include in the multilevel model. Nevertheless, as Möhring (2012) demonstrated, the fixed-effects approach is able to produce equally valid results as those from cross-level interactions in multilevel models. More important, country fixed effects models have the advantage of being able to control for all nation-level characteristics that may shape individuals’ outcomes. As discussed earlier, multilevel models used for cross-national family research can rarely include more than three or four nation-level predictors. Even though the country-specific error term (which is constant within but varies across countries) included in multilevel models is supposed to capture unobserved nation-level influences, it also must satisfy the same data assumptions as does the random error in ordinary least squares regressions, including independence, normality, and homoscedasticity. In the case of cross-national studies, because the countries included in the sample are nearly always from a convenient, rather than random, sample, it is highly likely the assumptions for the country-specific random error term are not satisfied. Using country fixed effect models with interactions between nation- and individual-level predictors, in contrast, can avoid this problem and more effectively address unobserved country-level influences.

Given that much cross-national family research use multilevel models is interested primarily in how the national context moderates individual-level associations (or disparities between subgroups that differ in gender, race, family status, or other characteristics) (e.g., Fuwa, 2004; Soons & Kalmijn, 2009; Treas et al., 2011), the fact that the fixed-effects approach described here would miss information on the main effects of nation-level predictors might not be of great concern. In this sense, this approach could be a more rigorous and valuable alternative to multilevel models. At least, researchers should utilize country fixed effects models to test the robustness of findings from multilevel models. After all, the latter models assume a normally distributed sample at the macrolevel, and this assumption is especially problematic when the macrolevel unit is country.

Problem with Influential Countries

In multilevel analyses, the number of degrees of freedom at the macro level is confined by the number of macrolevel units available. When conducting cross-national comparisons with multilevel models, researchers rarely examine more than 40 countries. Like fitting regressions on a sample of 40 respondents, a couple of respondents who happen to have especially high or low levels of predictor X and outcome Y could contribute to a statistically significant association between X and Y for the entire sample. In the case for cross-national research, a few influential countries with, say, exceptionally high prevalence of divorce and children's health problems could lead the former, a macrolevel predictor, to have a positive and significant effect on the latter in a multilevel model, even if the association between national divorce rates and children's health are generally weak in other countries. In a discussion on the issue of influential cases, Van der Meer and colleagues (2010) showed that removing merely 3 out of 53 countries was sufficient to turn a country-level coefficient in a multilevel model from positive to statistically nonsignificant.

Influential cases are always a potential problem for analyses with small samples, and cross-national family research almost never has enough countries in the analysis to be free of this problem. Thus, researchers’ best option is to conduct a thorough diagnostic analysis before deciding which countries to include in the final analysis. Before testing a nation-level characteristic's association with an individual-level outcome in a multilevel model, one's first step should always be to examine scatter plots for national characteristics and the national average of the outcome. This graphic exploration enables a visual detection of countries that are potential outliers. Researchers then can perform an influential case analysis by, for example, testing how the effect of the national characteristics in the multilevel analysis would change with or without the potential outliers. In addition to exploring the possible bias with descriptive graphs, previous research also has demonstrated the utility of more rigorous diagnostic tests— namely, with the calculations of DFBETAS or Cook's D, two indicators of each country's influence on the models’ estimates (Ruiter & De Graaf, 2010; Van der Meer et al., 2010). As specific guidelines have been proposed for the cutoff values for Cook's D and DFBETAS in the context of influential case analyses (Belsley, Kuh, & Welsch, 1980:28), researchers can properly justify their decisions to exclude certain influential countries from the final analysis, or report with confidence that the country-level results are not biased by any influential case, based on the calculated values of Cook's D and/or DFBETAS.

Because family researchers modeling nation-level attributes in an analysis of a pooled multinational sample are always starved for more degrees of freedom at the macro level, they might be reluctant to throw away data from influential countries based on Cook's D or DFBETAS values. Then, at minimum, researchers should ensure that the results of interest would remain and report how each coefficient would change in both significance and magnitude after excluding the influential countries. In the case when data from influential countries are eliminated entirely, it would also be beneficial to consider why such countries are exceptional and what the exclusion means for the generalizability of the findings.

I should note that the procedure of routinely checking and eliminating influential countries could potentially lead societies from less studied areas to be more frequently excluded from cross-national multilevel analyses. Many multi-country surveys select nations on a voluntary basis. More industrialized countries, particularly those in North America and Europe, are more likely to participate in multinational collaborative survey projects. Data sets commonly used for international comparisons, such as the International Social Survey Programme, usually include few countries from Asia, Africa, or Latin America, with those from North America and Europe making up the majority. In such unbalanced country samples, the few nations from outside of North America and Europe may easily be outliers because of major cultural, economic, and political differences across different regions of the world. Because properly controlling for regional differences is unfeasible when there are just one or two countries from certain regions, researchers might just have to apply multilevel models to mostly North American and European cases and accept the limited generalizability. Nevertheless, we could still examine the countries in less-studied areas with the strategies used for comparing a small number of countries. Moreover, instead of multilevel models, we could divide all countries in the sample into clusters based on their similarities in given dimensions and compare the patterns between clusters. A combination of a multilevel analysis for a cluster with relatively many countries (e.g., a cluster for Northern American and Western European countries) with comparisons of results between different clusters may be another way to make a strong and comprehensive case.

Problem with Measurement Incomparability

The problem of measurement nonequivalence has long been considered as a major challenge for cross-national family research (Thomas & Weigert, 1972). Although this challenge is not faced exclusively by cross-national research examining a large number of countries, the fact that such research tends to estimate regressions on data pooled from many different countries, with standardized measures across societies, makes its results, and the subsequent interpretations, more likely to be distorted by measurement nonequivalence. Because cross-national research handling a large set of nations is likely to focus more on the overall pattern than the specific cultural or social conditions of any individual country, its tendency to overlook measurement nonequivalence is also greater than research contrasting two or three countries. In addition, with side-by-side comparisons of models of different countries, researchers examining a small set of countries can detect measurement nonequivalence more easily, through noticing differences in a given measure's effects between countries, than those applying multilevel models to large country samples. Thus, measurement incomparability is a particularly relevant concern for family researchers analyzing a large number of countries.

The case of female-headed households, a type of household that is generally thought to be disadvantaged and worthy of attention (Buvinić & Gupta, 1997), illustrates the difficulty of ensuring measurement comparability in cross-national family research. Although many surveys ask respondents to report information on the head of household, and researchers can easily identify households headed by women based on that information, the precise structures of female-headed households can differ from country to country because of different cultural and social practices. Specifically, in some countries, such as Mexico, because many young mothers are incorporated into their parents’ or other relatives’ households, older widowed women and their adult children make up a large proportion of female-headed households (Villarreal & Shin, 2008). Conversely, in countries like the United States, nonmarried mothers with young children are likely to head their own households; thus such families are rather typical among female-headed households. Furthermore, in Asia, elderly widowed women living with married adult children are far less likely to be considered as the head of their households than their counterparts in Latin America and Africa (Bongaarts & Zimmer, 2001:Table 2). That is to say, the same three-generation family with an elderly widow, her adult son, and his spouse and children could be characterized as a female-headed household in Latin America but a male-headed one in Asia. As a result, the estimated effect of female-headed households in a model using pooled data from countries in both Asia and Latin America, for example, could be problematic.

One way to ensure measurement comparability is to conduct a thorough descriptive analysis to learn about how the predictors, especially the key ones, in the models are associated with other individual and family characteristics in different ways across countries. An exploratory analysis of household members’ age, marital status, and relationships to the head, for example, may show that across countries, female-headed households are in different family life-course stages. Researchers could thus decide to control for the main characteristics that differentiate such households in different countries or restrict the cross-national analysis comparing female- to male-headed households to households of similar structures (e.g., households with no adult children present).

Measurement nonequivalence is especially likely to be a threat to cross-national family research when its predictors or outcomes involve attitudes or values. One example of such research is a comparison of public opinions on family roles and practices across countries. Thomas and Weigert (1972) showed that depending on the language used, evaluative responses toward family practices from the same bilingual population would produce different measures. Individuals from different nations would be even more likely to interpret and respond to the same items differently, regardless of how well the items are translated into the local language. Moreover, cross-national family research studying values and attitudes are highly likely to use constructs created from multiple items. Because the underlying structure for the set of items used for a construct might not be the same in all countries, the problem of construct nonequivalence is common. Blasius and Thiessen (2006), for example, found five distinctive patterns of responding to the same set of items expressing views on single- and dual-earner families among 24 countries. In such a case, using a single construct that assumes the same underlying response structure for all countries in regression models would yield problematic results.

Given the possibility of construct nonequivalence, cross-national family research with constructs in the models would benefit from first applying methods proposed to screen the countries’ response structures, including multiple correspondence analyses and confirmatory factor analyses (Blasius & Thiessen, 2006; Davidov, Schmidt, & Schwartz, 2008). Upon determining nonequivalence of different countries’ response structures, researchers could then reselect items or modify the structure for the construct to achieve equivalence. Alternatively, they could restrict the comparison to a smaller set of countries that show equivalent response patterns. Unfortunately, these solutions are likely to confine research to narrower concepts (which allow less international variation in response structures), limit its ability to generalize arguments to certain other countries, or lead researchers to abandon multilevel modeling strategies altogether (if the number of countries becomes too small). It is, however, possible to compare the countries within each cluster that have equivalent response structures using side-by-side models and investigate whether the same or different national characteristics correspond to the variation within each cluster. Although such an approach may not provide as straightforward results as those from multilevel models including the same constructs for all countries, the more complex picture this approach yields might be closer to social reality, given that the assumption of construct equivalence across a wide range of countries could be unrealistic for many concepts that interest family researchers.

CONCLUSIONS

In this paper, I have reviewed the types of research questions in family studies that cross-national comparisons are well suited to answer, as well as the specific analytical strategies family scholars have utilized to address such questions. I argue that the precise analytical strategies for cross-national family researchers often depend on the number of countries compared in the study. As a result, family studies comparing relatively many and few countries tend to face different methodological challenges. Compared to that with only a few countries, cross-national research examining a large number of countries has the advantage of being able to model and test the effects of nation-level characteristics on the outcomes of interest. At the same time, however, to systematically estimate such effects, researchers must ascertain that no influential countries are driving the results and that the measures at both micro and macro levels are comparable across countries, especially if constructs are used. Although previous research has proposed several tools to detect influential cases and check construct equivalence, so far, cross-national family research has not been routinely applying those screening methods to determine the countries to be included in the analysis. Had the screening been done regularly, we might find that certain countries and certain issues (e.g., those related to values) are generally less suitable for multilevel analyses with pooled country data. A more conventional approach with side-by-side comparisons of model results for different countries or different clusters of countries might be more appropriate for studying cross-national differences regarding certain topics or involving a few “exceptional” countries.

In comparison with cross-national analyses that explicitly model nation-level characteristics and estimate their effects, research using side-by-side comparisons of models for different countries to infer the roles of national characteristics seems to yield rather indirect evidence. Nevertheless, as just discussed, in some cases a side-by-side comparison remains the best strategy for explaining how differences in national contexts affect individual or family outcomes. As I suggest in this paper, there are several ways we can make the conclusions drawn from such side-by-side comparison more credible. I argue that techniques that have long been used in single-country studies to address selectivity bias or compositional effects should also be applied to cross-national family research with a small country sample. The more we can rule out the influences of population (or subpopulation) compositions and the possibility of differential selectivity between countries, and the more supplementary evidence we can provide to support the argued mechanism for the consequences of certain national characteristics, the more likely we can make a convincing case about the impacts of social policies or the roles of broader contexts.

With all the potential comparability problems and necessary checks, as well as the arduous task of handling data from multiple countries, there is little doubt that conducting rigorous, high-quality cross-national research requires much work. Is it worth the hassle? The answer is a resounding yes. As I have briefly discussed in this paper, a cross-national research design is generally necessary for addressing questions regarding the influences of welfare policies, social institutions, and cultural norms on family dynamics. Furthermore, on many issues that interest family scholars, macrolevel factors can and have been shown to play very important roles—sometimes more important than any individual-level predictor we are able to observe. Only by successfully linking the macro to the micro can we gain a comprehensive understanding of family disparities and processes.