New Approaches to Demographic Data Collection

Abstract

As population scientists have expanded the range of topics they study, increasingly considering the interrelationship between population phenomena and social, economic, and health conditions, they have expanded the kinds of data collected and have brought to bear new data collection techniques and procedures, often borrowed from other fields. These new approaches to demographic data collection are the concern of this essay. We consider three main topics: new developments in sampling procedures; new developments in fieldwork procedures; and new developments in the kind of information collected in demographic and social surveys. We conclude with some comments on data sharing in the social research community and a list of major Chinese surveys publicly available to researchers. Where possible we illustrate our points with Chinese examples.

INTRODUCTION

In recent years the field of demography has been expanding rapidly and broadly, so that current demographic research now extends well beyond studies of fertility, mortality, and migration, the traditional concerns of the field. These developments have been of two main kinds.

First, much recent work in many social science and biomedical disciplines (sociology, economics, geography, public health, and medicine) has adopted a “population-based” approach (Duncan 2008:763–772). Sometimes this is as simple as basing analysis on data collected from samples of general populations, for example, studies of the incidence of disease based on probability samples of all residents of geographic areas rather than, say, employees of an enterprise or visitors to a particular clinic. But sometimes demographic concepts are merged with substantive ideas drawn from other disciplines, such as studies of social mobility that take account of fertility differentials by social status (Mare and Maralani 2006).

Second, there has been a great deal of “cross-fertilization” within these fields, resulting in the creation of new sub-specialities. Two good examples are the recent emergence of “bio- demography,” the incorporation of biological data and research questions into demographic studies; and “spatial demography,” the incorporation of spatial data (geo-coded information and satellite-generated measures and images) into population studies.

As is clear from these two examples, an important locus of these developments has been in the expansion of the kind of data collected and in data collection techniques. These developments should be of interest not only to population scientists but to many social scientists who do not think of themselves as demographers. In this essay, we consider three main topics: new developments in sampling procedures; new developments in fieldwork procedures; and new developments in the kind of information collected in demographic surveys. We conclude with some comments on data sharing in the social research community. Although these developments are worldwide, where possible we refer to Chinese examples.

SAMPLE DESIGNS AND SAMPLING PROCEDURES

This section describes several developments in sample designs and sampling procedures, including the move toward true probability samples of general populations; sample design issues; and new sampling procedures, particularly those designed to sample rare or difficult-to-find populations.

Probability Sampling

The conventional design for sampling general populations is multistage area probability sampling, in which a hierarchy of geographical units is identified (say, counties, townships, villages/neighborhoods, and households); units at each level are selected at random but (except at the household level) with the probability of selection proportional to the population size of the unit (PPS); and then individuals are selected to be interviewed. A national sample of 8,000 people might be created by selecting 100 counties, then two townships within each selected county, then two villages or neighborhoods within each selected township, then 20 households within each village or neighborhood. Within each selected household, one individual (or more) would be selected at random to be interviewed. Multistage samples often are more complex than just described because they are “stratified” by features of the geographical units, for example, whether the county is in a coastal, middle, or western province; whether it is predominantly urban or rural; the average level of education of the population; etc.

Probability sampling is not a new method. However, it has increasingly come to be recognized in China that neither convenience samples (for example, visitors to a clinic) nor samples of “typical” places provide an adequate basis for inference to broader populations; see the useful discussion by Duncan (2008:764–767) showing how such samples can result in misleading inferences. While casual samples still are widely used in health research, because of the difficulty of collecting health information from general populations, there is increasing emphasis even among medical and public health researchers in securing population-based samples because of known biases in hospital- or clinic-based samples. Two studies of the effect of prenatal malnutrition caused by the Great Leap Forward famine on the prevalence of young- adult schizophrenia provide a useful contrast. St. Clair et al. (2005) first suggested a link between the famine and schizophrenia, combining hospital records and historical population data for a single county. But Song, Wang, and Hu (2009), in motivating their study based on data from the 1987 Chinese National Disability Sample Survey, a very large national probability sample survey, noted that hospital-based studies in China are vulnerable to the selectivity of hospital admissions, which favors urban high status people. Because of the selectivity of hospital admissions, generalizations to the general population based on hospital records inevitably will be biased. Indeed, Song and his colleagues found sharp urban-rural differences in the linkage between exposure to the famine and schizophrenia that were entirely missed by St. Clair and his colleagues.

Even when Chinese surveys purport to be based on general population samples, all too often the samples are chosen on a casual basis. Even major studies do this. For example, the well-known China Health and Nutrition Study (CHNS),¹ a joint project of the Carolina Population Center and the Chinese Center for Disease Control and Prevention, which has followed the same people since 1989, is based on semi-probability samples of nine provinces concentrated in Eastern and Central China (with some primary sampling units (PSUs) chosen at random but others chosen purposively, and small areas within the PSUs chosen randomly). Thus, although results from this study are often presented as pertaining to China as a whole, they hardly do so, since, with the exception of Guizhou Province, the entire Western Region of China, where health and nutrition problems are most severe,² is omitted from the sample.

Fortunately, many studies conducted by the Ministry of Health and the Family Planning Commission do sample the entire population of China; but many of these studies are not readily available for use by the research community, a point we will return to. True national samples are less common in university-based studies, but they are becoming increasingly common. These are described in detail, with access information, in the Appendix.

Design Issues

The increasing opportunity for Chinese social scientists to carry out sample surveys lends added importance to the need for good sample design. Unfortunately, many Chinese surveys exhibit design flaws that make them less valuable than they could be. Thus, here we offer some points for consideration when designing new sample surveys.

Comparisons are the essence of social research

Unfortunately, researchers interested in a particular population often make the mistake of collecting data only from members of that population. The problem is that you cannot study a constant. If you want to know how the urban population compares to the rural population, you need a sample of both urban and rural people. Similarly, if you want to study, say, the determinants and consequences of internal migration, you need a sample of both migrants and non-migrants. This is an extremely simple point, but one that often is ignored in data collection efforts. For example, in 2002–2003 the National Bureau of Statistics carried out the National Survey of Temporary Migrant Children in China, a 9-city survey of 6,343 migrant households with children age 0–18. The restriction of the sample to the children of temporary migrants makes it impossible to assess the effect of migration on children’s lives. Of course, a description of migrant children is possible, but without a comparison to non-migrant children it is difficult to interpret the meaning of the descriptive statistics. From an analytic perspective, all that can be done is to make internal comparisons, e.g., comparing the conditions and experiences of migrant children residing in different cities (Zou, Qu, and Zhang 2005; Lu 2007). But, presumably, the question that motivated the data collection was how migration affects children. Given the design, this question cannot be answered.

Partial samples are vulnerable to sample selection bias

There is an additional problem with partial samples, such as samples of migrants or samples of the urban population only—even descriptive statistics may be biased due to sample selection. Consider the claim that in urban China intergenerational social mobility is particularly pronounced relative to what is observed in other nations (Whyte and Parish 1984). The difficulty with such a claim is that in China rural-to-urban migration is often a consequence of upward mobility, the outcome of a process in which the brightest and most ambitious of those from peasant origins do well and go far in school and thus qualify for urban jobs. Thus, de jure urban- only samples tend to be comprised both of those from urban origins, who experience typical patterns of intergenerational mobility, and formal rural-to-urban migrants (that is, those from rural origins who have not only moved to urban areas but also have acquired urban registration (hukou), who have experienced extreme upward mobility. Computations of mobility rates that fail to distinguish between these two groups will overstate the true rate of upward mobility in the Chinese population as a whole. See Wu and Treiman (2007:416–417,440–443) for a more detailed discussion of this issue.

NBS rural and urban surveys do not jointly cover all of China

Apart from the sample selection bias inherent in analyzing samples selected on the basis of the dependent variable, there is an additional problem with conducting separate surveys of the urban and rural sectors, which is the conventional practice of the National Bureau of Statistics (NBS). The NBS has two separate data collection units, one for rural China and the other for urban China. In both cases, samples are drawn from the lowest administrative unit, “village committees” and “neighborhood committees” respectively. However, not all of China is covered by village committees and neighborhood committees. In particular, commercial buildings, factories, hospitals, universities, and construction sites, as well as roadways and parks, are excluded, yet people live in these places. Thus, sampling only within the boundaries of villages or neighborhoods will exclude a portion of the population. This means that even when it is possible to combine NBS rural and urban surveys (which is seldom, because in general common questionnaires are not used), the combination will not yield complete coverage of the population of China. Moreover, a 2003–2004 pilot study for a national survey of migration and health encountered situations in urban fringe areas in which long-time residents were regarded as under the purview of the village administration but newly-arrived migrants were not (Treiman et al. 2006). In these areas, a part of the de facto resident population would be excluded from the sampling frame by definition.

So what to do? There are two possibilities. First, villages and neighborhoods could be sampled in the usual way, but in addition portions of a township not covered by either villages or neighborhood committees could be sampled. This would require careful estimation of the fraction of the population of a township living in such places to ensure that they are sampled in proportion to that fraction. However, in urban China there is no longer a consistently defined unit below the township (jiedao): “neighborhood committees” have in some instances been replaced by larger “community committees;” high rise buildings often have their own “property owners’ committees;” and so on. Thus, given that townships are too large to be enumerated (in 2000, they averaged about 25,000 in population size), a way of sub-sampling small areas (“enumeration districts” (EDs)) within townships would need to be devised. One way to do this would be to sample administrative units down to the township level and then, as an additional stage, to sample smaller areal units within each township, using a version of the spatial sampling procedures described below. For an example applying such a procedure, see the documentation for the 2008 IMHC survey.

Many surveys include only the easy-to-reach population

Until fairly recently, the conventional way of carrying out general population surveys in China has been to sample from the local population register (hukou) and, sometimes, from lists of temporary residents. This is no longer an adequate sampling strategy, at least in urban areas, because there is a great deal of residential mobility in contemporary China and many people fail to change their registration when they move. Computations from the IMHC survey show that 39 per cent of those living in cities changed their residence in the past five years and that, of these, 61 per cent moved within the same city the last time they moved. Moreover, 82 per cent of the within-city movers lack local hukou, presumably because their hukou is still held by their old neighborhood committee. Informal rural-to-urban migrants also commonly lack local registration. Fully 75 per cent of peasants (those with rural hukou) who reside in cities lack local hukou. These results are consistent with the findings of Landry and Shen (2005), who found, in a 2002 survey in Beijing, that 45 per cent of their respondents would have been missed had the survey team relied on registration lists. Because of the growing inadequacies of hukou lists, adequate sampling of the Chinese population now requires household enumeration procedures. We will discuss these in the section on Fieldwork Procedures.

Apart from reliance on registration lists, another shortcoming of many Chinese surveys is their propensity to conflate families and households. Adequate coverage of the population requires that all members of a dwelling unit or household be enumerated.³ The exclusion of roomers, servants, and employees from those eligible to be interviewed means that the sample does not adequately cover the population. Sometimes this is done on the ground that such people are “temporary” members of their households and will be counted in their permanent households. But sample surveys almost never interview members who are living elsewhere, even when they are recorded in household rosters. Because of this, the migrant population is often under- counted. Migrants also tend to be undercounted in panel surveys because few surveys attempt to interview those who have moved away between waves. But this, too, results in inadequate coverage of the population, especially in contemporary China where more than 40 per cent of the population live other than where they are registered and 21 per cent of the population has moved within the last three years, the typical length of time between waves of panel studies (both computations are from the IMHC).

A final way in which surveys fail to cover the entire population is a propensity to overlook people living in non-standard dwellings. We will discuss this point in the section on Fieldwork Procedures.

Clustering and stratification

Good sample designs strike a balance between coverage and cost. As already has been noted, general population surveys often are designed as multistage probability samples, with cases clustered within particular small geographic areas. The primary reason for clustering cases is to reduce costs—it is much less expensive to conduct interviews in several households within a limited geographical area than to send interviews to widely scattered households. However, the typical consequence of clustering is to inflate standard errors.⁴

Sometimes surveys are excessively clustered, so that the standard errors become very large; or, to state the same point differently, the standard errors become similar to those expected from a much smaller simple random sample. A case in point is a recent cardiovascular study said to be based on a “nationally representative sample of 15,540 Chinese adults aged 35–74 years in 2000–01” (Gu et al. 2005:1398–1399). Upon close inspection of the sampling description, it turns out to be based on 20 sampling points, a rural township and an urban “street committee” selected in each of 10 provinces via multistage sampling procedures that did not take account of differential population size. Even ignoring the fact that the provinces “were selected to be representative of the geographic and economic characteristics in their regions,” that is, were selected purposively, this sample was so highly clustered that it almost certainly had the precision of a much smaller sample.⁵

Also as previously noted, multistage samples often are stratified, by region, type of place, the socioeconomic level of the community, and so on. Stratification is used both to ensure that there is adequate coverage of all segments of the population, sometimes sampling segments at different rates (e.g., over-sampling migrants in order to have enough migrants to be able to compare them with non-migrants), and to reduce the deleterious effect of clustering, since stratification generally reduces standard errors. Indeed, it sometimes is possible to dramatically reduce the size of design effects by stratifying on variables strongly correlated with the main analytic variables (for a Chinese example, see Treiman et al. 1996; see also discussion of the same study in Treiman 2009:207–212).

Household vs. person samples

Many social processes of interest to population scientists pertain to the behaviors and decisions of families and households rather than or addition to those of individuals. For example, the decision to migrate is often seen as the outcome of family risk-diversification strategies—some grown children are kept home to work on the family farm while others go out for wage work, thus providing two alternative ways to protect against downside risk (Stark 1992; Roberts 1997; Massey et al. 1998). Surveys that take individuals as the sampling units may not be able to capture family structures and dynamics that affect outcomes. One common solution is to collect extensive information about other family and household members; but usually it is necessary to restrict the information collected to socio-demographic items since other family members cannot be expected to be reliable reporters regarding attitudes, values, preferences, and the reasons for making particular decisions, nor regarding retrospective information such as work, residential, medical, and other histories.

For this reason, surveys increasingly are designed to interview all members of each sampled household. The CHNS uses such a design, as do the Indonesian Family Life Survey (IFLS) and the Mexican Family Life Survey (MxFLS). Although relatively expensive since the interviewer must spend many hours with various members of a household (but not nearly as expensive as samples of the same size where one adult per household is interviewed), such surveys yield rich information on the characteristics and beliefs of each member of the household and also permit direct biometric, anthropometric, and cognitive assessments (see the discussion below of developments regarding the kind of information collected in sample surveys). An additional advantage is that such designs permit the estimation of household fixed-effect models, which provide a useful way of purging analysis of the effect of all unmeasured variables that are constant across members of a household (see Treiman (2009:Ch. 15) for a brief introduction to such models and Allison (2005) and Wooldridge (2006:Chs. 13 and 14) for more extended explications). Finally, household samples do not suffer from a bias inherent in the most common household interview design, in which households are chosen at random within small areas and one eligible person (say, one adult) per household is chosen at random to be interviewed. The problem with such designs is that in large households each individual has a smaller chance of being selected than do individuals in small households. The usual way this bias is corrected is by weighting the data for each person interviewed by the number of eligible respondents in the household. While this is a reasonable approach, it is not optimal since weighted data have larger standard errors than non-weighted data.⁶

The use of household samples becomes problematic when family members live apart, as when some members have “gone out” for work, or when individuals live in large collective households with non-family members, as is true of many migrants in China. In such cases, it may be preferable to sample individuals and then also to sample selected relatives, e.g., siblings, spouses living apart, parents, etc. Doing this will, of course, increase the expense of carrying out surveys, since it sometimes will be necessary to send interviewers to other locales to interview the selected relatives; but the by-now near universal use of mobile phones makes phone surveys with absent family members increasingly feasible. Interviewing absent family members one way or another is a useful way to gain the advantages of household samples in situations in which many families have members living in more than one dwelling unit.

New Sampling Methods

In this section we consider some relatively new sampling strategies designed for situations in which conventional probability samples of households do not work well. These include spatial sampling, contingent quota sampling, and respondent-driven sampling.

Spatial sampling

When an adequate sampling frame is not available, a useful alternative is to sample spatial units identified by “global positioning system” (GPS) coordinates, a method first used in studies of the environment—ocean temperatures, vegetation types, air pollution, etc. In such applications, the area to be studied is subdivided into a set of regular polygons, which are then selected at random. In studying human populations, however, investigators often want to take account of such factors as population density, in order to represent populations of people rather than populations of space, and also other aspects of population heterogeneity. These considerations add complexity to the sampling process, requiring measurements of density and heterogeneity and their incorporation into sampling designs. An interesting application is a respiratory health and demographic survey of households carried out in the Delhi, India, metropolitan area in 2004 (Kumar 2007).

In a different sort of application, Landry and Shen (2005) used spatial sampling as a way of overcoming the limitations of hukou lists in contemporary Beijing, especially in the coverage of migrants. Their strategy was to overlay a map of Beijing with a grid showing degrees of latitude and longitude and to randomly sample, but with probability proportional to a separately derived estimate of population size, 50 “square minutes” (trapazoidal areas of approximately 1.4 by 1.8 kilometers at Beijing’s latitude) as their PSUs and then, within each PSU, to randomly sample four “square seconds” (areas of approximately 54 meters by 54 meters) as “secondary sampling units” (SSUs). In each of the selected SSUs doors were counted to create a list of addresses from which to select households for interviewing. This strategy has clear advantages with respect to the completeness of coverage. Landry and Shen compared their results to data obtained from the Beijing Area Survey, which had sampled the same districts in Beijing, and showed that, as noted earlier, 45 per cent of the respondents to their spatial survey would have been missed by the Beijing Area Survey. See Gibson and McKenzie (2007:222–225) for additional examples of the use of spatial data as a sampling tool.

But the method also has some disadvantages, of which two are substantial. First, the PSUs and SSUs do not conform to any administrative boundaries, making it difficult to utilize administrative data as a supplement to the data collected from respondents. Second, the SSUs selected often result in buildings being divided, partly in and partly out of an SSU, which requires awkward adjustments. A third limitation, reliance on a simple count of doors, can be overcome by changing enumeration procedures, a point we will return to below. Still, it is a method worth considering, as an alternative or supplement to more conventional ways of implementing multistage sample designs.

Contingent quota-sampling

Quota sampling is a way of obtaining sufficient numbers of respondents within sample strata to achieve a specified level of statistical power. It is similar to stratified probability sampling in that a specific number of subjects from different subgroups are recruited. The difference is that subjects are recruited by convenience rather than randomly. The quotas are pre-determined (i.e., according to sex and age groups) and interviewers are instructed to fill their quotas as best they can for the randomly selected household. Once the quota for a subgroup is met, the interviewers no longer recruit subjects for that subgroup. More detailed discussions of the method can be found in Moser and Kalton (1971) and Kalton (1983).

While the non-random element is a major weakness of this approach and quota samples are essentially convenience samples, several studies suggest that quota sampling often represents a fair balance between technical rigor and the production of useful data (Moser and Kalton 1971; Cumming 1990; Owen, McNeil, and Callum 1998). Although it is not as rigorous as probability sampling, it is often chosen as the most cost-effective means of sampling hard-to-reach or hidden populations. One experimental study explicitly evaluates the performance of quota sampling vs. probability sampling and finds that in many situations quota sampling can be considered an acceptable alternative to probability sampling when the latter is impossible or difficult to accomplish (Cumming 1990). Quota sampling is especially useful when a sampling frame is not available. Another advantage of quota sampling is that researchers can be specific about the type of subjects desired for the study and can be assured that specific subgroups are represented adequately in the final study sample.

In addition to non-randomness, quota samples are vulnerable to bias resulting from giving interviewers excessive latitude in the way they meet the defined quotas. For this reason, a modification, “contingent quota sampling,” often is employed, in which a small geographic area is specified (for example, the “square seconds” used by Landry and Shen 2005) and interviewers are instructed to secure interviews with specified numbers of people with particular characteristics, e.g., men under age 40, men age 40 and over, etc., in each small area. The implicit assumption of this procedure is that, within small areas, there is no correlation between inclusion in the quota and other attributes of respondents. This assumption certainly is more reasonable within small geographic areas than over wider, and presumably more heterogeneous, sampling points, but may nonetheless be problematic. This procedure was utilized for part of the IMHC and in several other recent surveys for which the data collection was done by the Beijing- based survey research firm, Millward Brown ACSR. Specifically, if an interviewer was unable to gain entrance to a high rise apartment building (an increasingly common problem in Chinese cities), s/he was instructed to stand outside the building and approach residents as they entered the building, alternating males and females. The assumption of this approach is that those who agreed to interviews would be similar to members of the randomly selected households in the same buildings that interviewers were supposed to visit.

Respondent-driven sampling (a modern approach to “snowball” sampling)

For certain research topics it is necessary to collect data on difficult-to-find populations, such as prostitutes, HIV-positive individuals, jazz musicians, and so on. Such populations cannot be sampled in conventional ways since usually there is no complete list of the population to be sampled nor are targeted individuals sufficiently prevalent in the general population, even within small geographic areas, to make area probability sampling feasible. Moreover, when special sites are targeted, they tend to produce biased coverage because the kinds of individuals found at such sites tend to be a selected subset of all eligible individuals. (As a simple illustration, it is evident that, in the U.S., the kinds of Blacks who live in Black neighborhoods differ in substantial ways from the kinds of Blacks who live in non-Black neighborhoods.) Moreover, when the criterion for inclusion in the sample involves stigmatized behavior, such as prostitution or drug use, individuals may be reluctant to be interviewed.

Until recently, the only available sampling method was “snowball” (also called chain- referral or network-based) sampling. In this method, a few individuals in the group to be sampled, say, prostitutes, are identified and interviewed. In the course of the interview, they are asked to identify other prostitutes, who are in turn interviewed and also are asked to identify still other prostitutes. In this way the sample is expanded, like a snowball rolling downhill, until the target size is reached.

The obvious limitation of this method is that the “sample” is not a probability sample of the targeted population. “Sociometric stars,” that is, those who are central to networks, are much more likely to be named, and they may well differ in their characteristics from more isolated individuals. Moreover, because of the tendency for people to be more likely to interact with others similar to themselves, the choice of initial contacts is likely to affect the distribution of characteristics in the realized sample. Thus, valid generalization from the sample to the population of interest is usually impossible (Erickson 1979).

However, in 1997 Douglas Heckathorn, a sociologist at Cornell University, proposed an important improvement, a method he called “respondent-driven sampling” (RDS). Heckathorn combines snowball sampling with a set of procedures and a statistical model that generates weights for individuals in the derived sample that correct for the fact that some individuals are more likely to be included than others. Initial contacts (usually chosen purposively) are interviewed and are then asked to nominate others, customarily by distributing coupons which are returned to the investigators by those willing to be interviewed. A crucial aspect of the procedure is to collect information from nominators on the number of eligible individuals in their networks, to be able to estimate population coverage. In a series of papers (Heckathorn 2007; Volz and Heckathorn 2008, and the papers cited there), Heckathorn and his colleagues have shown that, when certain (strong) assumptions are met, his method yields asymptotically unbiased population estimates when conventional statistical inference procedures are applied; see also Heckathorn’s web site: http://www.respondentdrivensampling.org/.

There now has been considerable work evaluating RDS procedures, and in particular the assumptions required for correct statistical inference; see, in particular, Gile and Handcock (2010). Much of this work has focused on the impact of non-random selection of initial respondents, and the effect of two properties of the resulting networks in overcoming bias introduced by initial non-random selection: the degree of subgroup segmentation within the target population (that is, the extent to which subsequent nominations approximate random selection), and the number of links in the referral chain (the more the better).

Because it meets a strongly felt need—to be able to survey hard-to-locate populations—RDS has been widely adopted; Malekinejad et al. (2008) review more than 100 RDS-based studies in more than 30 nations. Merli’s research on prostitutes in Shanghai is a nice application of the method. She and her colleagues interviewed 522 female sex workers (FSW) in Shanghai in 2007 using RDS procedures (Merli et al. 2010), which enabled them to reach FSWs who were not employed in specific venues such as karaoke bars, hotels, and saunas. However, her data did not meet the assumptions specified by Heckathorn for valid statistical inference. This is a common outcome and suggests that RDS must be used cautiously. For additional discussion of the implementation of respondent-driven sampling designs, see Johnston et al. (2008).

Screening

Under certain circumstances it may be possible to screen for members of rare populations in the course of carrying out surveys for other purposes. If for example, a very large general population survey is mounted, as is done fairly frequently by the Ministry of Health and the Family Planning Commission, it would be possible to insert questions asking whether people have certain rare conditions or diseases. If the answer is affirmative, contact information would be solicited so that people with the disease or condition can be followed up with a specialized questionnaire. For example, the U.S. National Survey of Family Growth (NSFG) was based on a sample of reproductive age women previously interviewed in the U.S. National Health Interview Survey (NHIS) (National Center for Health Statistics 1987). Also, Harris et al. (1984) located victims of serious accidents by adding a screening question to a large number of omnibus surveys until a sufficient number was obtained. Of course the adequacy of this strategy turns on the completeness of the original sample. If the initial sample is truly representative of the population, then individuals with the rare condition also will be representative of the population of such individuals. It also turns on the ability to elicit honest answers from potential respondents. Questions about whether individuals are HIV positive or have had homosexual sex or extramarital affairs simply inserted into questionnaires concerned mainly with other topics are not likely to elicit candid responses.

Other methods for sampling hard-to-research populations

Several other methods for sampling hard-to-reach populations have been utilized by researchers. These include targeted sampling, time-and-place sampling, door-to-door screening, capture- recapture methods, and adaptive sampling. Space limitations preclude discussing them here; for a review, see the Chinese-language version of this paper or the version available as a Population Working Paper (PWP), both mentioned in the acknowledgment footnote.

FIELDWORK PROCEDURES

In this section we discuss new approaches to implementing sample surveys, including improved enumeration procedures; the use of incentives to increase response rates; methods for converting refusals; procedures for handling sensitive questions in the absence of privacy; methods for keeping track of respondents between waves in panel studies; and computer-assisted personal interviewing (CAPI) methods for recording responses.

Improved Enumeration Procedures

As noted above, in many nations, and increasingly in China, the conventional way to select households or individuals to be interviewed in sample surveys is first to select small areas, usually in a multistage process and usually with probability proportional to size. Then, within each small area, all households or all individuals are enumerated and a list is created from which households or individuals are selected at random to be interviewed. There are basically three ways to carry out the enumeration task (see also Treiman et al. 2006:84–95).

The simplest and least expensive option is to simply move through the area, listing each address. This method works well in residential neighborhoods consisting entirely of single family houses, such as are common in the U. S., although even in such neighborhoods it is likely that “mother-in-law” apartments, occupied guest houses, etc., will be missed. It also works well in high rise apartment areas, provided the enumerator is able either to gain entry to the building or to obtain information on the number of units in each building from the management company or from building guards. But, of course, what is generated is a list of households, which then requires a screening step with an informant in the selected households to determine which individual household members are to be interviewed.

A better, but infrequently used, approach is to develop a list of addresses together with information on the composition of each household—minimally, the number of eligible residents, e.g., adults. This can be done by engaging as an informant a neighborhood or village committee official who accompanies the interviewer, helping to locate addresses and approaching householders when necessary to obtain information on household composition. The advantage of a local informant is that non-standard dwelling places (sleeping spaces in shops, lofts over workshops, etc.) and “doors-behind-doors” (extra rooms in courtyards, etc.) are less likely to be missed. A second advantage is that a random sample of people, rather than households, can be drawn from the enumeration list, even though the exact individuals to be interviewed are not known and must be determined at the dwelling place by listing all eligible household members and randomly selecting the required number using a Kish table or similar procedure. A disadvantage of this method is that often it is not possible to accurately determine the size or composition of households simply by using informants.

The “gold standard” is to conduct a mini-census, visiting each household, determining who lives there, and recording for each resident the age, sex, and, where necessary, other identifying characteristics. For example, if a household consists of the employees of a small workshop, there may be several individuals of the same age and sex, in which case they need to be distinguished, e.g., the “tall girl from Sichuan,” etc. There are several advantages to the mini-census approach: (a) it makes it possible to draw a genuinely random sample of residents of the area, without the clustering that is inherent in sampling first households and then individuals within households; (b) the sample can be drawn by the supervisor, making it less possible for interviewers to cheat by substituting immediately available household members for those requiring a return visit; (c) and it improves the likelihood of complete coverage because household members can act as informants not only about their own households but about the presence of sub-households (“doors behind doors”) that the interviewer may have missed. But this method is seldom used because it is very expensive.

No matter which option is used, a map of the area needs to be created with addresses listed on the enumeration sheet also identified on the map (Treiman et al. 2006). Without such a map, it often is extremely difficult for interviewers to find the households selected for the survey. This is particularly true in crowded neighborhoods in which apartment units are subdivided and extra rooms are built in courtyards, on rooftops, and in storage areas to rent out to migrants; such separate dwelling units are difficult to identify as such and often lack separate addresses. A useful supplement to a map, which is now technically feasible, but which could arouse suspicion in the Chinese context, is to identify each address by geo-coordinates and a photograph (although the accuracy of GPS receivers needs to be kept in mind).

The Use of Incentives to Improve Response Rates

In principle, the validity of inferences drawn from sample surveys depends on having high response rates (see three special issues of journals devoted to the problem of nonresponse to sample surveys: in 1999, vol. 15, no. 2, of the Journal of Official Statistics, edited by Edith de Leeuw; in 2006, vol. 169, no. 3, of the Journal of the Royal Statistical Society: Series A (Statistics and Society), edited by Peter Lynn; and in 2006, vol. 70, no. 5 of the Public Opinion Quarterly, edited by Eleanor Singer; see also Groves and Couper (1998)). When some individuals in a probability sample cannot be interviewed (because they cannot be contacted or refuse to participate), the danger arises that statistics computed on the portion of the sample for which data are available are biased. This occurs if those not interviewed differ systematically from those who are interviewed with respect to the variables under consideration. Although substantial nonresponse does not always result in substantial bias, it often does (Groves 2006). Thus, the best protection is to secure interviews from as high a fraction of targeted individuals as possible. Unfortunately, response rates have been declining in recent years, both in China and elsewhere, mainly because of an increase in refusals (de Leeuw and de Heer 2002).⁷ In China, a combination of increased personal freedom, possibly rising levels of mistrust, rapid urbanization (around the world response rates are lower in urban areas), and changes in the housing environment (an increasingly large fraction of the urban population lives in restricted-access buildings), has resulted in a sharp reduction in response rates. Indeed, the only Chinese surveys able to secure high response rates these days are “official” surveys that people feel uncomfortable refusing; in the U.S., as well, government-sponsored surveys tend to have higher response rates than academic surveys, which tend to have higher response rates than commercial surveys (Groves and Couper 1998).

Because response rates have been declining throughout the world, there has been a fair amount of research on the effect of incentives on response rates. Singer and her colleagues conducted a meta-analysis of 39 U.S. studies using face-to-face interviewing in which the effect of incentives was experimentally assessed (Singer et al. 1999; see also Singer 2002). Their main conclusions were that incentives improve response rates, especially in studies with low response rates (where presumably other motivations for participation are absent); the larger the incentive, the larger the increase in response rates; both monetary and gift incentives help, but monetary incentives are more effective even when the value of the different types of incentives is equated; incentives matter more when the survey is burdensome because the questionnaire is long; and prepaid incentives appear to be more effective than promised rewards for completed interviews, although the limited number of comparisons makes this result less than definitive.

The use of lotteries as incentives is favored by some investigators. However, the evidence on the efficacy of lotteries, all based on mail or self-administered surveys, is mixed (Singer 2002:6). Moreover, there is some question as to whether lotteries would be effective in contemporary China because of the apparent low level of trust.

Methods for Converting Non-responses

It long has been understood that repeated “call-backs” are effective in reducing non-contact non- responses (Goyder 1985). Of course, remaining in an area for several days or even weeks in an attempt to eventually find someone home can add substantially to the cost of surveys, which is why survey houses, especially commercial survey houses, often are reluctant to do repeated call- backs.

Refusals are more difficult to “convert” than are non-contacts because the latter is mainly a matter of returning enough times to finally find someone at home. Three strategies are used to convert those who initially refuse: offering incentives, or increasing the value of the incentive if incentives were initially offered; elaborating on the value of the study, often by sending a different interviewer; and employing especially skilled interviewers. Sometimes, because of the costs involved, non-respondent conversion is attempted only for a subsample of initial refusers, and the results are combined using the principles of “double sampling” (Hansen, Hurwitz, and Maddow [1953] 1993; Levy and Lemeshow 2008). Starting with its 2004 survey, the U.S. General Social Survey (GSS) has been using double-sampling methods to convert non-responses, typically sampling half of the “temporary non-respondents” as of some cutoff date, usually 10 weeks after the start of interviewing (Davis, Smith, and Marsden 2009:2200). For another example of the use of double-sampling methods to improve coverage, see Jenkins et al. (2004).

An interesting example of the use of skilled interviewers to convert non-responses is the second (2005) wave of the MxFLS, conducted three years after the first wave of data collection. By 2005 about 850 of the approximately 35,000 initial respondents had emigrated to the U.S. When other family members were asked for the U.S. addresses of these migrants, many refused, probably because in many cases the migrants were in the U.S. illegally. However, when the study team sent an especially trained and highly skilled interviewer to revisit the households, he was able to obtain U.S. cell phone numbers and addresses for almost all of the emigrants; then another highly skilled female interviewer successfully contacted and interviewed a large fraction of this sample (personal communication from Elizabeth Frankenberg).

Handling Sensitive Questions in the Absence of Privacy

In many Chinese households it is difficult to conduct interviews in private. Often space is limited and even when it is not it is common and culturally acceptable for family members and also neighbors to gather around to watch the entertainment afforded by the interview. In such circumstances, candid and accurate responses to sensitive questions, such as extra-marital affairs, drug use, homosexual activity, or even whether people have had various diseases, whether they are happy, depressed, or have particular attitudes or opinions, are unlikely. People may refuse to answer the question or may give socially desirable responses. Because of this, procedures have been devised to elicit accurate responses in the absence of privacy and accurate estimates of extremely socially-undesirable behaviors such as wife-beating, criminal activity, or sexual abuse. These are discussed in the extended Chinese-language and PWP versions of this essay.

Panel Maintenance: Keeping Track of Respondents Between Waves

In panel surveys, where the same respondents are periodically re-interviewed (often approximately every three years, but sometimes much more frequently), it is crucial to minimize sample attrition because the ability to compare responses over time is limited to cases for which there are interviews at each time point in the comparison. Obviously, if a survey has an excellent response rate in the first wave of interviewing, say, 80 per cent, but then loses half of the wave 1 respondents through failure to keep track of those who have moved or to retain the motivation to participate of those who have not, the effective response rate for wave 2 is 40 per cent.

For this reason, high quality panel surveys devote considerable effort to “panel maintenance,” which includes periodically re-contacting respondents between waves, often with a small gift and a brief phone call or letter reminding them of the importance of the survey, and tracing those who have moved between waves. In order to do this, it is necessary to collect extensive information from each respondent at wave 1, including names, addresses, land-line and cell phone numbers, and also, crucially, the names, addresses, and phone numbers of relatives, friends, and neighbors who are likely to know the whereabouts of respondents if they have moved. Doing this in such a way as to avoid refusals and the provision of deliberately incorrect information by respondents requires considerable interviewer skill. For a useful discussion of the panel maintenance procedures used in the British Household Panel Survey (BHPS), see Laurie, Smith, and Scott (1999).

Computer-Assisted Personal Interview (CAPI) Procedures

With the increasingly low-cost availability of portable computing devices, it has become possible to carry out computer-assisted personal interviews in field settings. This is done by programming a portable computing device with the survey protocol. The advantages and disadvantages of this and allied procedures are discussed in the extended Chinese-language and PWP versions of this paper.

INFORMATION COLLECTED

There have been important new developments in the kind of data collected by social scientists concerned with the description and analysis of social structures and processes affecting general populations. Three areas have seen major expansion in recent years: the integration of data on individuals or families with data on characteristics of their social contexts (neighborhoods and communities but also classrooms, schools, hospitals, and other institutions); the integration of survey data with geographic information obtained from satellite-generated data and images; and the addition of biological data to population surveys.

Contextual Variables and Multilevel Designs

It often has been hypothesized that characteristics of communities affect individual and household outcomes and behaviors (e.g., Brooks-Gunn et al. 1993; Sampson, Morenoff, and Earls 1999). One strategy for studying such possibilities using data from sample surveys is to construct summary indicators of community characteristics by averaging data from all respondents residing in each community. However, this strategy is sub-optimal both because not all community characteristics are aggregates of individual characteristics and, even for those that can be reasonably represented as aggregates, auto-correlation difficulties are likely to arise when both individual-level and aggregate-level variables derived from the same information are included in the same model. For this reason, an increasing number of surveys of households and individuals also include a “community characteristics” component, often obtained by interviewing community officials and staff representatives from schools, health, and other community facilities; compiling records; and using the interviewer (or respondent) as an informant to report on the type and condition of housing, ease of access to facilities, e.g., the location of the nearest school, hospital, library, bank, etc.

The usual goal is to gather information that captures community infrastructures and levels of socioeconomic development. The measures commonly collected include community educational, health, and sanitation facilities, means of transportation, availability of electricity, and characteristics of the local economy. Most often the focus is on community characteristics at the time of the survey, but sometimes historical information is collected as well. From an analytic standpoint, data about the nature of the community in the past are extremely useful since population researchers often want to know about the nature of the social environment at the time respondents were growing up. Thus, for example, in a study of adult outcomes it would be much more useful to know whether a lower middle school existed in the respondent’s community when s/he was age 14 than whether a lower middle school existed in the community of residence at the time of the survey.

Good examples of sample surveys that systematically collect community characteristics via separate “community questionnaires” include CHNS, IFLS, and the Living Standards Measurement Study (LSMS), a collection of national and sub-national surveys conducted in over 30 developing nations. For good introductions to statistical procedures for simultaneously handling individual and community level data, see DiPrete and Forristal (1994) and Mason (2001).

Spatial Data

The increasing, and now widespread, availability of two types of geographic information has made it possible to incorporate spatial measurements of various kinds into population studies. The first are measurements of latitude and longitude via the “global positioning system” (GPS). The second are satellite-based environmental measurements and images of the earth’s surface, which can be tagged with geo-coordinates.

Geo-coordinates

The GPS is a satellite-based navigation system made up of a network of 24 satellites placed into orbit by the U.S. Department of Defense. The GPS originally was intended for military applications, but in the 1980s the U.S. government made the system available without cost for civilian use. Each GPS satellite circles the earth twice a day in a very precise orbit and transmits signal information to earth. GPS receivers record the signals from three or more satellites and use triangulation, based on the elapsed time between transmission and receipt, to calculate the user’s exact location. Relatively inexpensive GPS receivers (less than U.S. $100) are accurate to within about 15 meters and various enhancements can substantially increase accuracy. In addition, training of users is necessary to ensure accurate measurements for research purposes (Spencer et al. 2003).

Location and distance are key components of many processes social scientists study. For example, distance to market affects the choices by peasants of what crop to produce; the availability of local schooling is an important component of decisions about whether to keep children in school (Fafchamps and Wahba 2006); the degree of isolation from transport hubs may affect decisions to migrate; and so on. In the past, researchers usually have relied on subjective reports by respondents about distances, but such reports are known to be highly inaccurate, with error often correlated with outcomes of interest (Gibson and McKenzie 2007). Using GPS technology, it is possible to precisely measure the location of each household and each other point of interest (schools, roads, markets, clinics, or other public services, nearest cities, etc.). For useful examples of GPS-based studies of distance as a barrier to services, see Entwisle et al. (1997); and Rosero-Bixby (2004).

In a different kind of application, Rosero-Bixby and Palloni (1998) studied the relationship between population growth and deforestation in Costa Rica, which has experienced an increase in population and a decline in forest cover over a 40 year period. Their paper addresses a central debate in research on population and environment—the extent to which rapid population growth is associated with the massive deforestation currently underway in many tropical nations. They compiled geo-coded data from two population censuses and a series of land cover maps and used these to model the net impact of population growth on the 1973–83 probability of deforestation in about 31,000 750-square-meter parcels that were covered with forest at the beginning of the period. They showed a strong connection between population potential (the size of the population in surrounding areas) at the beginning of the period and subsequent deforestation.

Other satellite data

Satellite-based data, including both images and measurements of properties of the earth’s surface, are now available without cost to researchers for essentially all locations in the world, albeit with higher resolution in some places than others. Social scientists have made a variety of uses of such data, often combining satellite measurements with data collected through population surveys. For example, Frankenberg, McKee, and Thomas (2005) studied the impact of forest fires that blanketed the Indonesian islands of Kalimantan and Sumatra in 1997 on health outcomes by combining satellite measurements of aerosol levels in the atmosphere with longitudinal data for individuals from the IFLS. They showed that individuals who were exposed to haze from the fires experienced greater increases in difficulty with activities of daily living than did people living in nonhaze areas.

In another study, Frankenberg and her colleagues (Frankenberg et al. 2008) studied the effect of the 26 December 2004 tsunami on the mental health of residents of the Indonesian island of Sumatra, where 130,000 people died. Of interest here, they combined longitudinal sample survey data collected before and after the tsunami with community-level assessments of the extent of devastation caused by the tsunami. Devastation was measured by comparing satellite images from 17 December and 29 December for each of their 585 study sites. For each site, images for a .6- kilometer square centered on the study site were classified into three categories based on the degree to which the pre-tsunami ground cover visible in the first image had been replaced by bare earth in the second image. About 15 per cent of the areas were classified as heavily damaged (at least 20 per cent of pre-tsunami ground cover replaced by bare earth); about 35 per cent were classified as undamaged (no change in ground cover); and the remaining 45 per cent were classified as moderately damaged. They then showed that mental health was strongly affected by the level of damage, net of other factors.

A different application of remote sensing technology is to use night-light measurements to study population density. Zhuo and his colleagues (2009) generated population density measurements for 1-kilometer squares covering all of China based on 1998 night-light data. Because these data are geo-coded, they can be integrated with ground-based data from sample surveys. See also the U.S. Department of Energy’s global population database, LandScan, which provides annually updated population size estimates for 30-second by 30-second polygons (less than one kilometer square), using a regionally-specific model based on provincial-level census data and four geo-spatial input data sets (land cover, roads, degree of slope, and extent of night- time light) (see Dobson et al. 2000; Bhaduri et al. 2002; and http://www.ornl.gov/sci/landscan/).

Finally, there has been increasing interest in combining microdata (from surveys and censuses) with remotely sensed data on land cover and land use to study the impact of population on the environment, particularly global warming, with attendant changes in weather patterns; increased vulnerability to natural disasters; and the loss of bio-diversity. The collection of papers edited by Fox et al. (2003) provides a good introduction to this topic. See also Liverman et al. (1998), Matthews (2003), Walsh and Welsh (2003), and Walsh, Evens, and Turner (2004).

Biological Data

As social scientists have become increasingly interested in health, and health researchers have increasingly seen the advantage of population-based health studies, there has been a move to integrate questions about social and economic circumstances, health histories, and anthropometric and biometric measurements in general population surveys. One development, in particular, has been the collection of blood samples in order to do bioassays for the presence of various diseases and also of buccal cells to carry out genetic assays. However, the collection of venous blood in field conditions is difficult and expensive. This difficulty has led to the development of procedures for conducting bioassays and genetic assays from dried blood spots (DBS), which are much easier to collect and manage (McDade, Williams, and Snodgrass 2007). The IMHC and the China Health and Retirement Longitudinal Study (CHARLS) surveys are examples of Chinese surveys that have collected DBS successfully.

The basic procedure is that a respondent’s finger is sterilized and then pricked using a spring-loaded device. The resulting blood droplets (usually four or five) are then collected on filter paper, dried, stored in plasticine bags, and within seven days shipped to the laboratory that will do the bioassays, where they are stored in medical freezers (−20°C) until ready for analysis. In China, by law the person doing the blood collection must be medically trained. Thus, a nurse, doctor, or other health worker must accompany the interviewer to do the blood collection. This, of course, adds considerably to the cost and complexity of the fieldwork procedures; but it has the advantage of improving other biometric and anthropometric measurements because nurses and doctors are used to touching and physically manipulating patients and thus are able to put respondents at ease when they are taking body measurements, such as hip and waist circumference, procedures that may be awkward for interviewers, especially when dealing with the opposite sex.

Apart from blood spots, many population-based studies now include measurements of blood pressure, heart rate, lung capacity, hip, waist, and upper arm circumference, height and weight, and brief physical ability tests, e.g., the “timed chair stand” (how long it takes for a person to stand up and sit down five times), a grip strength test, and so on. Sometimes other biological specimens, such as buccal cell samples for DNA analysis and urine samples to study sexually transmitted diseases are collected as well. Examples of studies that include biological measurements are the U.S. National Longitudinal Study of Adolescent Health (Add Health), the U.S. National Health and Nutrition Examination Survey (NHANES), the U.S. Health and Retirement Study (HRS), IFLS, MxFLS, IMHC, and CHARLS. All these studies are available for analysis by the research community.

Two particularly interesting examples of the use of biological data in social research are a field experiment on the efficacy of iron supplements in Indonesia (Thomas et al. 2006) and a study of gene-environment interactions in the U.S. (Guo, Roettger, and Cai 2008).

Iron-deficiency anemia is a serious problem in many developing nations, increasing the risk of reduced immune response, delayed cognitive and physical development, and fatigue and reduced work capacity. In conjunction with an on-going panel study of the Indonesian population (IFLS), Thomas and his colleagues carried out an experiment in Java involving approximately 17,000 participants in which weekly iron supplements were administered to all members of half of the participating households selected at random. Measurements of hemoglobin, a marker for anemia, were taken before the supplements began and again after the first six months of intervention. In addition, changes in employment status and income were assessed. The results were striking. Men who were iron deficient prior to the intervention and who were provided supplements substantially increased their hemoglobin levels, increased the probability of working, reduced their hours of sleep, lost less time to illness, became more able to conduct physically arduous activities, and improved their emotional health. Among the self- employed, hourly earnings, and therefore monthly earnings, increased. Benefits for women were in the same direction but the effects were more muted.

Guo and his colleagues carried out an analysis of data collected as part of Add Health, which extracted DNA from buccal cells for a subsample of the 20,000 adolescents in the initial sample. They used data from about 1,100 males for whom measures of both DNA and delinquency were available to study the interaction of genetic and environmental influences on delinquency. They were able to show that three genetic polymorphisms were significant predictors of serious and violent delinquency when added to a social-control model of delinquency. But two of them interacted strongly with family processes, school processes, and friendship networks. Specifically, the risk of delinquency only increased among those who both had the genetic propensity and suffered from weak social control. This is one of the first studies to show genetic-environment interactions in the kinds of behaviors ordinarily studied by social scientists.

Improved Measurement

Here we touch briefly on the value added by longitudinal data and discuss the advantages of multiple measurement of concepts. There also have been useful new developments in scaling and measurement procedures, but space limitations preclude discussing them here. For a discussion of two of these developments, the use of anchor points to calibrate self-reports of health status and of the method of “unfolding brackets” to obtain information on income and wealth, see the expanded Chinese-language and PWP versions of this essay.

Longitudinal data

We already have referred several times to longitudinal, or panel, data, in which the same individuals are queried at two or more points in time. Such data have many advantages, including in particular their ability to fix people’s circumstances, conditions, behavior, and psychological states in time, and to purge from the analysis factors that are constant over time via fixed- or random-effects models, and hence to improve the possibility of being able rigorously to establish causal effects. But, as noted earlier, it is particularly important in panel studies to minimize panel attrition.

Even when it is not possible to mount panel studies, it sometimes is possible to generate over-time measurements by including retrospective questions in questionnaires, although in this case it is important to be sensitive to the possibility of recall error (Smith and Thomas 2003). One useful way to ask such questions is to collect “event history” rosters, for example, asking people to report each time they changed residence, each time they changed jobs, each time they married, and so on (for examples of the use of such rosters, see the questionnaires for the Life Histories and Social Change in Contemporary China (LHSCCC) and IMHC surveys).

Multiple measurement

It is well known that responses to survey questions, even factual questions such as the level of education completed by the respondent’s father, sometimes are erroneous. Even random error has serious consequences for statistical analysis, generally reducing the strength of the association between variables and, if the degree of error differs between variables in a model, distorting the relative strength of different effects. Systematic error, for example a tendency to overstate the similarity between mothers and fathers or to “yea-say,” giving positive responses regardless of the content of the question, is even worse, seriously biasing statistical estimates. Methods for dealing with measurement error, particularly random error, are well summarized in Alwin’s (2007) monograph. Most methods require either multiple measurements or multiple indicators.

Multiple measurements are independent measurements of a single underlying dimension either in the same survey (for example, by asking attitude questions using a 7-point scale near the beginning of a questionnaire and an 11-point “feeling thermometer” near the end of the questionnaire, or by asking about both the years of school completed and the highest level of schooling attained, etc.) or, if the phenomenon can be assumed to be constant over time, as father’s education usually is, by asking the same question in two or more waves of a panel study (see Smith and Stephenson 1979). Ganzeboom, Treiman, and Ultee (1991:292) discuss the advantages of multiple measurement and Ganzeboom (2005, 2007) provides examples of the advantage gained by measuring occupations in two different ways.

Multiple indicators are items that, at least in part reflect some underlying dimension, for example, asking people whether in the past year they had experienced unfair treatment in a variety of circumstances—by the police, by shopkeepers, etc.—in order to construct a “harassment” scale; see the questionnaire for the IMHC study.

Multiple measurements make it possible to compute estimates of measurement reliability, which can then be used to correct multivariate models using “errors-in-variables regression” (Kmenta 1997:352–357; Draper and Smith 1998:89–91; Treiman 2009:258–261). Multiple indicators also permit utilization of “multitrait-multimethod” models via confirmatory factor analysis procedures (Alwin 2007). Such models are a special case of structural equation models (SEM), which permit the estimation of systems of equations, often involving unmeasured or latent constructs; for a good introduction to such models see Bollen (2010).

For variables that must be coded, such as narrative reports on occupational position, an alternative to multiple measurement is multiple coding by independent coders—although building multiple measurements into the questionnaire is preferable. If, say, a set of narrative reports is independently assigned codes in the Chinese Standard Occupational Classification (CSCO) by two different coders, the codes can be converted into status scores (Ganzeboom, De Graaf, and Treiman 1992; Ganzeboom and Treiman 1996) and reliability estimates can then be computed or the two versions can be used in the “measurement model” portion of a SEM. It is important to recognize that in order to fully exploit the power of either multiple measurement or multiple coding, it is necessary to apply these procedures to all the variables entering into one’s model. For example, to study intergenerational occupational mobility, it doesn’t help much to have multiple measurements of the respondent’s occupation but not of the father’s occupation.

DATA SHARING

Several times we have mentioned data “in the public domain,” that is, freely available for use by the research community. The sharing of data is highly desirable, both because no one researcher or research group will have the imagination or energy to fully exploit the large and very expensive data sets that constitute the data base for much of contemporary social research and because independent validation and replication of results is the way science advances and error is corrected (Fienberg 1994). In the U.S., the two leading funders of scientific research, the National Science Foundation and the National Institutes of Health, require that all sample surveys and other systematic data collection efforts paid for by them must be deposited in an archive or otherwise made available⁸ within a reasonable period after the data are collected (the convention is two to three years, but many studies make their data available as soon as they are analysis- ready). The Organization for Economic Co-operation and Development (OECD), a consortium of about 30 democratic nations, mainly in Europe, has adopted a similar policy (OECD 2007).

Unfortunately, China is a laggard in this respect. To our knowledge, there currently are only three data archives in China: the Chinese Social Survey Open Database (CSSOD) at Renmin University; the China Survey Data Network at the China Center for Economic Research, Peking University; and the Databank for China Studies (DBS) at the Chinese University of Hong Kong. But the holdings of each of these archives are quite limited. Moreover DBS charges a usage fee. Nor are there strong norms encouraging researchers to share their data. As a result, most of the studies of the Chinese population freely available for research use either were conducted by or in conjunction with foreign researchers or have been made available in foreign archives with English-language documentation, but are not readily available from Chinese sources or with Chinese-language documentation. It is our hope that as China increasingly participates in the international research community, this situation will change.

A happy exception is the Chinese General Social Survey (CGSS), a Chinese version of the U.S. GSS. This survey was initiated in 2003 by Yanjie Bian, at the Hong Kong University of Science and Technology, with an explicit commitment to make each wave of data publicly available. Although the 2003 survey surveyed only urban China, each subsequent survey has been a probability sample of essentially the entire population of China. So far, this survey, which from 2008 has been carried out by Renmin University, has been conducted six times, in 2003, 2004, 2005, 2006, 2008, and 2010, with another wave scheduled for 2012. The 2003, 2005, 2006, and 2008 data are publicly available with the 2010 data scheduled for release sometime in 2012.

Other major sample surveys of the Chinese population that are available to researchers are described in the Appendix. Apart from creating research opportunities for scholars lacking the formidable resources of time, energy, and funding required to conduct their own surveys, the availability of multiple national probability sample surveys of the Chinese population has several additional advantages. First, it permits replication of results obtained from a single survey. If two or more surveys based on independent samples yield the same conclusions, we can have far greater confidence in the validity of the results than when they derive from a single survey. Second, it often is possible to combine surveys conducted in different years to permit cross- temporal comparisons (see Treiman 2012 for an example combining the 1996 LHSCCC and 2008 IMHC). Third, surveys can be combined to increase the sample size and thus the power of the analysis, which is particularly important when relatively small subcategories of the population are studied (see Zhang and Treiman 2011 for an example combining the IMHC and the 2008 CGSS).

APPENDIX

Chinese Sample Surveys of Interest to Social Scientists (Listed in the order of the year of initial data collection)⁹

Date:	1982
Title:	China Census of Population
Sample size:	10,039,191
Sample specifications:	1% public use sample drawn from census records
Design:	Single cross section
Principle investigator(s):	National Bureau of Statistics
Available from:	IPUMS International (https://international.ipums.org/international/)
Language of documentation:	English
Additional comments:	Chinese language documentation is available from DCS.
Date:	1985
Title:	China In-Depth Fertility Survey -- Phase I
Sample size:	13,307
Sample specifications:	Married women < age 50 who are permanent household members in Hebei, Shaanxi and Shanghai Provinces.
Design:	Single cross-section. Individual, household, and community questionnaires.
Principle investigator(s):	National Bureau of Statistics
Available from:	DCS
Language of documentation:	English
Date:	1987
Title:	China In-Depth Fertility Survey -- Phase II
Sample size:	49,458 households; 39,210 women.
Sample specifications:	Married women < age 50 who are permanent household members in Gansu, Guangdong, Guizhou, Liaoning, and Shandong Provinces.
Design:	Single cross-section. Individual, household, and community questionnaires.
Principle investigator(s):	National Bureau of Statistics
Available from:	DCS
Language of documentation:	English
Date:	1988, 1995, 2002, 2007
Title:	CHIP: Chinese Household Income Project
Note on series:	There are four surveys in this series, based on different samples and with different Principal Investigators. In some sense, this is a set of repeated cross-sections, although the design and the variables have changed from survey to survey. Also, the specificity of documentation varies from survey to survey. Where available, details are given separately for each year.
Sample size:	1988: 10,258 rural households (51,352 people), 9,009 urban households (31,827 people); 1995: 7,998 rural households (34,739 people), 6,931 urban households (21,694 people); 2002: 7,000 rural households (20,632 people), 9,200 urban households (37,969 people); 2007: unknown.
Sample specifications:	While it usually is asserted that data were collected from national probability samples of China, it is evident that not all provinces were included and that included provinces were chosen purposively. For details see Eichen and Zhang (1993) and Li et al. (2008).
Design:	Cross sections. Separate samples of urban and rural populations; varying questionnaires across surveys: individuals, households, school age children; migrants; community characteristics.
Principle investigator(s):	1988: Keith Griffin, and Renwei Zhao; 1995: Carl Riskin, Renwei Zhou, and Shi Li; 2002: Shi Li; 2007: unknown.
Available from:	ICPSR (http://www.icpsr.umich.edu/icpsrweb/ICPSR/), 1988: Study # 9836; 1995: Study # 3012; 2002: Study # 21741. The 2007 have not been deposited at ICPSR and do not appear to be publicly available as of January 2012.
Language of documentation:	English
Additional comments:	These surveys were carried out by the Chinese National Bureau of Statistics Urban and Rural Survey Units. The 1988 sample is described in Eichen and Zhang (1993) and the 1995 and 2002 samples are described in Li et al. (2008). There currently is no documentation of the 2007 survey.
Date:	1989, 1991, 1993, 1997, 2000, 2004, 2006, and 2009
Title:	CHNS: China Health and Nutrition Survey
Sample size:	Approximately 4,400 households, yielding approximately 19,000 individuals
Sample specifications:	Multistage semi-probability sample of 9 provinces
Design:	Panel (rotating panel). Also community questionnaires.
Principle investigator(s):	Barry Popkin and Fengying Zhai
Available from:	Carolina Population Center (http://www.cpc.unc.edu/projects/china)
Language of documentation:	English; Chinese
Date:	1990
Title:	China Census of Population
Sample size:	11,835,947
Sample specifications:	1% public use sample drawn from census records
Design:	Single cross-section
Principle investigator(s):	National Bureau of Statistics
Available from:	IPUMS International (https://international.ipums.org/international/)
Language of documentation:	English
Date:	1993
Title:	China Housing Survey
Sample size:	2,096
Sample specifications:	Probability sample of households in Shanghai and Tianjin
Design:	Single cross-section.
Principle investigator(s):	John R. Logan and Yanjie Bian
Available from:	ICPSR (http://www.icpsr.umich.edu/icpsrweb/ICPSR/)
Language of documentation:	English
Date:	1994
Title:	The State and Life Chances in Urban China 1949–1994
Sample size:	4,074
Sample specifications:	National probability sample of people age 25–65 residing in urban areas in China.
Design:	Single cross-section
Principle investigator(s):	Xueguang Zhou and Phyllis Moen
Available from:	ICPSR (http://www.icpsr.umich.edu/icpsrweb/ICPSR/)
Language of documentation:	English
Date:	1996
Title:	LHSCCC: Life Histories and Social Change in Contemporary China
Sample size:	6,090
Sample specifications:	National probability sample of people age 20–69
Design:	Single cross-section.
Principle investigator(s):	Donald J. Treiman, Andrew Walder, and Qiang Li
Available from:	UCLA SSDA (http://www.sscnet.ucla.edu/issr/da/).
Language of documentation:	English, Chinese.
Additional comments:	Also available from CSSOD. The documentation at CSSOD is not complete, but is in Chinese.
Date:	1998, 2000, 2002, 2005, 2008
Title:	CLHLS: Chinese Longitudinal Healthy Longevity Survey
Sample size:	Initial wave: 9,073 people age 80+; see project web site for other sample sizes.
Sample specifications:	People age 80 and older in 1998. Also, a comparison group age 65–79 and other special samples.
Design:	Panel, with replacements for deceased
Principle investigator(s):	Yi Zeng
Available from:	Center for the Study of Aging and Human Development, Duke University (http://centerforaging.duke.edu/index.php?option=com_content&view=article&id=115&Itemid=152)
Language of documentation:	English
Date:	1999–2000
Title:	CHFLS: Chinese Health and Family Life Survey
Sample size:	3,426
Sample specifications:	National probability sample of people age 20–64, with an oversample of high STD areas.
Design:	Single cross-section
Principle investigator(s):	William Parish and Edward O. Laumann
Available from:	http://www.src.uchicago.edu/prc/chfls.php
Language of documentation:	English; Chinese
Date:	2003, 2004, 2005, 2006, 2008, 2010
Title:	CGSS: Chinese General Social Survey
Sample size:	2003: 5,894 (urban only); remaining years, approximately 10,000 each.
Sample specifications:	National probability sample of people age 18–69
Design:	Repeated cross sections
Principle investigator(s):	Initially, Yanjie Bian; since 2008, Lulu Li
Language of documentation:	Chinese and English
Available from:	CSSOD
Additional comments:	Currently only the 2003, 2005, 2006, and 2008 data are available for public distribution, with the 2010 data schedule for release sometime in 2012.
Date:	2008
Title:	IMHC: Internal Migration and Health in China
Sample size:	3,000
Design:	Single cross-section
Sample specifications:	National probability sample of people age 18–64, with an over- sample of high in-migration areas.
Principle investigator(s):	Donald J. Treiman, William M. Mason, Shige Song
Available from:	UCLA SSDA (http://www.sscnet.ucla.edu/issr/da/)
Language of documentation:	English and Chinese. Currently, the Chinese documentation is limited.
Date:	2010–2020
Title:	CFPS: Chinese Family Panel Studies
Sample size:	14,798 households
Design:	Panel, with a new wave of household interviews every year and a new wave of adult interviews every other year.
Sample specifications:	Nearly national probability sample (excluding Tibet, Qinghai, Xinjiang, Ningxia, Inner Mongolia, and Hainan).
Principle investigator(s):	Yu Xie
Available from:	Institute of Social Science Survey, Peking University (http://www.isss.edu.cn/)
Language of documentation:	Chinese and English.
Additional comments:	A pilot study (N = 2,375 households) was conducted in 2008 in Beijing, Guangdong, and Shanghai, with a second wave of interviews conducted in 2009. The main survey was first completed in 2010.
Date:	2011-
Title:	CHARLS: China Health and Retirement Longitudinal Study
Sample size:	Approximately 10,000 households, 17,000 individuals.
Design:	Panel, with a new wave of interviews every two years.
Sample specifications:	National probability sample of adults age 45 and older. See Zhao et al. (2009) for details.
Principle investigator(s):	Yaohui Zhao, Justin Yifu Lin, John Strauss, Albert Park
Available from:	National School of Development (China Center for Economic Research) at Peking University (http://charls.ccer.edu.cn/charls/)
Language of documentation:	English and Chinese.
Additional comments:	A pilot study (N = 1,570 households, 2,685 individuals) was conducted in 2008 in Zhejiang and Gansu provinces. A national sample survey was scheduled for 2011, but no documentation is currently available.