Using Nonmedically Trained Interviewers to Collect Biomeasures in a National In-Home Survey

Abstract

A key operational component of the National Social Life, Health, and Aging Project (NSHAP) is the use of nonmedically trained interviewers to collect biomeasures in a national in-home interview of health and aging. Few studies have integrated in-home biomeasure collection using nonmedically trained interviewers on a large scale. In this article, we discuss our approach to using nonmedically trained interviewers to collect biomeasures in the home. The article focuses on activities that impact the ability to integrate biomeasures into survey research, including developing field methods, recruiting and training interviewers, and monitoring data collection activities. In addition, cooperation rates and measures of interviewer productivity and data quality are provided to evaluate our approach.

Background

Collecting biomeasures in the survey context is a relatively new tool for survey researchers. The National Social Life, Health, and Aging Project (NSHAP), an innovative, multi-modal study of older adults that examines the interaction between aging, social relationships, and health, is one example of a population study actively integrating social and biological data collection. Rather than presenting data resulting from the biological data collection, this article focuses on methodological and operational considerations for collecting biomeasures using nonmedically trained interviewers. We begin with an overview of NSHAP and other studies collecting biomeasures. We continue with a review of the study’s field methods and provide the methodological background for the use of nonmedically trained interviewers to collect biomeasures. We then discuss the importance of interviewer recruitment, methods for training nonmedically trained interviewers to collect biomeasures, and monitoring data collection and data quality during the field period. Last, we evaluate our methodology by examining respondent cooperation, interviewer engagement, interviewer performance, and data quality and provide a summary of issues for future consideration.

Overview of NSHAP

Between July 2005 and March 2006, NORC field staff completed 3,005, 2-hour, in-person interviews, with a nationally representative sample of community-residing adults aged 57 to 85. Interviews were conducted in both English and Spanish. The weighted response rate was 75.5% using the American Association for Public Opinion Research (AAPOR 2006) RR2 definition.¹ Most respondents received $100 in cash for their participation, but this was increased by increments of $100 to a maximum of $500 in the closing weeks of the field period to encourage participation from pending cases.

The NSHAP interview contained three components: (1) a detailed in-person questionnaire focused on physical, mental, sexual health, and social networks; (2) the collection of thirteen biomeasures; and (3) a post-interview mail questionnaire to collect survey data to supplement the in-home interview. To limit burden on the interviewer and respondent and to limit the time in the respondent’s home to 2 hours, the survey was modularized, so that all respondents were asked a core set of questions and biomeasures and a predetermined subset of modules. Figure 1 provides a list of biomeasures collected by questionnaire section (part of the core or part of the modules).

Figure 1.

Biomeasures by questionnaire section (core or module)

The modular format of the in-home interview resulted in each respondent being asked to participate in eight to twelve biomeasures depending on their random assignment. Neither the respondent nor the interviewer was aware of which modules the respondent would be asked to complete. Articles by O’Muircheartaigh, Eckman, and Smith (Forthcoming) and Smith et al. (Forthcoming) provide fuller descriptions of the sample design and the data collection methodology.

Survey Research Using Field Interviewers to Collect Biomeasures

The medical literature defines a biomarker as “a distinctive biological or biologically derived indicator (as a biochemical metabolite in the body) of a process, event, or condition (as aging, disease, or exposure to a toxic substance)” (Merriam-Webster Medical Dictionary 2003) and refers to the assay generated from a specimen as the biomarker. For example, albumin and cholesterol, two assays derived from blood, are biomarkers of cardiovascular health. However, as biomarkers become increasingly integrated into survey research, a more accurate definition is needed that encompasses not only “markers” but rather a broad range of biological, anthropometric, functional, and sensory measurements that can be collected from survey respondents. As such, the NSHAP research team adopted the term “biomeasure” to more accurately reflect a broader definition applicable to survey research.

Biomeasures have been collected in conjunction with survey data for many years. Depending on the specific requirements of the study, including type of biomeasure, cost, and demographics of the study population, biomeasures and survey data can be collected from respondents by using one or more data collection models. At one end of the spectrum—in terms of cost, range of biomeasures possible, and quality of data—is the centralized collection of biomeasures by health care professionals at a medical facility. In this case, respondents might be asked to visit a hospital, clinic, or specialists’ office to complete one or more biomeasures. The questionnaire can either be administered during these visits or collected at another time. A variation of this model that retains many of the benefits of using high-quality facilities, equipment, and trained medical staff is the use of mobile clinics that can be moved from location to location. In the United States, a prime example of the latter is the National Health and Nutrition Examination Survey (Centers for Disease Control and Prevention 2004). Internationally, Papua New Guinea, Columbia, Egypt, and Pakistan have undertaken national health examination surveys utilizing the mobile clinic design (National Academy of Sciences 2000).

At the other end of the spectrum is self-administered biomeasure collection using equipment that is mailed or left with the respondent. One example of this method is the collection of a DNA sample through a mail-back system used on the Wisconsin Longitudinal Study (Oliver and Crose 2007). Compared to in-home collection, this method can be cost effective but it is limited by the range and complexity of biomeasures that can be collected: some equipment may not be suitable for mailing or handling by the respondent or the correct administration of the biomeasure protocol may require training beyond what can be provided to the respondent in simple written instructions. In addition, the quality of the respondent’s address information and control of the specimen collection and shipping can be problematic (i.e., was the collection protocol followed correctly, and has it been handled and shipped as instructed?).

Somewhere in this spectrum of biomeasure collection methods is the use of nonmedically trained field interviewers to collect biomeasures. Recent developments of robust, portable, reliable, and easy-to-use equipment have made a wider range of in-home biomeasure collection feasible on larger samples using nonmedically trained field interviewers. Field interviewers give us opportunities to capture biomeasure data efficiently from nationally representative populations and do so cost effectively. Field interviewers can also overcome difficulties (such as respondent-restricted mobility because of poor health, lack of transport, or remoteness) to gain the participation of population subgroups that might otherwise be missing or underrepresented in the study population. For example, the 2004 wave of the Health and Retirement Survey (HRS) used field interviewers to conduct a breathing test (peak flow), grip strength, timed walk, and record height and weight (The Institute for Social Research 2006); and Wave III of the National Longitudinal Study of Adolescent Health (Add Health) collected oral mucosal transudate (OMT), urine, saliva sample, weight, and height (Add Health Biomarker Team 2003). Furthermore, the availability of field-friendly equipment has made the collection of biomeasures possible around the globe. Since 1984, thirty of the seventy-five countries participating in MEASURE Demographic and Health Surveys (DHS) have combined survey data with an array of biomeasures including anthropometric measurements, blood spots, and HIV testing (Macro International, Inc. DHS 2007).

Methods

Developing Field Methods and Procedures to Collect Biomeasures

The integration of biomeasures into NSHAP enhanced the study’s scientific objectives in three major ways. First, the integration of biomeasures permitted estimation of the incidence and prevalence of certain diseases among older adults (e.g., bacterial vaginosis, human papilloma virus). Second, they permitted the detection of undiagnosed or subclinical conditions such as hypertension and diabetes that may have a potent effect on sexual function and/or overall well-being. Third, they provided objective measures of health status (e.g., height, weight, vision) for use in conjunction with self-reported measures. The range of biomeasures collected in NSHAP represented a balance between the scientific value of the data, minimally invasive techniques accepted by respondents, and practical considerations of implementation.

A key element of NSHAP was the use of nonmedically trained field interviewers to collect biomeasures from survey respondents. This required NSHAP to prioritize the use of minimally invasive collection methods that could consistently and easily be taught to nonmedically trained interviewers. York, Lindau, and Mahay (2004) proposed that the criteria for minimally invasive collection include minimal risk to the respondent or interviewers during collection, minimal psychological discomfort, minimal physical demand, and quick, cognitively simple collection with maximum protection of privacy.

Previous studies have successfully used medical technicians and phlebotomists to collect biomeasures in population studies (Crimmins and Seeman 2001; Weinstein and Willis 2001). To confirm that our approach to training nonmedically experienced field interviewers was the best approach for NSHAP, we explored alternative collection methods, such as contracting with a paramedical or mobile examination company. Benefits of subcontracting the biomeasure collection included: (1) reducing the time required by NSHAP staff for training and developing training materials and (2) the ability to employ already trained and experienced collection staff. However, for NSHAP, these benefits did not outweigh the costs, which included the risk of incomplete interviews (i.e., failure to collect biomeasures) if separate appointments needed to be scheduled for the interview and biomeasure collection and the considerable effort required to coordinate schedules between interviewers and phlebotomists during the field period. Weighing these choices, combined with the knowledge that studies such as Add Health and HRS have previously used field interviewers to successfully administer a limited set of biomeasures, we proceeded with using nonmedical interviewers to collect minimally invasive biomeasures.

Before conducting the training, we devoted significant resources to the development of field methods. Each protocol was designed to be minimally invasive and safe for both the interviewer and respondent. Each collection protocol was discussed as part of an interdisciplinary biomeasure working group and included collaborations with the principal investigators, field staff, laboratory staff, and external researchers. Because existing protocols for collecting biomeasures were often based on clinical or laboratory administration, the working group focused on ensuring that the biomeasures were universally collectable in the home setting. When available, the working group adapted clinical collection protocols, manufacturer guidelines, laboratory guidelines, or modifications from other population studies such as HRS (2006), NHANES (2004), AddHealth (2003), and other sources.²

To ensure a consistent development process, five additional considerations were discussed by the NSHAP biomeasure working group before including each biomeasure in the in-home interview:

1. Adapt to in-home collection and shipping: Do field procedures already exist to measure and collect the data in the home or are procedures based on collection in the clinical or laboratory setting? Can time and temperature sensitive specimens be shipped in accordance with federal guidelines?

2. Adapt to lay administration: Can the field collection procedures be administered by a nonmedically trained individual or is a medical background required?

3. Tailor the procedures to the target population: Can these procedures be easily used with respondents in the target age group? Are there special considerations for the NSHAP sample who are aged fifty-seven to eighty-five?

4. Choice of equipment: What equipment is needed to conduct this measure? Is the equipment safe, portable, easy-to-use, reliable, and economical? Can it be used easily by the respondent and the interviewer?

5. Examine interviewer and respondent burden: Even if a measure is feasible, is the burden (of each measure and the sum of all measures) too great for the interviewer or the respondent?

Additional Factors Shaping the Development of Field Biomeasure Methods

A pretest of the field procedures, the study design, and the requirements for the protection of human subjects can also significantly impact the development of field methods and the training of nonmedically experienced interviewers.

Pretest

A small pretest of fifty-three respondents was conducted in the summer of 2004. The purpose of the pretest was to test the field procedures, questionnaire, and biomeasures before beginning Wave I. Guided by the five considerations listed above, we learned many valuable lessons about the feasibility of integrating biomeasures into a population survey. Specifically, as discussed later in this article, the pretest established the feasibility of using nonmedically trained field interviewers to collect biomeasures and showed that traditional interviewer skills such as gaining cooperation were crucial to completing cases. However, with respondents being asked to participate in all questionnaire and biomeasure items, interviews often lasted as long as 4 hours. The pretest indicated that a shorter interview was required and led to the modularization of the in-home interview. The pretest also demonstrated a need to increase the pretest incentive of $25 to $100 in Wave I. Based on the pretest experience, the biomeasure protocols and training plans were revised to include a set of core and modular biomeasures to reduce time spent in the home. Time and budget constraints did not permit another pretest. Instead, project staff conducted a series of in-home timings with recruited age-appropriate respondents to evaluate the revisions to the questionnaire and biomeasure protocols.

Study Design

It is also important to consider the project’s study design when developing field methods. For example, because of NSHAP’s national probability sample, biologic samples collected during the interview could not easily be stored and shipped from a centralized location to the laboratory. As such, field methods were developed in which interviewers shipped samples directly to the laboratory. Shipping protocols are further described in our approach to in-person training.

Informed Consent and Human Subjects

The inclusion of biomeasures also brings unique challenges for researchers and institution review boards that create and review protocols incorporating biomeasures. Drawing on experiences from the NSHAP pretest, we made every effort to comprehensively provide for informed consent throughout the interview. As such, a participant’s right to seek and obtain additional information regarding the questions or biomeasures at any time during the interview and, the right, at any time, to be able to withdraw consent without penalty was a key aspect of our informed consent procedures.

The pretest and Wave I data collection led us to believe that at a minimum, human subjects protocols should address the rationale for and knowledge to be gained by including biomeasures, the procedures to which respondents will be subjected, the potential benefits to respondents and society, and the potential risks to respondents and precautions to minimize those risks. In addition, we paid special attention to drafting an informed consent statement in easily understandable language that fully informed respondents about what measures they will be asked to provide and for what conditions their samples will be tested. A discussion of whether respondents will receive their results (and which results) should also be addressed. NSHAP respondents received their results from the biomeasure collection. Results that were immediately available (weight, waist, height, blood pressure, smell, taste, touch, vision, get up and go) were provided at the end of the interview. Results that required processing at a laboratory (assays from vaginal swabs and HIV testing) were provided to the respondent via an anonymous toll-free results hotline. Results from measures using analysis methods that were not determined to be clinically relevant or valid at the time of the study (assays from blood spots and saliva) were not provided to respondents.

Of equal importance to human subjects protections is the maintenance of respondent confidentiality and the confidentiality of the biological data collected. A review of the research safeguards for data security is essential. Additional safeguards, such as obtaining the Certificate of Confidentiality issued by the National Institutes of Health (National Institutes of Health 2003), can also be used as a tool to prevent forced disclosure of participant information.

Furthermore, a discussion of whether samples will be saved or discarded is necessary. During the NSHAP interview, respondents were asked to consent to the storage of their blood samples for future analysis related to aging. Because the storage of blood samples included the possibility of consenting to genetic analysis, some participants were hesitant. To manage these situations, interviewers were trained to explain the process and offer consent options. The decision to offer respondents a choice regarding the storage of their samples was based on the biomeasure working group’s interpretation of the guidelines for research involving human and biological materials (National Bioethics Advisory Commission 1999). Specifically, Recommendation 9 suggests that respondents should be given choices to help them understand the implications of their decision to consent to storage.

As a result, NSHAP respondents were given three options: (1) to store their blood and be contacted before it is used; (2) to store their blood without future contact before it is used; and (3) to decline storage. In addition, all respondents were informed that they could withdraw their stored samples at a later date. The combination of providing these options, a consent form written in easily understandable language, and a skilled interviewer, helped the participants make an informed decision. Ultimately, 89% of respondents agreed to allow their blood samples to be stored for future analysis, 58% of whom agreed that is was not necessary to recontact them before using their sample for other analyses.

Examining the Importance of Interviewer Recruitment and Screening

After developing the field methods, it was imperative to recruit, screen, and hire interviewers who were capable of collecting the biomeasures during the in-home interview. The project was planned with the intention of hiring nonmedical interviewers, but we were initially uncertain about the characteristics and skill set of an ideal NSHAP interviewer. NORC’s typical field interviewers are not recruited based on their biomeasure collection skills. So, although NORC interviewers had the skills to gain respondent cooperation and administer a questionnaire, we were uncertain if these same skills would transfer to collecting biomeasures.

The pretest provided valuable insights into what skills the best candidate must have to collect both biomeasure and questionnaire data. We recruited both experienced survey interviewers as well as individuals with some medical background, such as paramedical examiners or medical assistants. Thirteen interviewers were hired for the pretest; about half had a traditional survey interviewing background and half had some medical background. Although we did not conduct a formal experiment, the interviewers’ performances were evaluated during and after the field period. The three key evaluation criteria were the number of completed interviews, the cooperation rates for the biomeasures, and the quality of the laboratory samples. These three criteria allowed us to determine which interviewers were able to gain the respondent’s cooperation for participating in NSHAP, were comfortable collecting the biomeasures and making the respondent comfortable with the collection procedure, and were able to collect the best quality samples. This evaluation identified three valuable lessons about the ideal interviewer for Wave I of the study.

1. Traditional interviewer skills such as good verbal and listening skills are critical. Even if interviewers were comfortable with biomeasure collection, they must have excellent communication skills to explain the importance of the study to respondents and gain their cooperation. Interviewers who are highly proficient only in the collection of biomeasures would not be able to get into a respondent’s home if they lacked communication skills.

2. A medical background is not necessary for collecting biomeasures if minimally invasive techniques are used for collection. Based on their pretest performance, we concluded that it is possible to train nonmedically trained staff to follow the procedures for biomeasure collection if you use minimally invasive collection techniques.

3. The candidates must be comfortable and willing to collect the biomeasures. Not everyone is comfortable with biomeasure collection, even if simple and easy to administer collection procedures are developed. Further, if an interviewer indicates a willingness to collect the biomeasures but is demonstrably uncomfortable with the procedures, there is a risk that respondents may sense this apprehension and consequently the biomeasure cooperation rates may suffer. Although it is best to assess the candidate’s comfort level during recruitment, before time and resources are spent on training, sometimes discomfort can only be identified at training.

These three pretest recruiting lessons guided the Wave I interviewer recruitment process and informed our strategy to recruit experienced survey research interviewers. Before recruiting began, we sought to determine if our experienced interviewers would be willing to work on this unique project. A project description that focused on the biomeasures and a data collection timeline was mailed to our pool of experienced interviewers to determine interest in working on the project. Out of 304 interviewers who received our initial inquiry, 248 (82%) were interested in working on NSHAP, which created our hiring base. The remaining 18% either had time conflicts or were not comfortable with the study’s subject matter or collecting and storing the biomeasures. Experienced interviewers were staffed wherever possible and were evaluated on their previous work, feedback from the field management staff, and proximity to the NSHAP sample. Some new interviewers were recruited to supplement our interviewer team. Using our pretest knowledge, we sought to fill these remaining positions with individuals who had traditional interviewer skills. Because of some sensitive questions and biomeasures (e.g., self-administered vaginal swabs), only female interviewers were allowed to interview female respondents. Although the majority of the field interviewing staff was female, both male and female interviewers were staffed for the project.

To ensure that the candidates were comfortable and willing to collect biomeasures, we conducted a thorough screening before hiring candidates. Both experienced interviewers and newly hired interviewers participated in this screening. During screening, candidates were provided with clear expectations about their duties on the project. For example, for measures such as weight and blood pressure collection, a written or verbal explanation of the measures was provided. For more complex measures, additional detail was necessary to provide a clear picture of the interviewer duties. For example, candidates were provided with a pictorial outline of the key steps in the blood spot protocol to make it apparent they would be pricking the respondent’s finger and placing blood spots on collection paper. In addition, candidates were informed that saliva samples must be stored in their freezers until the designated shipping date and were shown pictures and provided measurements of the saliva storage box.

A final point considered during recruitment was the interviewer’s attitude toward the project. NSHAP was an ambitious project, and the interviewers had the crucial task of “selling” the project to the respondents. Feedback received from the interviewers in training evaluations, from team calls throughout data collection and in debriefing sessions after data collection ended indicated that they truly believed in the purpose and objective of NSHAP. We feel that this positive attitude helped interviewers gain respondent cooperation.

Exploring Approaches for Training Nonmedically Trained Interviewers to Collect Biomeasures Once hired, the next important step was to train the interviewers on proper data collection techniques. We instituted a multidimensional approach to training that included a home study package, in-person training, and booster trainings during data collection.

Home Study Package

A home study package was sent to all hired interviewers to complete before they arrived at the in-person training. The home study package contained a field manual, video, and homework assignment. Interviewers were asked to review the field manual, which covered everything from administrative activities to a description of the questionnaire. The manual provided an overview of the biomeasure collection and contained step-by-step collection instructions with photographs or diagrams to illustrate the process.

Interviewers were also required to watch a training video before arriving at training. The video was created specifically for NSHAP training purposes and it depicted a scripted interview that focused on biomeasure collection during the interview. The video portrayed a mock interview so the viewers would observe the interviewer explaining the biomeasure to the respondent, interacting with the respondent during the biomeasure, entering data into the computer, and transitioning to the next biomeasure. In addition, a separate segment of the video instructed interviewers on how to ship specimens to the corresponding laboratories. The interviewers were not expected to learn all the collections procedures solely through watching the video. Instead, its purpose was to give the interviewers an overview of the collection process and see how the individual measures flowed together. After reading the field manual and watching the video, interviewers were expected to answer questions on key points and bring their answers to the in-person training. This helped the interviewers focus on the key training points while reviewing the materials and ensured that the interviewers reviewed the home study materials before coming to in-person training.

Although the primary purpose of the home study package was training, it was also the last step of the screening process. The video in particular gave the interviewers a clear picture of their tasks regarding biomeasure collection and shipping. Requiring interviewers to observe their job duties before attending in-person training contributed to the low interviewer attrition we discuss later in this article.

In-Person Biomeasure Training

Over the summer of 2005, NSHAP trained 130 field interviewers to administer the questionnaire and biomeasures. Approximately forty to forty-five interviewers attended one of three trainings. Each training was subdivided into smaller groups of ten to twelve interviewers who rotated through modules as a group. The in-person training was comprised of four days of NSHAP-specific instruction divided between two project training days of active lecture, each focusing on field operations and administering the questionnaire or collecting the biomeasures, a self-study day, and a certification day.

Active Lecture

Project training on biomeasure collection employed an active lecture format to train interviewers to collect each of the thirteen biomeasures. Instruction included modules on each measure, informed consent, safety and universal precautions, and equipment storage and use. A consistent format was followed for each biomeasure module: The trainer would explain the collection procedure, the trainer would demonstrate the collection procedure, and then the trainee would practice the collection procedure. This approach was modeled after the principles of active learning described in the Cone of Learning (Dale 1969; Houle 1972), which proposes that after 2 weeks, we tend to remember 90% of what we do, 50% of what we watch, and 20% of what we hear. Using these principles, each biomeasure was presented in three steps, “tell, show, and do,” and were repeated for the instruction of each biomeasure protocol. For the tell step, the trainer described the collection steps and emphasized important details using PowerPoint slides as a visual aid. During the show step, the trainer played the role of the interviewer and the assistant trainer played the role of the respondent. In this demonstration, the “respondent” is cooperative so that the trainees could focus solely on the collection steps. In the do step, the trainees performed the biomeasure collection steps with a partner.

Self-Study

Practice sessions were built into the training to enforce consistent collection of the methods learned during the active lecture day. Self-study modules included mock interviews, appointments with trainers, and shipping and blood spot practice. Throughout the self-study day, interviewers rotated through these modules with the goal of obtaining dedicated small group and individual instruction. For example, during the mock interviews, field interviewers were matched with a partner and required to perform “dry runs” of the entire interview. These mock interviews provided the interviewers with both an opportunity to put all the pieces of the interview together and to continue practicing the individual biomeasure collection procedures. Furthermore, it gave the pair a chance to role play the respondent, increasing awareness of how it feels to be a respondent asked to provide various biomeasures. Feedback received from their peers in these sessions helped interviewers focus on areas for improvement. During their appointment with a trainer, two interviewers were paired with one trainer for a practice session driven by interviewer requests and training needs. In the blood spot collection session, interviewers worked directly with trainers and staff from one of the laboratories to improve their collection techniques. Finally, in a module on shipping, the interviewers followed an overview and demonstration on shipping and were then required to package samples by following the step-by-step shipping protocols. All of these pieces helped build interviewer competency and confidence. By the end of the training, interviewers moved from an interest in biomeasure collection—“I can learn this”—to mastery of a skill set that allowed them to enter the field saying—“I can do this.”

Certification and Prefield Practice

The final element of the in-person training was certification. Each interviewer was individually certified by a trainer to ensure that they correctly followed collection protocols and were comfortable obtaining samples from respondents. In addition, before being allowed to begin data collection, interviewers were instructed to recruit a friend or family member and conduct a final run-through of the entire in-home interview. This mock interview gave the interviewer an opportunity to practice the interview with someone unfamiliar with the project and allowed the interviewers additional time to process the information from training before completing their first case.

Booster Training

An additional training component used during the field period were booster trainings. These brief training sessions were conducted remotely after the interviewer had been certified, returned home, and began data collection. There were two purposes of the booster training: (1) to reinforce specific aspects of in-person training and (2) to provide additional training on new issues encountered in the field. Booster training sessions were held for both questionnaire- and biomeasure-related topics (such as gaining cooperation for the vaginal swabs and refreshing the proper collection technique for the blood spots). The booster training addressed these issues through e-mails, visual aides (such as pictures or examples of correctly filled-out laboratory paperwork, feedback from field managers, and group phone calls. The group phone calls were particularly helpful as they provided an opportunity for interviewers to both ask questions of their peers and to give helpful advice and examples of techniques that were working for them in the field.

Quality Control during Data Collection

The significant investment in the development of field methods and the training of nonmedically experienced interviewers were essential first steps toward collecting quality data in the field. The next steps consisted of the careful monitoring of the data collection.

To ensure the quality of the data collected, matching results from the biomeasures to the questionnaire data was of key importance to NSHAP. Careful tracking of the specimen was required to be confident that the data recorded in the questionnaire were correctly matched with the data delivered to and analyzed at a laboratory. For NSHAP, this tracking was operationalized as an interactive reconciliation system that allowed remote tracking of the biological specimen from the point of in-home collection to delivery in the final dataset. Using materials specifically developed for this task, field management staff located across the country, outside laboratory contractors, and the project management team in Chicago were able to communicate in a timely manner about the transport status of each specimen. Figure 2 provides an overview of the chain of custody by transmission mode.

Figure 2.

Overview of specimen transmission from in-home collection to final dataset

In addition, it was essential to understand sources of error and their impact on the data. Types of error include collection error, data entry error, transport error, and forces of nature (e.g., power outages or severe weather responsible for loss of samples or temperature fluctuations that degrade the specimens). A tracking and reconciliation system works best when it operates in tandem with data collection. If reconciliation activities are left until the end of the data collection, it may be too late to discover missing samples or data. At that point, taking corrective action, such as the recollection of samples, may not be practical (e.g., because of respondent and interviewer availability or cost) or desirable from an analytic perspective (as the samples are now being collected at a different time than the questionnaire data).

In addition to monitoring specimen quality, it was also crucial to examine the role of respondent participation and interviewer effects on data quality. As described below, during the field period we looked at their impact on data quality by examining cooperation rates and interviewer effects.

Cooperation Rates

Similar in purpose to tracking the survey’s response rate to inform our understanding of measurement error, cooperation rates for each biomeasure were monitored regularly over time to monitor progress, track changes in cooperation, and inform training needs. One example of this monitoring was examining potential effects of an increased monetary incentive to nonrespondents. To examine potential incentive effects on biomeasure cooperation, we separated the respondents into two groups: respondents who received the original incentive of $100³ and respondents who received an increased incentive of $200 to $500. Although chi-square tests of association uncovered some differences between incentive groups (similar or lower cooperation for respondents receiving the increased incentive on most measures), most differences were not significant. However, respondents who received the increased incentive had significantly lower levels of cooperation (p <.01) for the saliva, taste, and distance vision measures. This analysis of cooperation rates during data collection allowed us to examine preliminary effects of incentives on biomeasure cooperation between early and late responders and was used to inform field interviewers if there was a need to modify their approach to gaining cooperation as a result of the higher incentive.

Interviewer Effects

We also examined the effect of interviewers’ characteristics on biomeasure collection by reviewing completion rates by interviewer for each measure. This allowed us to ensure that interviewers were comfortable with the collection. If one field interviewer had less cooperation than the others on a certain measure, perhaps it was not because their respondents did not want to participate, but rather that the interviewer was not comfortable with collecting the biomeasure. Monitoring completion rates by interviewer was a useful tool during the data collection period, but the rates were interpreted very carefully and considered with other monitoring efforts.

Another method to explore data quality was examining interviewers’ success with specimen labeling, quality, and shipping. If there were specific labeling or shipping problems, this information was passed on to the interviewer. At other times, all feedback, both positive and negative, was grouped by interviewer and shared with the interviewer. This helped everyone see whether positive or negative instances were isolated or habitual. Group feedback was also provided if emerging trends or issues experienced by multiple interviewers were discovered.

Results

To assess the success of using nonmedically trained interviewers to collect biomeasures, we conducted analyses on operational factors including respondent cooperation, interviewer engagement, interviewer performance, and data quality.

Respondent Cooperation

Completed interviews were obtained from 3,005 individuals: 1,455 males and 1,550 females. Cooperation rates from the interviews indicate that nonmedically trained interviewers were successful in obtaining respondent participation in biomeasure collection, with most measures having cooperation rates of 90% or higher.⁴ Listed in Table 1 are the weighted cooperation rates listed from highest to lowest. In addition, because respondents were randomly assigned to participate in core and modular measures as described in Figure 1, the number of eligible respondents for each measure is also provided.

Table 1.

NSHAP Biomeasure Cooperation Rates

Measure	Cooperation Rate+	Eligible Respondents
Height	98.6%	2,977
Touch	98.4%	1,505
Weight	98.4%	2,977
Blood Pressure	98.4%	3,004
Smell	98.3%	3,004
Waist	97.2%	3,004
Distance Vision	96.0%	1,505
Taste	95.9%	3,004
Get Up and Go	93.6%	1,485
Saliva	90.8%	3,004
OMT	89.2%	972
Blood spots	85.0%	2,494
Self-administered vaginal swabs	67.5%	1,550

⁺Person-level weights are adjusted for non-response by age and urbanicity.

Interviewer Engagement

A frequent question during the initial planning phase of NSHAP was whether interviewers would be willing to collect biomeasures. As a result of our thorough recruitment and training process, we did not experience interviewer attrition during the in-person training as a result of difficulties with biomeasure collection. Our interviewer certification levels and field productivity are also a reflection of interviewers’ willingness to collect the measures. Of the 130 interviewers who attended in-person training, 128 were certified at training and completed an interview.

In addition, because most large field projects with lengthy data collection periods inevitably suffer some level of interviewer attrition, we planned to hold additional trainings to accommodate newly hired interviewers. However, no attrition trainings were needed and the field staff remained stable throughout the data collection period. Training evaluations and post data collection interviewer debriefings showed high levels of job satisfaction, with interviewers commenting that they had enjoyed the additional challenges of learning and administering the biomeasures. They also expressed interest in resuming work on NSHAP in upcoming data collection waves.

Interviewer Performance

Interviewer performance with the biomeasures was assessed by examining biomeasure cooperation rates by individual interviewers. The cooperation rates varied by interviewer, with certain biomeasures demonstrating more variation than others. For example, an assessment of the cooperation rate for weight by individual interviewer showed relatively modest variation: All interviewers achieved cooperation rates between 75 and 100%. By comparison, the self-administered vaginal swab cooperation rate by individual interviewer showed the most variation. Out of 125 interviewers, fifty-seven achieved self-administered vaginal swab cooperation rates between 75 and 100%, forty-three between 50 and 74%, seventeen between 25 and 49%, and eight between 0 and 24%. Although this example illustrates variation in cooperation rates across interviewers, all conclusions should be considered in context when communicated with the field staff, and a direct comparison is not recommended. For instance, interviewers are not randomly assigned respondents and certain interviewers are assigned more difficult respondents, which may impact biomeasure cooperation. Also, some interviewers were assigned a small number of cases, and we wanted to be careful about drawing general conclusions about their performance from a small number of completed interviews.

Data Quality

Another indicator of the interviewers’ success with biomeasure collection is the quality of the specimens collected and delivered to the laboratories for analysis. All laboratories provided feedback on the quality of the specimens received, including how they were collected, stored, and shipped as well as if the assay results were within expected ranges. All laboratories that processed NSHAP specimens reported receiving high-quality samples that were similar in quality and outcomes to other studies.

Feedback from laboratories regarding the blood spots and self-administered vaginal swabs illustrates the quantity and quality of the specimen collection. Blood spot specimens were evaluated by the quantity of subsamples they produced for analysis, with four subsamples being needed to complete all tests. NSHAP collected more than 2,000 specimens and the mean number of subsamples was eleven per specimen, well above the minimum requirement for processing. The quality of the blood spots collected was also high, with 99% of all subsamples usable. Samples were not usable if they were too small, overlapping, or blotted against the paper. Based on this evidence, the laboratory concluded that in-home blood spot collection by NSHAP interviewers was feasible and successful (Williams, Lindau, and McDade 2006). Similarly, the majority of self-administered vaginal swab specimens—957 of 1,016 (94%) bacterial vaginosis and vaginal candidiasis samples, 1,010 of 1,012 (99%) human papilloma virus (HPV) samples, and 923 of 1,016 (91%) cytology samples—were adequate for analysis. Based on this assessment, the laboratory concluded that the quality of the self-administered, vaginal swab specimens was high (Lindau, Drum et al. Submitted; Lindau, Hoffman et al. Submitted). General distributions of other measures are currently being analyzed against results from other population studies.

Discussion

The integration of biomeasures into survey research presents many new opportunities and challenges. In this article, we presented one model for collecting these measures using nonmedically trained interviewers. Our experience on NSHAP builds on the work of previous domestic and internationally based studies integrating biomeasures into population research and confirms the feasibility of using field interviewers to collect minimally invasive biomeasures from respondents. Specifically, we found that quality data can be consistently and easily collected from respondents while maintaining fidelity to collection methods. Overall, we were very satisfied with the high respondent cooperation with most biomeasures having cooperation rates 90% or higher, the willingness of interviewers to work on NSHAP, the solid interviewer performance, and the high-quality specimen collection.

Biomeasure data collection is only as successful as the processes taken to plan, pretest, train, and monitor the effort. During planning, field methods must be adapted or developed for in-home and lay-administration and tailored to the target population. Collection equipment and supplies must be safe, portable, and easy to use. Further, all methods must be implemented with a goal of limiting interviewer and respondent burden. A pretest of these methods is recommended as a valuable tool for identifying improvements and refining field methods. In addition, interviewer recruitment and training are key steps in successfully implementing the biomeasure collection methods. Our experience suggests hiring individuals with traditional interviewer skills who are comfortable and willing to collect biomeasures. Using a multidimensional training with a focus on individual feedback and hands-on practice was an effective approach in training interviewers to collect biomeasures.

Ultimately, ensuring the quality of the data collected was crucial to the success of the project. On NSHAP, this meant tracking samples from in-home collection to the final dataset. Tracking cooperation rates and interviewer effects also afforded opportunities to provide regular feedback on mistakes and reinforce positive outcomes. Although this article presents many of the lessons learned on NSHAP, continuing analysis of the methodology and the resulting data are needed. At a minimum, we recommend further studies of the quality of biological results and analysis of biomeasure nonresponse.

Finally, although we are eager to apply our lessons learned to other field projects, it should be noted that in-home biomeasure collection is a rapidly evolving undertaking. As biological data collection becomes increasingly integrated into survey research, there is a need to continue to develop new field-friendly methods to stay abreast of rapidly developing advancements in technology. Undoubtedly, such advances will soon allow for the collection of a wide array of minimally invasive biomeasures previously unavailable to survey researchers. In addition, the changing demographics of an aging population will require the examination of cultural differences in the acceptability of in-home biomeasure collection. As we build on the model discussed in this article for future waves of NSHAP data collection, we are mindful of these technical advances and cultural considerations that will necessitate changes and refinements to collection and training protocols.

Glossary

NORC

National Opinion Research Center

Biographies

Angela Jaszczak, MPH, is a Senior Survey Director in the Health Care Delivery and Health Outcomes Department at the National Opinion Research Center (NORC), which is affiliated with the University of Chicago. In addition, she was the Associate Project Director of the National Social Life, Health and Aging Project (NSHAP) data collection activities. She has a background in public health and health education and has worked on several projects and surveys related to health care. Jaszczak has given presentations on integrating biomeasures into survey research at numerous conferences. She is currently working on the second wave of NSHAP.

Katie Lundeen, MA, is a Survey Director in the Health Care Delivery and Health Outcomes Department at the National Opinion Research Center (NORC), which is affiliated with the University of Chicago. She has a background in Sociology and has worked on several health care survey projects. Lundeen has given presentations on training field interviewers to collect biomeasures in the home at several conferences. She is currently working on the second wave of NSHAP.

Stephen Smith, MA, is a Senior Vice President and Director of the Health Care Delivery and Health Outcomes Department at the National Opinion Research Center (NORC), which is affiliated with the University of Chicago. His work encompasses a broad range of studies, including population-based health behaviors and outcomes, patient and community health needs assessments, patient satisfaction, physician communication, and physicians’ delivery of health care. He was Project Director for the data collection activities on the first wave of the National Social Life, Health, and Aging Project (NSHAP), and will play a similar role for the second wave of the study, which is now in the planning and development phase.