Systematic Review of the Use of Online Questionnaires among the Geriatric Population

Abstract

Background/Objectives

The use of internet-based questionnaires to collect information from older adults is not well established. This systematic literature review of studies using online questionnaires in older adult populations aims to 1. describe methodologic approaches to population targeting and sampling and 2. summarize limitations of Internet-based questionnaires in geriatric populations.

Design, Setting, Participants

We identified English language articles using search terms for geriatric, age 65 and over, Internet survey, online survey, Internet questionnaire, and online questionnaire in PubMed and EBSCO host between 1984 and July 2012. Inclusion criteria were: study population mean age ≥65 years old and use of an online questionnaire for research. Review of 336 abstracts yielded 14 articles for full review by 2 investigators; 11 articles met inclusion criteria.

Measurements

Articles were extracted for study design and setting, patient characteristics, recruitment strategy, country, and study limitations.

Results

Eleven (11) articles were published after 2001. Studies had populations with a mean age of 65 to 78 years, included descriptive and analytical designs, and were conducted in the United States, Australia, and Japan. Recruiting methods varied widely from paper fliers and personal emails to use of consumer marketing panels. Investigator-reported study limitations included the use of small convenience samples and limited generalizability.

Conclusion

Online questionnaires are a feasible method of surveying older adults in some geographic regions and for some subsets of older adults, but limited Internet access constrains recruiting methods and often limits study generalizability.

INTRODUCTION

The Internet is a powerful and increasingly commonplace platform that can be used to conduct survey research. An advantage of using the Internet for survey research is the reduction of the costs and burdens associated with mailed and in-person questionnaires.^1,2 Another advantage is that the Internet can access populations in real-time, and can reach segments of the population traditionally difficult to contact such as home-bound older adults.³

Online questionnaires have been successful in research targeting defined groups of individuals who are members of organizations or institutional structures such as medical students⁴ and health professionals.⁵ This method has also been used to reach populations such as adults with depression⁶ and prostate cancer.⁷ However, despite increasing Internet use and availability in the general population, elderly population, and in nursing homes,^8,9 it is not known how extensively the Internet has been used in studies focusing on older adults.

To address this gap, we conducted a systematic literature review of studies using online questionnaires in older adult populations in order to 1. describe methodologic approaches to population targeting and sampling among older adults and 2. summarize limitations of Internet-based questionnaires in geriatric populations. The findings of this study will help inform future efforts to use Internet surveys as a tool in geriatric research.

METHODS

Inclusion criteria

Types of studies and participants

The initial search strategy sought to identify studies that used the Internet to administer a questionnaire to a population of older adults. Inclusion criteria included: (1) self-administered, online questionnaire use as part of the study method; (2) study population mean age of 65 years or older; and (3) publication in English. We excluded publications that were not peer reviewed, had a study population whose mean age was less than 65 years old, or described studies where the online questionnaire tool was used in-person along with study personnel.

Information Sources and Search Strategies

The literature search was conducted using PubMed (1966 to June 2012) and EBSCO host (1984 to June 28, 2012). The search query used for PubMed was: (geriatric OR aged, 65 and over) AND (Internet survey OR online survey OR Internet questionnaire OR online questionnaire).

Study Selection

Abstracts were independently reviewed by two members of the research team (MR, JT). The full-text articles of abstracts meeting inclusion criteria were independently reviewed by the same two research team members to confirm eligibility for inclusion. Reference lists of included articles were reviewed to identify additional relevant articles. Conflicts were resolved through discussion.

Data Extraction and Synthesis

One author (MR) extracted the following data from the included studies: study population, mean age of participants, number of subjects, country, study aim, sampling and targeting methods used, outcome measures and study limitations.

RESULTS

Study Selection

A total of 11 studies met inclusion criteria for final inclusion.^10–20 The initial search using PubMed returned 336 articles. Of these, 322 were removed after reviewing the abstracts because they failed to meet inclusion criteria. After abstract review, 14 full-text articles were reviewed to determine final inclusion by 2 investigators (MR, JT). No unpublished relevant studies were obtained.(Figure 1)

Figure 1.

Article Selection Process

Description of studies

The reviewed studies are summarized in Table 1. The first study was published in 2002¹¹ and the most recent was published in 2012.¹² The studies included a total of 8,656 participants, with sample sizes ranging from 10 to 3,358 participants. Participants mostly included community dwelling elders (n= 9)^{10–13,16–20}, but also included older adults from a retirement community (n= 1)¹⁵ and an elder care facility (n=1).¹⁴

Table 1.

Study Characteristics

Author	Year	Aim	Population	Mean Agea (years)	N	Response Rate	Country	Recruitment Strategy	Online Measurements	Online Limitationsb	Study Limitationsc
Chang10	2004	To describe the process and feasibility of having elders respond to standardized questionnaires in an intervention program for caregivers of persons with dementia.	Spouse caregivers of persons with dementia	68	21	52.4% (11/21) completed online questionnaire	USA	Investigators met in-person with senior centers and dementia support groups	Caregiver demographics, emotional status, caregiver burden, social support	Many respondents did not use Internet to submit surveys, but submitted paper surveys	Limited generalizability
Clark11	2002	(1) To describe the experiences of online questionnaire completing and participation in an online structured interview and (2) to determine if a chat room interview can yield rich qualitative data.	Internet users aged 65 years and older living alone	67.8	10	100% (10/10) completed online questionnaire 90% (9/10) completed online interview	USA	Investigator visited senior websites, joined online chat room discussion, and sent email requests to personal friends and family members	Qualitative description of participants’ experiences in completing the online questionnaire and participating in a chat room interview.	Participants limited to those persons who could be found via personal contacts and chat rooms.
DiBonaventura12	2012	To assess the impact of chronic obstructive pulmonary disease (COPD) on health related quality of life, resource utilization, work productivity and activity impairment compared to controls without COPD.	Employed adults aged 65 years or older, drawn from the U.S. National Health and Wellness Internet Survey (NHWS).	54% 65–69 yr 17% 70–74yr 21% 75–79yr 8% ≥80yr	3,358 (297 with COPD and 3,061 controls without COPD)	Response rate not provided for the sub-sample of respondents aged 65 or older with COPD. Only provided for the parent sample of adults aged 18 or older (n=92,759 of 501,239 persons contacted)	USA	Sample drawn from annual, cross-sectional web-based consumer panel comprised of adults aged 18 years or older. Panel members were recruited through opt-in emails, co-registration with panel (NHWS) partners, e-newsletter campaigns, online banner placements.	COPD diagnosis, demographics health history, health related quality of life, work productivity, activity impairment, and healthcare resource use.		Sample not representative of all older workers in USA with COPD. Causal inference cannot be determined. Recall bias may be source of error.
Harada13	2011	To examine the relationship between strength-training behavior and environmental factors	65–75 year old males and females, living in Japan	68.2±2.8	293	97.7% (293/300)	Japan	Japanese Internet research service organization randomly selected subjects to invite via e-mail	Demographics, regular strength training behavior, environmental factors using the International Physical Activity Questionnaire,		(1) Small sample size; (2) Limited generalizability and selection bias.
Latimer Hill14	2007	To determine the usefulness of Internet-based surveys for research in older adults to study the association between sleep and falls. Secondary aim to compare traditional (in person) questionnaire response to Internet-based questionnaires.	Older users of the Greypath Website, and a comparison group of older people living in residential aged care facilities who completed in-person questionnaires.	Online respondents: 70±5 In-person respondents: 81±8	150	Not reported. Sample included 150 surveys used (number received not reported)	Australia	Advertising and seminars to residential aged care facilities	Demographics, quality of sleep, falls, injury, medication, daily exercise, and use of walking aids and spectacles.	Unable to perform physical measures (e.g. visual acuity, blood pressure monitoring, gait analysis) which were performed on in-person respondents	Sampling bias from online participants
Nahm15	2004	To examine the psychometric aspects of an instrument to measure computer-mediated social support (defined as a person’s basic social needs gratified through online interactions with others)	Adults 62 years or older, able to use the Internet/email, residing in a selected retirement community.	78±5	38 first survey; 35 second survey	100% (38/38) first survey. 92.1% (35/38) second survey	USA	In-person recruitment at computer clubs within retirement communities	(1) Modified Lubben Social Network Scale (2) Modified Medical Outcomes Study Social Support Survey	2 of 47 interested subjects were unable to use the Internet and email and another 2 did not have email	Use of small, white and well-educated, convenience sample
Oladimeji16	2008	To quantify the association between self-reported adverse drug events (ADEs) and risk factors such as the number of pharmacies used by patients and their concern and necessity beliefs about medicines.	Patients from the Harris Interactive Online Panel aged 65 years or older.	72.9 ± 5.7	1220	Not reported. Sample included 1220 surveys (number received not reported)	USA	Recruitment within a confidential, marketing-firm maintained panel of individuals who have opted to be invited to participate in telephone and/or online surveys.	Demographics, self-reported ADEs, self-rated health, number of medications, sum of symptoms experienced, concern and necessity beliefs about medicines, number of pharmacies, and whether medications were skipped to save money.	Unable to validate outcomes (ADEs)	Generalizability - subjects were a convenience sample that included few minorities and low-income individuals
Russell17	2008	Exploratory, electronic survey examining the relationship between Internet communication and access to social capital (defined as “resources that emerge from one’s social ties”) in later life.	Internet users aged 55 years or older and not employed full time	Not reported	154	Not reported	Australia	Purposive recruiting via Internet search engines, online interest groups for older people, and online leisure groups.	Social capital (measured by self-reported relationships with informal networks of family and friends, and formal ties to voluntary associations); Duke social support index		Participants not representative of all older adults in Australia
Sekeres18	2011	To assess patient perceptions regarding their disease, treatment, and prognosis.	Patients with myelodysplastic syndrome (MDS) registered in a disease foundation database who had provided an email address, working knowledge of English, computer proficiency, and Internet access	65 (median)	358	11.5% (361/3131), including 358 usable questionnaires	USA	Email invitation to patients in disease registry database who provided an email address.	Patient perceptions regarding disease severity, prognosis, and treatment outcomes	Validity of diagnosis was reviewed by checking the internal consistency of the disease-specific responses. Response bias – respondents able to participate were less impaired.	Reliance on self-reported data. Generalizability – respondents had ‘lower risk’ MDS, and had received more disease-modifying therapies than general MDS population.
Shiyanbola19	2012	To quantify the use of inappropriate medications among older adult outpatients and to describe the association between inappropriate medication use and concerns and beliefs about medicines, and self-reported ADEs.	Online panel from Harris Interactive Online Panel, who were ≥65 years of age, residents of the USA, and enrolled in the Medicare health plan.	72.7	874	85.4% (874/1024)	USA	Recruitment of a confidential, marketing-firm maintained panel of individuals who have opted to be invited to participate in telephone and/or online surveys.	Self-reported ADEs, inappropriately prescribed medications and concerns/beliefs about medications	Population was highly educated, affecting generalizability and measurement of ADEs.	Use of secondary data restricted and minimized the validity of measures such as the Beers criteria and ACOVE quality indicators.
Unni20	2010	To determine whether beliefs in medicines are associated with forgetfulness and carelessness in taking medications.	Medicare enrollees aged 65 and older and who were Internet users, administered by Harris Interactive.	72.3 baseline survey; 73.2 follow-up survey	1061 in baseline survey; 891 in follow-up survey	87% (1061/1220) baseline 87% (891/1024) follow-up	USA	Recruitment of a confidential, marketing-firm maintained panel of individuals who have opted to be invited to participate in telephone and/or online surveys.	Demographics, prescription drug insurance, prescription medications used, healthcare utilization, medication adherence, symptoms experienced, beliefs in medications, self-reported adverse drug events and self-reported medication adherence.		Generalizability of study due to use of convenience sample and limited to Internet users. Self-reported scale was used to measure non-adherence and was not verified.

^aUnless otherwise specified.

^bOnline limitations defined as limitations associated with using the Internet as the study platform.

^cStudy Limitations defined as methodologic limitations of the study not specific to using the Internet as the study platform.

Abbreviations: ACOVE, Assessing Care of Vulnerable Elders Project

The mean age of the study populations ranged from 65 to 78 years. While some studies included participants younger than age 65,^15,17 all study populations had a majority of participants aged 65 or older. Geographically, studies were conducted in the United States (n=8),^{10–12,15,16,18–20} Australia (n=2),^14,17 and Japan (n= 1).¹³ Some studies recruited participants that had a specific disease diagnosis (including dementia,¹⁰ chronic obstructive pulmonary disease [COPD],¹² and myelodysplastic syndrome [MDS]¹⁸), specific insurance plans (Medicare health plan)²⁰ or residence in a retirement community,¹⁵ while others targeted adults without regard to a clinical diagnosis, insurance, or residence. Most surveys used self-reported questionnaires (n=10),^11–20 but the one study that included patients with dementia used family caregivers as proxy respondents for the patients with dementia and self-report for the caregivers themselves.¹⁰

Study designs included were cross-sectional (n=9)^{10–14,16–19} and follow-up surveys (n=2)^15,20 that were both descriptive (n=3)^10,11,15 and analytical (n=8).^{12–14,16–20} One study measured the test-retest reliability of an instrument and reported that 92% of participants completed both initial and 1-week follow-up questionnaires.¹⁵ One study also included qualitative interviews.¹¹ Studies addressed three broad topic areas: medication use^16,19,20 Internet use^{10,11,14,15,17} and attitudes, perceptions, quality of life relating to specific diseases and functional states.^12,13,18

Recruiting and Sampling Strategies

In five of eleven studies, investigators retained either commercial marketing firms^12,16,19,20 or an internet research group¹³ to conduct their recruiting. Three of the largest studies recruited subjects from the Harris Interactive Online Panel.^16,19,20 Harris Interactive is a market research firm that maintains a panel of participants using a variety of communication tools, including the Internet, to reach study participants. Institutions or investigators can hire Harris services to collect data, analyze and interpret results. Kantar Health, a similar firm but with a healthcare focus, conducts the National Health and Wellness Survey (NHWS) that was used one of the reviewed studies.¹² (http://www.kantarhealth.com) The NHWS is an annual, cross-sectional study of adults, aged 18 years or older, that uses a self-administered, Internet-based questionnaire. The overall NHWS sample is drawn using a stratified random sampling procedure based on gender, age, and race/ethnicity in order for the study sample to be representative of the demographic composition of the target adult population. This firm maintains a web-based consumer panel that can draw from the US, Europe, Asia, and Latin America. For the study included in this review, investigators performed a subset analysis of a sample of 75,000 questionnaires of adults 18 and older in the U.S., and focused on the subset who were adults aged 65 years or older (n=3,358).¹² One smaller study conducted by Harada et al.¹³ in Japan used an “Internet research-services organization” to recruit subject via an email that contained the URL link to the survey.

The remaining studies were fielded by individual or small investigator teams (n=6)and used a variety of different recruitment strategies. One investigator team targeted the members of an online member of a large website called the Greypath Website (www.greypath.com.au).¹⁴ This Australian-based website that provides a forum for older people and generates more than 1,000,000 hits each month. The website is intended for the exclusive use of seniors and includes chat rooms and other online resources that served as the recruitment tools for the online survey.¹⁴ The remaining studies used a combination of personal visits, emails, paper flyers in computer clubs, and internet search engines for recruiting. Chang¹⁰ recruited caregivers of patients with dementia through personal visits to senior centers and support groups. Clark recruited study participants¹¹ through investigator visits to senior websites, online chat room discussions, and by personal email requests to the investigator’s personal friends and family members. Nahm et al.¹⁵ recruited participants through a retirement community that had two computer clubs with about 100 members; flyers explaining the study were posted on facility bulletin boards, and emails were sent to interested participants with a URL link to the online survey. Russell et al.¹⁷ recruited using Internet search engines and online interest groups for older adults. Sekeres et al.¹⁸ recruited patients with MDS who were registered in, and provided email addresses to, the A plastic Anemia & MDS International Foundation database. Invitations to participate were sent via email with a URL link to the survey within the email, and offered an incentive for participation (a U.S. $15 Amazon.com gift card to the first 300 respondents).

Reported Limitations

The authors of the studies in this review listed several limitations for their work that can be categorized as those unique to Internet surveys, those exacerbated by the Internet platform, and those common to self-reported questionnaires. Internet-specific challenges include technical issues such as Internet provider service access. The most common technical challenge appeared to be the ability to use the Internet. To address this constraint, some studies specifically included in-person educational interventions to recruit and teach older adults about how to access and use the Internet.^10,15

Whether respondents who complete Internet questionnaires are similar to non-Internet respondents is also a concern specific to Internet-based surveys. Sekeres et al.¹⁸ compared their online sample to an in-person version of their questionnaire, and found that the distribution of baseline characteristics and disease severity of both samples comparable. While Latimer Hill et al.¹⁴ also reported that outcomes were comparable for data from their Internet and in-person samples, they reported that the Internet sample was significantly younger than the in-person sample (75 years ± 5 versus 81 years ± 8).

While generalizability is a common concern for many studies regardless of Internet use, lack of generalizability appears to be exacerbated by use of the Internet for survey administration. Lack of generalizability was the most commonly reported limitation in the reviewed studies. Most studies were unable to assess, or did not report, how representative their sample was of their target population. Russell et al¹⁷ reported that their participants were mainly English speaking, married homeowners from higher socioeconomic status (SES) backgrounds. In some studies, investigators reported that sampling bias (i.e. skewing of the population toward higher SES respondents) was felt to contribute to response bias and outcome measurement error. For example, in the ADE studies in which the respondent populations tended to be more highly educated, investigators felt that the subjects were more likely to report ADEs because they were more aware of, and better able to identify, ADE symptoms.^17,19

Other author-reported limitations from the reviewed studies were germane to any self-reported questionnaire. For example, submission of useable questionnaires is a common concern. One study examined this and found that 150 of 152 Internet questionnaires were usable, rejecting only 2 as nonsensical.¹⁴ Other common limitations include the use of small convenience samples, the possibility of recall bias, and inability to validate self-reported responses. One investigator who conducted physical measurements (such as visual acuity tests) in their in-person study sample noted the inability to perform the same tests in their Internet survey.¹⁴

DISCUSSION

Our systematic review revealed a small number (n=11) of studies. The majority of the articles were published on or after 2007, during a period when the percent of adults over 65 years with access to the Internet increased from approximately 33% to just over 50%.⁹ The major issues with using the Internet platform for questionnaires in aging research appear to be related to recruitment and generalizability.

While some of the studies in this review had large sample sizes of 800 or more, all of these studies used data from web-based panels maintained by one of two commercial marketing firms.^12,16,19,20 These studies that used marketing firm samples tended to yield high-income, educated respondents from industrialized nations. Surveys administered by individual or small investigator teams tended to have smaller sample sizes.^{10,11,13,15–17} These studies used in-person or individual email invitations and typically captured convenience samples that could not be generalized to other populations.

All of the studies reviewed had sampling and generalizability issues that constrained the rigor of the study findings.² Because not all older adults have Internet access, not all the potential subjects had an equal opportunity to be sampled. This introduces error into variable measurement because the study sample does not properly represent the underlying population. Further sampling errors are introduced when studies use disease registries or elder care facility populations to target subjects, because these sub-samples are often not generalizable to larger populations. These measurement and sampling errors are felt to contribute to response bias, such as was illustrated in the ADE studies where investigators felt that their highly-educated subjects were more likely to report ADEs since they were more aware of ADE symptoms.^16,19

In addition, high levels of poverty among older adults²⁴ limit access to up-to-date, innovative technology, such as high-speed Internet or video chatting. Having older technology leads to hardware and software incompatibilities that contribute to nonresponse error because browser incompatibility increases the likelihood that respondents are not able to complete questionnaires.²

Physical issues for older adults also present a challenge. For example, questionnaire item placement, presentation, and typeface size may affect response since many older adults have visual impairment issues. Slow typing also necessitates patience for study investigators who use the Internet for qualitative research via chat rooms.¹¹ While some believe that computer engagement may help some patients maintain cognitive acuity and intellectual engagement,²⁵ the prevalence of cognitive and physical limitations also limit the usefulness of Internet-based surveys by the oldest-old and frailest elders.

It is important to consider our study limitations when drawing conclusions from this review. Studies in this review were conducted in only a few industrialized nations, and many studies were conducted by large marketing firms. None of the included studies were from developing nations or included highly frail or poor populations. Despite theoretical penetration into nursing homes, ^26,27 none of the studies targeted this population. It appears that the Internet can be used primarily with more independently functioning elders in the absence of a proxy or caregiver respondent.

In summary, older adults are becoming a growing presence on the Internet in the U.S., with three-quarters of elderly users going online every day, and one-quarter using social networking sites.²⁸ While increasing access to the Internet by older adults holds promise for investigators hoping to use this platform for survey-based descriptive and analytic studies, some segments of the elderly population are being left behind, such as frail elders, poor elders, and elders residing in developing nations. Successful, large-scale recruiting of diverse elderly populations using Internet search engines or online advertising has yet to be demonstrated; this remains an important area for further investigation. At the current time, the usefulness of the Internet as a research platform appears to depend on individual study aims, characteristics of the target samples, and Internet access in the countries of interest.