Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2013 Sep 1.
Published in final edited form as: J Health Commun. 2012 Sep;17(8):990–1000. doi: 10.1080/10810730.2012.700999

Using Collaborative Web Technology to Construct the Health Information National Trends Survey (HINTS)

RICHARD P MOSER 1, ELLEN BURKE BECKJORD 2, LILA J FINNEY RUTTEN 3, KELLY BLAKE 4, BRADFORD W HESSE 4
PMCID: PMC3490625  NIHMSID: NIHMS411852  PMID: 23020764

Abstract

Scientists are taking advantage of web-based technology to work in new collaborative environments, a phenomenon known as Science 2.0. The National Cancer Institute (NCI) created a web-based tool called HINTS-GEM that allows a diverse group of stakeholders to collaborate in a virtual environment by providing input on content for the Health Information National Trends Survey (HINTS). This involved stakeholders providing new suggested content and commenting and rating on existing content. HINTS is a nationally-representative survey of the US non-institutionalized adult population (see Finney Rutten et al. [this journal] for more information about the HINTS program). This paper describes the conceptual development of HINTS-GEM and provides results of its use by stakeholders in creating an improved survey instrument.

Open data, data harmonization, and empowering communities are the basic tenets underlying the National Cancer Institute's (NCI) Health Information National Trends Survey (HINTS). In this report, we describe a web-based tool called HINTS-GEM, which was created to leverage the collective intelligence of a large group of researchers and other stakeholders committed to using national surveillance as a means to improve cancer prevention and control.

With the rapid increase in the use of the Internet and its capabilities as a communication tool, scientists are taking advantage of collaborative web technology to accelerate discovery in a new participative environment, a phenomenon referred to as Science 2.0 (Shneiderman, 2008). This builds off the idea of Web 2.0—defined by technologies such as wikis, blogs and other means for communicating information and collaborating with other users (e.g., seeing comments and ratings by users of Amazon.com) with specific application to the scientific arena. These technologies enable a new collaborative environment of openness, transparency, and crowd-sourcing (i.e., wisdom of crowds; Surowiescki, 2004). There is also growing evidence that efforts within this new “networked” scientific arena can lead to improved outcomes when input is solicited from a diverse group of dedicated individuals with varying levels of expertise (Nielsen, 2012). Likewise, there is a recent push within the Federal government toward openness and transparency, exemplified by the Open Government Initiative (http://www.whitehouse.gov/open/) which seeks to “…establish a system of transparency, public participation, and collaboration.” This movement extends to Federal data resources through sites such as Data.gov and Healthdata.gov which provide—free to the public—a large number of datasets and data tools.

This movement toward openness and transparency within the health domain aims to increase knowledge and engage and empower communities (broadly defined) to improve their health. The Community Health Data Initiative (CHDI; http://www.hhs.gov/open/datasets/communityhealthdata.html) is a Department of Health and Human Services program under the Open Government initiative that seeks to empower consumers and communities to get more value out of the myriad sets of health data that exist in the U.S. This initiative provides data and tools in user-friendly formats to increase disease prevention, health promotion, health care quality and performance. Now, more than ever, the public has access to quality data to inform their health decisions and empower their communities around wellness.

In addition to efforts to promote and enable use of existing data through initiatives like the CHDI, communities are increasingly encouraged to collect their own data and to use existing measures to ensure that their results can be compared with existing data. In keeping with this idea, a recent Institute of Medicine report on the role of measurement in the Federal surveillance system states that communities need data and indicators of health to make important decisions; however, the report cautions against creating a plethora of indicators that could create more confusion than clarity (For the Public's Health: The Role of Measurement in Action and Accountability; http://www.nap.edu/catalog/13005.html). One of the seven recommendations to come from this report states that the Department and Health and Human Services should create: a) a core, standardized set of indicators that can be used to assess the health of communities; and b) a core, standardized set of health-outcome indicators for national, state, and local use. One effort to address the need for standardized indicators is the recently created Health Indicators Warehouse (http://www.healthindicators.gov/) which provides a large number of indicators organized by topic, geography, or initiative, with the goal of providing outcomes that can be harmonized and directly compared across different levels. Thus, it is important not only to make data publicly available to empower communities, but also to encourage the use of shared indicators of health outcomes to maximize the comparability and utility of data collection efforts.

HINTS and HINTS-GEM

HINTS-GEM is an extension of another NCI-created tool – the Grid-Enabled Measures (GEM) portal – which contains behavioral and social science and other types of measures organized by theoretical constructs, and is designed to enable researchers to use common measures with the goal of exchanging harmonized data. Further detail on GEM is provided below. Engaging a variety of researchers in the process of building a HINTS instrument is not new; collaboration has been central to the HINTS program since its inception, and HINTS data have always been made available to the public. However, for the fourth iteration of HINTS (HINTS 4), the NCI made a commitment to build upon the Open Government Initiative and activities like the Community Data Health Initiative to capitalize upon technology-mediated social participation to enable a transparent and participatory process for the development of survey content, with the goal of increasing the breadth and depth of input from a community of researchers and health advocates.

The Health Information National Trends Survey (HINTS) Program

HINTS is a national health communication survey conducted by the NCI, which has the vital mission of developing and implementing programs that prevent and reduce the incidence of cancer. HINTS was designed to support the mission of the NCI's Health Communication and Informatics Research Branch (HCIRB) by providing a means to systematically evaluate the public's knowledge, attitudes, and behaviors relevant to health communication and cancer prevention and control, which have not adequately been studied through other national data collection efforts prior to HINTS. See the Rutten et al. article in this issue for more detailed information about the HINTS program and HINTS 4.

Purpose of HINTS

The HINTS framework takes into account that the successful development and communication of public messages about cancer prevention, detection, diagnosis, treatment, and survivorship require comprehensive understanding of individuals' access to cancer related information; perceived trust in information sources; cancer- and health-related knowledge; and in-depth knowledge of the factors that facilitate or hinder communication. HINTS aims to assess the public's use of health information in an environment of rapidly changing communication and informatics options, and allows the intramural and extramural research community access to the data to conduct research into the relationships between health information, knowledge, attitudes, and behaviors. Prominent constructs and resultant item development for HINTS were informed by the emerging theories of health communication (Glanz, Lewis, & Rimer, 1997), media usage (Viswanath & Finnegan, 1996), risk information processing (Croyle & Lerman, 1999; Fischhoff, Bostrom, & Quadrel, 1993), diffusion of innovations (Rogers, 1995) and behavior change (Weinstein, 1993). A more detailed discussion of the conceptual framework underlying the scope of HINTS content is published elsewhere (Nelson et al., 2004).

HINTS-GEM

For the first three HINTS data collection efforts, instrument development relied upon a collaborative process involving NCI staff and colleagues external to the government, from academic and research settings. HINTS 4 involves four data collection cycles comprised of both core/trended items and special topic modules which will be administered over the course of three years, The collaborative process for instrument development for HINTS 4 has expanded to include not just interpersonal interaction with NCI staff and external experts, but also technology-mediated participation made possible by the Grid Enabled Measures (GEM) portal.

Rationale and Goals

The guiding framework for HINTS identifies multiple factors to help understand the role of health communication in cancer prevention and control (Nelson, et al. 2004). Given the variety of perspectives that must be taken into account for successful instrument development, the HINTS program has always sought input for the HINTS instruments from a community of researchers. These researchers represent a number of disciplines including behavioral science, clinical psychology, social psychology, communication science, and health behavior. For HINTS 4, the NCI asked researchers who wanted to “help build a better HINTS” to use HINTS-GEM as a forum for their input.

The NCI had four goals in building HINTS-GEM:

  1. Increase the efficiency, organization, transparency, and success of building an item pool for the HINTS 4 instruments;

  2. Broaden participation in the instrument building process, both with respect to the number of participants and the substantive content represented by participants' areas of expertise;

  3. Enhance the degree to which researchers are engaged with the HINTS Program – e.g., using HINTS data or HINTS instruments in their own research – by increasing their opportunities to be a part of the HINTS 4 development from the very start; and

  4. Keep a community of researchers engaged and up-to-date with the HINTS 4 modular data collection process over its three-year administration period.

Design

HINTS-GEM copied the basic design and infrastructure of the NCI's Grid Enabled Measures Database (GEM; www.gem-beta.org). Briefly, GEM is a grid-enabled, interoperable website of behavioral and social science measures. In GEM, researchers have the opportunity to search, download, and provide feedback on measures as well as share data that result from use of the measures. In GEM, content is primarily divided into Constructs and Measures, where Constructs represent larger areas of inquiry (e.g., Cancer Information Seeking) and Measures are scales or single-items that assess a specific outcome (e.g., During your last search for cancer information, how concerned were you about the quality of the information?). Each Measure is assigned to one Construct, though each Construct may contain several measures (a “one-to-many” relationship). (See Figure 1.)

Figure 1.

Figure 1

Open in a new tab

Measures page. (Color figure available online.)

HINTS-GEM development began in June, 2010. At that time, 81 Constructs appeared in GEM; these were imported into HINTS-GEM so that the site was initially populated with these 81 constructs. Next, HINTS-GEM was populated with all the items that had appeared in previous HINTS administrations (the 2003, 2005, and 2008 instruments). These 526 items were assigned to relevant Constructs by the HINTS Management Team. As needed, new Constructs were created.

The main functional capabilities of HINTS-GEM are contained within the Construct and Measures tabs (see Figures 1 and 2). The Constructs tab shows a data spreadsheet with four columns: The Construct name, its definition, its status, and the number of comments attached to the Construct. A Construct's status could have one of two values: Imported from GEM (if the Construct was pre-populated into HINTS-GEM via GEM) or Under Consideration (if the Construct was created by a HINTS-GEM user). (See Figure 2.)

Figure 2.

Figure 2

Open in a new tab

Constructs page. (Color figure available online.)

The Measures (i.e., Items) tab shows a data spreadsheet with six columns: The Construct associated with the Measure; the Measure text; the HINTS data set(s) in which the Measure previously appeared (a missing value for this column indicated the Measure was newly proposed for HINTS); the Measure's status; its source; and the number of comments attached to the Measure. A Measure's status could have one of three values: Recommended for Inclusion in HINTS 4; Recommended for Exclusion from HINTS 4; or Under Consideration. (See Figure 1.)

Functional capabilities

Five functional capabilities exist within HINTS-GEM:

  1. Users can contribute content to the dataset by adding new Constructs or Measures (i.e., items) for consideration by the user community. Meta-data (information that describes the Construct or Measure (e.g., definition, source) are required for both. When adding a new Construct, users are required to supply a definition but are encouraged to enter other information such as theoretical foundation and synonymous constructs. When adding a new Measure, users are required to assign the Measure a Construct (and are given the option to create a new Construct, if the one they are looking for does not exist, as part of the process); to specify the Measure's response option(s); and to indicate whether the Measure is a trend measure (i.e., has appeared on a previous iteration of HINTS); appears on another survey (and if so, which survey); is central to a theory of health behavior (and if so, which theory); or appears in the Cancer Data Standards Registry and Repository (caDSR)1. Finally, users are asked to provide a brief comment that explains their rationale for adding the Measure to HINTS-GEM.

  2. Users can comment on Constructs or Measures using free text.

  3. Users can rate Measures on a scale from one to five with five being the highest rating.

  4. Users can propose alternatives to Measures in lieu of creating a new Measure, if what they want to propose is substantively similar or related to a Measure that already appears in HINTS-GEM.

  5. Users can sort the interface by any column header and can search HINTS-GEM for specific content.

It is worth noting that the meta-data requirements for adding new Measures to HINTS-GEM were intended to encourage users to consider factors related to measure standardization and data harmonization. While users were free to add any Measure they chose to HINTS-GEM, by requiring meta-data on a Measure's trend potential, its appearance in another surveillance effort, its relationship to theory, and its status in the caDSR, the HINTS Program aimed to encourage HINTS-GEM users to propose new content for HINTS 4 commensurate with one (or more) of these features. The hypothesis was that doing so would solicit new content from HINTS-GEM users that met these requirements and would therefore lead to the HINTS 4 data having good potential for harmonization with other surveillance efforts at multiple levels (e.g., local, regional, national, and global).

Building a HINTS-GEM Community

Concurrent with the development of HINTS-GEM, the NCI began work on building a community of researchers to use the site. This work happened in two phases:

  1. Enlisting participation from HINTS “Champions”: Twenty-one HINTS Champions (i.e., individuals who had contributed to HINTS development in the past or who were known by NCI to be HINTS data users) from the extramural research community and internal to NCI were initially invited to be the first HINTS-GEM users in August, 2010. These Champions participated in an on-line HINTS-GEM orientation in September 2010. Champions were assigned content areas (i.e., Constructs) based on their areas of substantive expertise and/or content that they had helped to develop in previous HINTS instruments. Champions were charged with three tasks to complete by the end of October 2010. First, they were asked to review the Measures already contained within HINTS-GEM (i.e., Measures that had appeared in a previous iteration of the survey) and assign the appropriate status to each Measure. If a Champion wanted the Measure to be considered for HINTS 4, then they indicated a status of “Recommended for Inclusion in HINTS 4.” If they thought the Measure should be excluded from HINTS 4, they changed the status to “Recommended for exclusion from HINTS 4.” Finally, if they wanted a larger community to have input into the decision, they left the Measure's status as “Under Consideration.” Second, Champions were asked to populate HINTS-GEM with new Measures for consideration in HINTS 4. Finally, Champions helped to disseminate information about HINTS-GEM to a broader community of research using prepared email blasts and PowerPoint slides for use at conferences or in communication with their respective professional societies.

  2. Enlisting participation from general users: Concurrently, NCI prepared a larger HINTS-GEM promotion campaign for launch at the 2010 American Public Health Association (APHA) annual meeting which was held in early November, 2010 in Denver, Colorado. HINTS Program staff was available on-site during the meeting to demonstrate HINTS-GEM and to register new users to the site. HINTS-GEM Fact Sheets were available at the meeting; information about HINTS-GEM was disseminated via the HINTS website (http://hints.cancer.gov); and an email describing HINTS-GEM (and directing potential users to an on-line HINTS-GEM orientation) was sent to all email addresses on record with the HINTS Program. These email addresses represent individuals who had requested to download HINTS data in the past or who had reached out to the HINTS Program for another reason. General HINTS-GEM users had all the same functional capabilities as HINTS Champions except that general users were unable to change the status of Measures.

Periodic email announcements and HINTS-GEM News items were sent and posted to encourage continued participation in HINTS-GEM after the official launch to a broad community of researchers at APHA. The HINTS Program provided technical support to HINTS-GEM users as needed. Communication with the HINTS-GEM community followed a phased approach, first emphasizing adding measures to HINTS-GEM (November 2010–December 2010), then moving to commenting on measures (January 2011), and finally focusing on rating measures in HINTS-GEM (February 2011–March 2011). In March, 2011, all measures in HINTS-GEM with a status of “Recommended for Inclusion in HINTS 4” or “Under Consideration” were submitted to the Office of Management and Budget (OMB) —as required by all public surveys—as an “over-inclusive item pool.” This pool represents the group of items that researchers will select from as they work with the HINTS Program to build the HINTS 4 instruments for the four cycles of data collection. Final disposition for the full pool of items by HINTS status can be seen in Table 1.

Table 1.

Measure Disposition by HINTS Status

Measure Disposition Measures from Previous HINTS Iterations (n=526) Measures Newly Proposed for HINTS 4 (n=647)
Recommended for inclusion 37.6% 41.7%
Recommended for exclusion 36.5% 6.3%
Under consideration 25.9% 51.9%
Open in a new tab

Results

In total, there were 51 HINTS-GEM Champions and an additional 87 users who contributed to HINTS-GEM. Most users came from academia (52%) or government (30%), though the private sector (10%), advocacy groups (4%) and HMO/Medical Centers (4%) were also represented. Although users were required to register in order to participate (for tracking and accountability purposes), they were only asked for their name and affiliation so detailed information about the users is limited. It is very likely that many more people were involved peripherally, for example, by reviewing the materials or discussing with colleagues, but chose not to contribute directly to the website. These types of participants have been termed `lurkers' though that is a bit of a misnomer as they are contributing to the community but not in obvious ways. (Preece & Shneiderman, 2009).

HINTS-GEM was initially seeded with 81 Constructs from GEM and 526 measures from all three previous iterations of HINTS. By the end of the campaign, a total of four new constructs and 647 new measures had been proposed, resulting in a total of 85 Constructs and 1173 Measures in the HINTS-GEM database. The total number of measures (both existing and new) were spread across the constructs with several having a large number of measures (Tobacco Use= 130; Colorectal Cancer= 75, Use of Technology= 69, Health Information Seeking=60) and others having very few measures (for example, Belief in a Just World=1; Religiosity and Spirituality=1). A total of 60 alternative measures were proposed as potential replacements or alterations for existing measures.

Across all measures, the number of comments ranged from 0–8 with 167 (14%) having no comments and a majority (71%) having 1 or 2 comments. Regarding ratings, a large majority had 0 ratings (89%) and for those that were rated, most had only 1 related comment (9%). The ratings themselves tended to be negatively skewed such that 87% of measures with ratings had an average value of 4 or greater (range 1–5, with 5 being the `best' measure). In regards to the reasons for including a new measure, out of the 647 new measures proposed, the following results were seen: 1) This is a trend measure (4%); 2) This measure appears on another survey (19%); 3) This measure is central to a theory of health behavior (9%); and 4) This measure is designated in the Cancer Data Standards Registry and Repository (0%).

The first cycle of HINTS 4 data were collected from October, 2011 until February, 2012 with a total of 3,959 surveys completed using a postal frame and a self-report questionnaire that was returned by mail. There were a total of 199 items on the survey with 101 items (50.7%) being new items that have never been asked in previous HINTS surveys and the remaining items originating from previous iterations. Most of these latter items are considered “core” items that can be tracked over time to assess trends. The percentage of new items demonstrates the utility of interfaces like HINTS-GEM to facilitate a collaborative, open process for research where survey development can be a technology-mediated endeavor that results in new constructs and items being introduced and implemented.

The Future of Survey Design: Openness, Data Harmonization, and Empowering Communities

Now more than ever, researchers and survey methodologists have access to vast amounts of information and data—it truly is the era of Big Data (2008, Nature Magazine). However, a lack of resources to manage this data deluge—including computer processing power which is sorely lagging behind (King, 2011)—are hampering our ability to move science forward quickly and efficiently. We need better tools so that we can more fully take advantage of these data.

There is also a growing sense within the Federal government surveillance system that there is a pressing need to more fully utilize existing data resources and to be more efficient in data collection efforts. Conducting more surveys does not seem to be the answer; conducting better surveys in a systematic and coordinated fashion does. This means creating agreed-upon health indicators and outcomes that can be shared and used by others. If this can be accomplished more readily, the ability to compare across data collection systems will be enhanced (Institute of Medicine, 2010). It also means systematic planning across data collection systems to avoid duplication of efforts, or just as importantly, identify gaps that need to be filled. This can decrease costs, increase efficiency and allow researchers to mutually benefit from each others' work and build a cumulative science. Evidence of tools to enhance collaboration and harmonize data are already available but are primarily focused on enhancing smaller-scale research protocols, rather than on capitalizing upon population-level surveillance systems. These tools,including PhenX (consensus measures for Phenotypes and eXposures; https://www.phenx.org/), GEM (Grid-enabled Measures; www.gem-beta.org), the NIH Toolbox (http://www.nihtoolbox.org/default.aspx) and PROMIS (Patient Reported Outcome Measurement Information System; http://www.nihpromis.org/) share the overarching goal of encouraging use of common measures across the research community. The overall idea is that if researchers can agree a priori on which measures to use in their research, the ability to share resulting harmonized data and build a cumulative science increases.

HINTS-GEM was built to increase the HINTS Program's commitment to and enablement of measure sharing and data harmonization. The results presented here suggest that the NCI achieved success at several levels through use of HINTS-GEM. Not only did the number of researchers who engaged in the HINTS development process greatly increase over years past, but the amount of new content proposed, as well as consensus regarding existing HINTS content, increased as well. Additionally, the more than 100 HINTS-GEM users who engaged in the process of building the HINTS 4 item pool are now in a position to use the consensus-driven measures found in HINTS-GEM in their own research, thus allowing for harmonization between local and national surveillance efforts.

The HINTS Program has already engaged in this sort of partnership: in 2009, the NCI partnered with the University of Puerto Rico to field a HINTS survey in the U.S. territory of Puerto Rico. Because there was a conscious effort to reuse the same items from HINTS 2008, there now exists ways of making direct comparisons in outcomes between the two surveys and associated geographic areas. The development of similar partnerships is currently underway, and these future efforts will be able to make use of the HINTS-GEM infrastructure to increase the efficiency and effectiveness of these endeavors.

There are several next steps for using HINTS-GEM. The site will be used to solicit further input to build consensus around the items that are selected for the Cycle 2, 3, and 4 HINTS 4 instruments. HINTS-GEM will also be used to communicate with the HINTS community about final item selections so that researchers can field local HINTS data collections in concert with the national-level data collection if they so choose. Finally, when HINTS 4 data are collected, the data will be made publicly available on HINTS-GEM, with the opportunity for researchers to share their own local HINTS data collections via the site.

Conclusion

Technology is now enabling access to vast amounts of data and it also provides new ways of collaborating, conducting, and communicating science. This new paradigm, referred to as Science 2.0, has the capability of moving science forward in ways that we are just starting to understand. This paradigm was utilized by the HINTS program as it took collaborative science regarding survey content to a new level. The data from the HINTS-GEM experiment suggest that the results of this elevation in changing the process of collaboration will lead to better surveillance instruments, more actively engaged researchers, harmonized data across surveillance efforts and greater power to detect meaningful results that can be translated into policy and practice aimed at improving health and wellness.

Acknowledgments

Funding

This project has been funded in whole or in part with federal funds from the National Cancer Institute, National Institutes of Health, under Contract No. HHSN261200800001E. The content of this publication does not necessarily reflect the views or policies of the Department of Health and Human Services, nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government.

Footnotes

1 The caDSR is a tool associated with the cancer biomedical informatics grid (caBIG) and it creates and deploys common data elements to be used by the cancer research community. For more information see: https://cabig.nci.nih.gov/concepts/caDSR/).

References

  1. Croyle RT, Lerman C. Risk communication in genetic testing for cancer susceptibility. J Natl Cancer I Monographs No. 1999;25:59–66. doi: 10.1093/oxfordjournals.jncimonographs.a024210. [DOI] [PubMed] [Google Scholar]
  2. Fischoff B, Bostrom A, Quandrel MJ. Risk perception and communication. Ann Rev Publ Health. 1993;14:183–203. doi: 10.1146/annurev.pu.14.050193.001151. [DOI] [PubMed] [Google Scholar]
  3. Glanz K, Lewis M, Rimer BK, editors. Health behavior and health education: Theory, research and practice. Jossey-Bass; San Francisco: 1997. [Google Scholar]
  4. Institute of Medicine For the Public's Health: The Role of Measurement in Action and Accountability. 2010 http://www.iom.edu/Reports/2010/For-the-Publics-Health-The-Role-of-Measurement-in-Action-and-Accountability.aspx.
  5. King G. Ensuring the data-rich future of the social sciences. Science. 2011;331:719–721. doi: 10.1126/science.1197872. [DOI] [PubMed] [Google Scholar]
  6. Nature magazine 2008 http://www.nature.com/news/specials/bigdata/index.html.
  7. Nelson DE, Kreps GL, Hesse BW, Croyle RT, Willis G, Arora NK, Rimer BK, Viswanath KV, Weinstein N, Alden S. The Health Information National Trends Survey: Development, design, dissemination. J Health Commun. 2004;9:433–460. doi: 10.1080/10810730490504233. [DOI] [PubMed] [Google Scholar]
  8. Nielson M. The new era of networked science. Princeton University Press; Princeton: 2012. Reinventing discovery. [Google Scholar]
  9. Preece J, Scheiderman B. The reader-to-leader framework: Motivating technology-mediated social participation. AIS Transactions on Human-Computer Interaction. 2009;1:13–32. [Google Scholar]
  10. Rogers EM. Diffusion of Innovations. The Free Press; New York: 1995. [Google Scholar]
  11. Shneiderman B. Computer science. Science 2.0. Science. 319:1349–50. doi: 10.1126/science.1153539. [DOI] [PubMed] [Google Scholar]
  12. Surowiescki J. The wisdom of crowds: why the many are smarter than the few and how collective wisdom shapes business, economics, societies, and nations. Doubleday; New York: 2004. [Google Scholar]
  13. Viswanath K, Finnegan JR. The knowledge gap hypothesis: Twenty five years later. In: Burleson B, editor. Communication Yearbook 19. Sage Publications; Thousand Oaks: 1996. [Google Scholar]
  14. Weinstein ND. Testing four competing theories of health-protective behavior. Health Psychology. 1993;12:324–333. doi: 10.1037//0278-6133.12.4.324. [DOI] [PubMed] [Google Scholar]

RESOURCES