Abstract
Challenges arise in building the knowledge needed for evidence based practice partially because obtaining clinical research data is expensive and complicated, and many studies have small sample sizes. Combining data from several studies may have the advantage of increasing the impact of the findings, or expanding the population to which findings may be generalized. The use of common data elements will allow this combining and, in turn, create big data, which is an important approach that may accelerate knowledge development. This article discusses the philosophy of using common data elements across research studies and illustrates their use by the processes in a Developmental Center grant funded by the National Institutes of Health. The researchers identified a set of data elements and used them across several pilot studies. Issues that need to be considered in the adoption and implementation of common data elements across pilot studies include theoretical framework, purpose of the common measures, respondent burden, team work, managing large data sets, grant writing, and unintended consequences. We describe these challenges and solutions that can be implemented to manage them.
Keywords: common data elements, clinical research
Introduction
Research findings establish new knowledge, validate previous findings, expand findings to a new population, and help build a body of knowledge on which practice - in a practice discipline such as nursing - is based. Challenges arise in building the knowledge needed for evidence based practice partially because obtaining clinical research data is complicated, and many studies have small sample sizes. Combining data from several studies may have the advantage of increasing the impact of the findings, or expanding the population to which findings may be generalized. In addition, having standardized measures that can be shared will capitalize on the benefits of “big data” for enhancing scientific benefit. Big data is a broad term for any collection of data that is large and complex enough to become difficult to process.
Synthesizing findings across multiple studies is complex (Kim, Pressler, Jones, & Graves, 2008). The current manner in which data are collected to support knowledge generation can be a slow and expensive process (Boyd et al., 2011; Riley, Glasgow, Etheredge, & Abernethy, 2013). One way to increase the speed of accumulating data, incorporating findings, and reducing expense is for researchers to collect and report common data elements, which facilitates creating common databases (Riley, Glasgow, Etheredge, & Abernethy, 2013). The purpose of this article is to discuss the philosophy of using common data elements across research studies and illustrate their use by the processes used in a National Institute of Nursing Research (NINR) P20 Exploratory Center grant entitled Interdisciplinary Health Heart Center: Linking Rural Populations by Technology (NR011404). The P20 researchers identified a set of data elements and used them across several pilot studies. We will also describe challenges that arose and solutions that can be implemented to manage them.
Definitions of Common data elements
The National Institutes of Health (NIH) is among the groups advocating that researchers use common data elements in order to facilitate comparing and combining data across studies, including data elements derived from electronic health records. The NIH definition of common data elements (CDE) is “a data element that is common to multiple data sets across different studies” (http://www.nlm.nih.gov/cde) (National Institutes of Health, 2013).
When designing research to answer a particular question, researchers select key concepts that are important to the question. In most cases, other researchers have also investigated the concepts and, over time, used multiple measures and methods to assess concepts. Data generated from the various methods may be similar but not necessarily equivalent. In contrast, common data elements are generated from the same set of instruments used to consistently measure a set of concepts of interest to many researchers. Comparison of data across studies is more accurate and relevant when researchers are investigating questions using the same data elements and measures.
Common data elements
Several initiatives have been launched to create tools to collect common data. As a result, a variety of proposed sets of common data elements can be found on the web. An example is the Quality of Life in Neurological Disorders (Neuro-QOL); a set of self-report measures that assess health related quality of life of adults and children with neurological disorders. A collaborative, multisite group constructed these tools with a contract from the National Institute for Neurological Disorders and Stroke (NINDS). Measures, which include English and Spanish versions, are available for use without permission and at no charge, from their website (Northwestern University, 2013).
Another example is the PhenX Toolkit (Hamilton et al., 2011). To facilitate replication and validation across studies, RTI International (Research Triangle Park, North Carolina) and the National Human Genome Research Institute (Bethesda, Maryland) are collaborating on the consensus measures for Phenotypes and eXposures (PhenX) project. The goal of PhenX is to identify 15 high-priority, well-established, and broadly applicable measures for each of 21 research domains. PhenX measures are selected by working groups of domain experts using a consensus process that includes input from the scientific community. The selected measures are freely available to the scientific community via the PhenX Toolkit, thus providing the research community with a core set of high-quality, well-established, low-burden measures intended for use in large-scale genomic studies. The PhenX Toolkit website (https://www.phenxtoolkit.org/) release 5.8 contains 339 standard measures related to complex diseases, phenotypic traits and environmental exposures (RTI International, 2014). Use of PhenX measures facilitates combining data from a variety of studies, stimulating investigators to expand a study design beyond easily accessible sample. All Toolkit content is available to the public at no cost.
In addition to creating tools, others have worked to catalog tools. An example is the National Cancer Institute’s (NCI) Cancer Biomedical informatics Grid (caBIG). The purpose of this project, which was launched in August 2007, was to contend with various barriers to data exchange by addressing legal, regulatory, policy, proprietary, and contractual barriers. An assessment of the impact of caBIG (Board of Scientific Advisors Ad Hoc Working Group, 2011) found that the original goals were highly relevant to cancer research; however, caBIG was seen to have expanded beyond those goals to the implementation of an overly complex and ambitious collection of NCI-branded software tools. These tools have been fully adopted by only a few NCI-designated Cancer Centers, and tools from established commercial vendors have been found to be more useful. Although caBIG was retired, the project led to the development of a platform used to develop GEM, Grid-Enabled Measures Database, a dynamic web-based database for researchers (https://www.gem-beta.org/public/About.aspx?cat=5) (National Cancer Institute, 2012). The database was designed to allow users to collaborate in building consensus on the use of common elements and measures and to facilitate data sharing and harmonization. The database currently (November 4, 2014) has 891 measures of 343 concepts. Anyone can view the website, and all are invited to add and edit information, measures, and concepts.
The above examples illustrate a value in sharing common data elements and measures, as well as several processes that have been used to develop these common data elements. In addition, barriers to use of CDE are also illustrated, especially in the example of caBIG. Examples below are from our experience on a smaller, college level when our Center grant was under development and subsequently funded.
Example of common data elements in the Center grant
Process
While preparing the P20 grant application, College of Nursing faculty members were invited to submit competing proposals vying to be included as pilot studies. The core research interest was in promoting health of individuals who were at-risk for, or diagnosed with, cardiovascular disease and who were living in rural areas. Five researchers were selected by the PI and core directors as their proposals were well aligned with the overall research center goals. After selection, the core directors and pilot PIs met over eight weeks to discuss concepts of shared interest (see Figure 1 which illustrates the process of developing common data elements).
Figure 1.
Process of developing common data elements for building science: Illustrated with examples from our P20 funded Center
Note: This process assumes that principal investigators of the pilot study and pilot study projects have already been selected.
PI = Pilot Study Principal Investigator
Each week, proposed pilot study concepts were reviewed for inclusion in the set of common data elements and measures. In addition, several possible measures of the concept were reviewed. The group considered including concepts that were, theoretically, outcomes that would improve in response to the intervention. The group also discussed the value of standardizing the length of the interventions and the times at which follow-up data would be collected to measure intervention outcomes in each pilot study. The rationale for using standard timeframes was because patients after a cardiac event (coronary artery bypass surgery, percutaneous coronary intervention, or cardiac rehabilitation participation) typically see their symptoms improve/resolve over the first 6 weeks and cardiac rehabilitation starts within 1–2 weeks post-event and lasts 6–8 weeks depending on insurance coverage. Thus, most investigators have measured responses to capture change in symptom improvement or health outcomes at 3 and 6 weeks, and 3 and 6 months (Macken et al., 2014; Pragodpol & Ryan, 2013; Zimmerman et al., 2011). It is important to be able to compare data with prior research on a topic using the standard time points for follow-up data collection. This was also designed to reduce differences across the studies and improve the quality of cross study comparisons. Five common concepts were used by all pilot study PIs: demographics, rurality, quality of life, self-reported physical activity, and objective measured physical activity (see Table 1).
Table 1.
Examples of Final Common Data Elements Examined in Center Grant by Concept and Measure
| Concept | Measure (s) | Variables Constituting Common Data Elements |
|---|---|---|
| Demographic Characteristics of Sample | Demographic Questionnaire | Age, race, ethnicity, marital status, education, insurance, income, work status, job type |
| Rurality |
|
|
| Quality of Life | EQ-5D: Euroqol Quality of Life Scale |
|
| Physical Activity (Self-Report) | Behavioral Risk Factor Surveillance System |
|
| Physical Activity (Objective Measure) (PA) | Actigraph Accelerometer (Model GT3X) |
|
METs = Metabolic equivalents is a physiological measure expressing the energy cost of physical activities
Standard times for measuring outcomes is important with any chronic illness. Another example is with cancer, where symptoms change over the course of treatment and the disease, so measuring symptoms at times that they are expected to change, as well as before and after an intervention, is essential. Recent research examined symptoms over time after hematopoietic stem cell transplantation (HSCT) (Cohen et al., 2012). Since symptoms change over the course of the HSCT, collecting data at times that has been established in prior research when symptoms change is important.
After the concepts of interest were selected, decisions about common measurement tools were made (see Table 1): 1) Demographic Questionnaire, 2) Rurality measured by the Montana State University Rurality Index (Weinert & Boik, 1995), and Rural and Urban Commuting Area Codes (RUCA codes), 3) Euro-Quality of Life Scale, 4) Subjective measure of physical activity (items from the Behavioral Risk Factor Surveillance System), and 5) Objective tool to measure activity (accelerometer). Common data elements for measuring demographic characteristics allowed comparisons across similar groups of patients using demographic categories (different income or job type groups). The rural index and RUCA codes were used because of the difficulties in defining and categorizing rural individuals in relation to geography and allowed comparisons across individuals living in large vs. small or isolated rural areas. The quality of life measure was used (EQ-5D) because it is a standardized, generic measure of health status for evaluation of clinical and economic outcomes across studies. Finally, being able to compare physical activity (PA) outcomes across the five pilot studies was facilitated by including both self-reported and objective measures of physical activity.
Physical activity as an illustration
The center grant focused on improving heart health, and most pilot studies included interventions related to physical activity. Physical activity is one example but this can apply to the measurement of any concept. Measurement issues can impede the sharing of comparable data across studies. People understand what is meant by physical activity from a practical point of view. However, the term physical activity is not specific enough from the perspective of research and knowledge generation. Various aspects of physical activity that can be measured are duration, frequency, intensity, time, and any combinations of these indicators (see Table 1). Each of these aspects of activity may require a different measurement technique; however, using the same self-report or objective measure in all studies provides comparable data and decreases the chance of attempting to measure a given aspect with multiple different tools. That is to say, all these aspects can be measured by self-report, by actigraphy, or by other measures. If each is measured by a different tool, it will be confusing to compare across tools. This problem is seen frequently in reports of systematic reviews.
Issues to consider when using common data elements and measures
Issues that need to be considered in the adoption and implementation of common data elements across pilot studies include theoretical framework, purpose of the common measures, respondent burden, team work, managing large data sets, grant writing, and unintended consequences.
Theoretical framework
The first issue was whether to use a single or multiple theoretical perspectives across the studies. Our initial draft of the P20 grant application used a sole theoretical perspective with the rationale that this would provide a single, unifying and parsimonious framework. However, all pilot study PIs based their interventions on several different theoretical perspectives [i.e., Health Promotion Model, (Pender, Murdaugh, & Parsons, 2010), Social Cognitive Theory (Bandura, 1997), the Transtheoretical Model (Prochaska, DiClemente, & Norcross, 1992), and health effects of life transitions model (Kaiser, Kaiser, & Barry, 2009)].
We decided to use multiple theoretical perspectives because applying multiple theories simultaneously can increase the understanding and explaining of complex behavioral changes that patients undergo as they attempt lifestyle changes. Multiple theories may reveal additional theoretical mechanisms that explain how and why behaviors change more effectively than a single theory. Our goal was to obtain a better understanding of the dynamic and real-world competing explanations of the participants’ responses to the behavioral interventions being investigated. An interesting observation is that earlier NIH funded centers grants used a single conceptual framework while more recently funded centers have used multiple frameworks. However, this did create come issues with the selection of the common data elements to ensure that the measures were consistent with the theoretical framework.
Making the decision about the theoretical perspective early in the writing process is important. Ideally this decision would be made before the call for pilot studies is announced. Expectations would then be clear and the pilot grants would not need to be revised. Researchers whose projects do not fit with the direction selected can look for other funding opportunities.
Purpose of common measures
Communicating and sharing an understanding of the need for common measures is important. Researchers often select concepts and measures without considering common measures because they are focused on a single theoretical framework. Investigators who believe they were already measuring the relevant concepts might be reluctant to include common data elements. If a researcher has not participated in a center grant or as part of a large team’s research they may not be aware of the advantages of sharing results across studies, particularly the value of evaluating multiple theoretical perspectives simultaneously. The PIs may also not be aware of the concepts of interest to colleagues across their institution. Ongoing round table discussions of the concepts of interest to all researchers in a given institution is one method of obtaining conceptual consistency prior to grant submission. Exploration of conceptual consistency across programs of research can assist teams in identifying where common measures are and are not appropriate.
Another important factor to consider is whether the measure is related to an outcome measure or some other common measure in the study. In order to build science, a focus in specific areas should use common outcomes. Outcomes are the end result of care as well as the foundation of professional accountability. Findings in relation to outcomes also have the potential to be translated into powerful health-care decisions. Whether the outcome is a generic or a specific measure will depend on the overarching focus that unites the studies. If the focus is more general, common data elements may be appropriate for secondary outcomes or possible generic outcomes; however each study must carefully select primary outcomes that are specific to the unique intervention, the mechanistic action of those interventions, and the clinical population. A good example of this is the use of the Patient Reported Outcomes Measurement Information System (PROMIS) as common measure of health status for physical, mental, and social well–being. PROMIS measures can be used as primary or secondary outcomes. They have been tested and documented to have validity and reliability across many different populations. However, validity of other measures being used in clinical studies may be another issue. We examined the reliability and validity of the measures when we were deciding on them. Since patients in our studies had primarily cardiovascular problems, we needed to examine existing evidence of reliability and validity in the cardiovascular populations.
Another issue that relates to validity of study results is the problem of comparing findings across published studies when they have different measures of the same constructs. Systematic reviews have to contend with this issue. Balancing how one builds science in an area where different measures are used means researchers need to balance building big data in a center versus building science in the context of previously published studies in the area of inquiry. Researchers need to evaluate the sensitivity and specificity of measures of the same concepts compared to a gold standard if that exists, so some conclusions regarding the data can be made.
Changes in funding opportunities also support the concepts of common measures. For example, the American Heart Association has recently started funding mechanisms for network grants. Each of the 3 research proposals in the network has to have a common theme and clearly demonstrated synergy between the different research proposals: one addressing basic science, one in clinical science, and one in population science. Another example is NINR’s emphasis on Center grants collaborating with each other and attempting to have common data elements or measures across different studies and other funded centers.
Respondent burden
Another important concern with common measures is that they might increase respondent burden, especially in situations where multiple theoretical perspectives are being used. In situations where the selected frameworks do not share all concepts, concepts and measures may need to be added to each study to create consistency. Data must be collected for all selected concepts across all studies.
In our proposed studies, potential participants were already managing a chronic illness (heart failure) or recovering from a recent cardiac intervention (percutaneous coronary intervention or coronary artery bypass graft surgery). The PIs were concerned that adding even a few more questions to assess concepts from other frameworks would be too much for the subjects. The fear was that potential subjects would not volunteer or might withdraw early. However, while respondent burden is an important issue to consider, none of the studies in our center found respondent burden related to these measures to be an issue. Those who were not willing to complete the intervention were unwilling to wear the actigraph accelerometer or felt the intervention was too complex.
Team work
Working collaboratively enhanced group ideas and components of our research. Working together on the P20 helped investigators feel they were supported and valued members of a larger team. Our teams have grown to more closely represent colleagues in other disciplines where use of common concepts and measure is more customary. Discussions related to selection of common measures can assist in the research socialization process for early career faculty members and doctoral students.
Managing large Data Sets
There are benefits and certainly drawbacks to managing large data bases. Large or big data sets require resources to help manage the data, combine the coding dictionaries, merging data files to use for comparisons, and the continual maintenance of adding more data for ongoing studies. Innovation in findings and new scientific discoveries may be in the way we ask the questions that cannot be posed with smaller data sets. In addition, patient centered outcomes research can involve comparative effectiveness research, and asking what works best in what subpopulation of patients. Answering some of these questions cannot be done with small data sets.
Grant writing
Adding a concept and especially a new measurement tool to a study after the original design phase is time consuming and challenging. The new concepts and measures will need to be included throughout the application (i.e., aims, background, methods, and measures), generating a great number of revisions. In some cases, existing content may need to be condensed or even eliminated because of space limitations. In our Center, the Center Directors of the larger grant and the pilot study PIs worked together to write templates of the various grant sections for the pilot PIs to insert into their proposals. This approach reduced workload and facilitated IRB submissions and grant implementation. These templates have remained available and have served as the basis for additional grant applications, a process viewed as time saving by college investigators.
Unintended consequences of using CDE
While we have noted contributions of using CDE, unintended consequence also arise, as we discussed with respondent burden for example. Another consequence is that the use of CDE requires a greater time commitment to set up the study because of the additional meetings with other PIs that are needed. Another unintended consequence was that faculty might feel they were giving up important aspects of their research if they change measures. Balancing what is best for an individual study compared with what is best for a group of studies may be difficult.
Conclusions
There are two major implications from this manuscript. The first is the increasing requirement to use common data elements across studies. The second implication is summarizing different ways to implement use of common data elements across research studies.
Requirement to Use Common Data Elements
The future of science indicates that more and more groups (e.g., the NIH Common Fund, PROMIS program, etc.) are proposing or requiring that common measures and common data collection procedures be used in clinical research. Discussion of this change in national circles is occurring now. The expectation of the use of common measures should become the norm within colleges of nursing and a part of the culture of nursing research. Using common measure with different populations will enhance our ability to build science in measuring concepts across different populations and ultimately lead to more generalizability of results and comparisons across studies. Use of common data elements and measures would also provide advantages in comparative effectiveness studies and in integrative and systematic reviews. Comparing results across studies that have used similar measures would enhance our ability to disseminate and translate information into practice.
Facilitating the adoption of common measures can be a change in the research culture of an organization. This idealized culture requires that individuals balance developing unique expertise and being a member of a team. Successful groups obtain that balance by having individual members bring expertise and the team members bring supporting additional expertise that is needed for the study. Colleges of nursing can build these teams by making strategic hiring decisions to bring in faculty who share interests in the areas that the college has identified as the focus or foci of their work.
The conceptual basis for using common measures should be introduced to doctoral students and should be the basis of ongoing educational efforts in colleges of nursing and across other disciplines. Students and faculty members alike should be recruited with full knowledge of the concepts of interest and the tools being used to examine them. Colleges of nursing could then build their reputation on solving one or more key problems of the discipline, building the knowledge base using a more systematic approach.
Expecting students to select a research question based upon work of an existing team is common in basic sciences and may speed completion of dissertations. Student dissertation work might extend current questions into different populations, might consist of a secondary data analysis, or might address a follow-up question to a concluding study. Requiring students to use existing common measures will socialize them into the process of developing a body of research rather than conducting isolated studies with little chance to translate them into actual practice change.
An additional technique is to incorporate the use of common measures into review criteria for internal grants and manuscript review criteria. As experience increases with each common measure, the expertise of the community will increase. Knowledge of the literature, both of the psychometrics of the measure and of the research using that measure will accumulate.
With time, the purposes and value of common measures will become obvious. Researchers will have exposure to both the theoretical and practical advantages of this approach to research working within and across research teams. This approach will assist with the selection and socialization of doctoral students to working in teams, and the incorporation of them into an existing research team. We further recommend that students work with this team as part of the expectations for early research courses. With this approach students will be able to fulfill specific research functions as part of an actual team, based upon their existing skills.
Ways to Implement Common Data Elements
Expectations about using common data elements and measures in all studies where appropriate can be created using several approaches. One way to build these expectations when developing a research focus in a college of nursing is to provide presentations from both internal and external experts on topics to assist the research community to develop a shared understanding of common measures. Presentations should address the rationale for this approach as well as specifics related to implementation of common measures.
Presentations and/or demonstrations of data analyses across studies will provide another way to appreciate the power of common measures. These demonstrations will be a concrete representation of the value of this approach. The demonstration should allow time for attendees to ask questions and to begin to internalize this expectation. Specifically, attendees should be expected to identify where they may be using related concepts and engaging in conversation to determine if they have sufficient conceptual consistency to use the identified common measure.
When beginning to use common measures, we recommend a focus on common concepts. An easy first step is to identify a common measure for collection of demographic data. Selection of a core set of demographic concepts is much less controversial and thus may set the stage for selection of measures for complex psychological or physiological concepts related to motivation for behavior change. Colleges of nursing often have niche areas of research expertise. One way to build on and continue to develop those niche areas is by using common data elements to move the research forward more rapidly.
Work groups can be set up to address common concepts and measures. These conversations may be easier when the group is not facing an immediate grant deadline. By starting early, discussions of how to move toward common measures and/or national standards can occur in a less stressful environment. One or two people can maintain the literature related to each common measure. Written descriptions and the rationale for their selection can be drafted and used by all. In this way the use of common measures provides value to grant writers and may be seen as a way to decrease work by sharing resources. Actual data collection tools can be identified or developed once and used by all, again providing value to the research community within the organization. Those joining a research grant or project must share an understanding of the common measures to be used and the purpose for doing so.
In summary, the use of common measures is expanding across research arenas. This offers strength in building nursing knowledge for more immediate application to the clinical environment. Common measures are best addressed at the start of any initiative. Using common measure with different populations will enhance our ability to build science in measuring concepts across different populations and ultimately lead to more generalizability of results and comparisons across studies.
Highlights.
The use of common data elements (CDE) is discussed.
Using CDE may accelerate knowledge development
We described the use of common data elements in an NIH funded Center grant
Challenges and solutions to the use of CDE are described
Acknowledgments
Funding: This research was supported in part by Grant P20 NR011404 from the National Institute of Nursing Research, National Institute of Health (NINR, NIH).
Footnotes
Conflicts of interest: The authors have no funding or conflicts of interest to disclose.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Contributor Information
Marlene Z. Cohen, Email: mzcohen@unmc.edu, Professor and Kenneth E. Morehead Endowed Chair in Nursing Associate Dean for Research, University of Nebraska Medical Center, College of Nursing, 985330 Nebraska Medical Center, Omaha, NE, USA.
Cheryl Bagley Thompson, Assistant Vice Chancellor Academic Affairs/Student Affairs, Associate Professor College of Nursing, University of Nebraska Medical Center, Omaha, NE, USA.
Bernice Yates, Professor, College of Nursing, University of Nebraska Medical Center, Omaha, NE.
Lani Zimmerman, Professor, College of Nursing, University of Nebraska Medical Center, Omaha, NE Lincoln, NE 68588-0220.
Carol H. Pullen, Professor, College of Nursing, University of Nebraska Medical Center, Omaha, NE
References
- Bandura A. The exercise of control. New York: Freeman; 1997. Self-efficacy. [Google Scholar]
- Board of Scientific Advisors Ad Hoc Working Group. An assessment of the impact of the NCI cancer biomedical informatics GRID (caBIG) National Cancer Institute; 2011. [Google Scholar]
- Boyd LB, Hunicke-Smith SP, Stafford GA, Freund ET, Ehlman M, Chandran U, Klemm JD. The caBIG(R) life science business architecture model. Bioinformatics (Oxford, England) 2011;27(10):1429–1435. doi: 10.1093/bioinformatics/btr141. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cohen MZ, Rozmus C, Mendoza T, Padhye N, Neumann J, Gning I, Aleman A, Giralt S, Cleeland C. Symptoms and quality of life in diverse patients undergoing hematopoietic stem cell transplantation. Journal of Pain and Symptom Management. 2012;44(2):168–180. doi: 10.1016/j.jpainsymman.2011.08.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hamilton CM, Strader LC, Pratt JG, Maiese D, Hendershot T, Kwok RK, Haines J. The PhenX toolkit: Get the most from your measures. American Journal of Epidemiology. 2011;174(3):253–260. doi: 10.1093/aje/kwr193. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kaiser MM, Kaiser KL, Barry TL. Health effects of life transitions for women and children: A research model for public and community health nursing. Public Health Nursing (Boston, Mass) 2009;26(4):370–379. doi: 10.1111/j.1525-1446.2009.00792.x. [DOI] [PubMed] [Google Scholar]
- Kim J, Pressler SJ, Jones J, Graves JR. Generating scientific models of knowledge using arcs. Clinical Nurse Specialist CNS. 2008;22(6):286–292. doi: 10.1097/01.NUR.0000325386.45459.a0. [DOI] [PubMed] [Google Scholar]
- Macken LC, Yates BC, Meza J, Norman J, Barnason S, Pozehl B. Health-related quality-of-life outcomes in coronary artery bypass surgery patients and partners. Journal of Cardiopulmonary Rehabilitation and Prevention. 2014;34(2):130–137. doi: 10.1097/HCR.0b013e3182a528ba. [DOI] [PMC free article] [PubMed] [Google Scholar]
- National Cancer Institute. About GEM. 2012 Retrieved November 4, 2014, Retrieved from https://www.gem-beta.org/public/About.aspx?cat=5.
- National Institutes of Health. Common data element (CDE) resource portal - glossary. 2014 Retrieved November 4, 2014, Retrieved from http://www.nlm.nih.gov/cde/glossary.html.
- National Institutes of Health. Common data element (CDE) resource portal - Summary table for NIH CDE initiatives. 2014 Retrieved November 4, 2014 Retrieved from http://www.nlm.nih.gov/cde/summary_table_1.html.
- Northwestern University. 2013 Http://Www.neuroqol.org/HowDoI/GetPermission/pages/default.aspx. Retrieved November 4, 2014, Retrieved from http://www.neuroqol.org/HowDoI/GetPermission/Pages/default.aspx.
- Pender NJ, Murdaugh C, Parsons MA. Health promotion in nursing practice. 6. Upper Saddle River, NJ: Pearson/Prentice-Hall; 2010. [Google Scholar]
- Pragodpol P, Ryan C. Critical review of factors predicting health-related quality of life in newly diagnosed coronary artery disease patients. Journal of Cardiovascular Nursing. 2013;28(3):277–284. doi: 10.1097/JCN.0b013e31824af56e. [DOI] [PubMed] [Google Scholar]
- Prochaska JO, DiClemente CC, Norcross JC. In search of how people change. applications to addictive behaviors. The American Psychologist. 1992;47(9):1102–1114. doi: 10.1037//0003-066x.47.9.1102. [DOI] [PubMed] [Google Scholar]
- Riley WT, Glasgow RE, Etheredge L, Abernethy AP. Rapid, responsive, relevant (R3) research: A call for a rapid learning health research enterprise. Clinical and Translational Medicine. 2013;2(1):10-1326-2-10. doi: 10.1186/2001-1326-2-10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- RTI International. PhenX toolkit. 2014 Retrieved November 4, 2014, Retrieved from https://www.phenxtoolkit.org/
- Weinert C, Boik RJ. MSU rurality index: Development and evaluation. Montana State University. Research in Nursing & Health. 1995;18(5):453–464. doi: 10.1002/nur.4770180510. [DOI] [PubMed] [Google Scholar]
- Zimmerman L, Barnason S, Hertzog M, Young L, Nieveen J, Schulz P, Tu C. Gender differences in recovery outcomes after an early recovery symptom management intervention. Heart & Lung. 2011;40(5):429–439. doi: 10.1016/j.hrtlng.2010.07.018. [DOI] [PMC free article] [PubMed] [Google Scholar]