Abstract
Since 2006, the National Institutes of Health has provided institutional infrastructure grants, called Clinical and Translational Science Awards (CTSAs), to support adult and pediatric clinical and translational research in United States institutions. A CTSA Consortium Child Health Oversight Committee workgroup developed metrics to measure the impact of CTSAs on child health (CH) research. A cross-sectional survey to collect metric data was distributed to the 46 institutions that received CTSAs during 2006-09. Thirty-seven (80%) institutions responded to the survey. Data regarding 7 metrics were reported by >70% of responding institutions: the proportion of overall funding (median, interquartile range; 0.12, 0.06–0.19) and pilot grants (0.15, 0.11–0.21) supporting CH research; the proportion of active clinical research center studies involving children (0.23, 0.15–0.35); the proportion of IRB-approved (0.24, 0.16–0.30) and funded (0.22, 0.18–0.30) studies involving children; the proportion of mentored research training awards to CH investigators (0.18, 0.11–0.28); and, the proportion of CTSA leadership positions held by pediatricians (0.18, 0.12–0.28). CTSAs provide substantial support for CH research, although additional investment in CH research is needed to improve the health of children. These metrics provide an initial means to track the impact of CTSAs on CH research.
Introduction
The National Institutes of Health (NIH) has an extensive history of supporting child health research through a variety of infrastructure grants, including funding for clinical research centers (CRC), institutional clinical research curricula and training and career development programs for child health researchers, pediatric clinical research networks, and education and meeting activities (1, 2). In 2006, the NIH launched the Clinical and Translational Science Award (CTSA) program with the vision that these enhanced institutional infrastructure grants would better catalyze the translation of new scientific discoveries into improved health of human populations (3, 4). Existing pediatric CRCs and some institutional training and career development programs were subsumed into the CTSAs as linked awards; there was no requirement for a separate child health component. The NIH Reform Act of 2006 changed this framework in order to preserve independent funding and infrastructure for pediatric CRCs (5). Subsequent CTSA Request for Applications (RFA) have allowed for the appointment of a pediatric co-principal investigator under a single CTSA who would have direct authority over a separate budget or would otherwise secure institutional independence of pediatric CRCs with respect to finances, infrastructure, resources, and research agenda. As of 2011, 58 of the 60 CTSAs include child health components and 9 have principal investigators who are themselves pediatricians (6). However, none of the CTSAs have chosen to create a separate budget for child health, which makes it difficult to directly identify the resources devoted to support of child health research in individual institutions and to measure the effectiveness of the CTSA program as a whole in supporting child health research over time.
Concurrent with the funding of the first group of 12 CTSAs in 2006, the CTSA Consortium-Child Health Oversight Committee (CC-CHOC) was established as a part of the broader CTSA Consortium governance structure and was charged with the responsibility of facilitating child health research throughout the Consortium (7, 8). A CC-CHOC workgroup was formed to develop and test a measurement strategy to: 1) enable CTSA institutions to uniformly assess their current levels of support, research activity, training of new investigators, and multicenter collaborations related to child health research; 2) track changes in these parameters over time within institutions; and 3) eventually estimate the impact of the CTSA Consortium, as a whole, on child health research. This report describes the initial development and evaluation of the measurement strategy and presents data on specific metrics from CTSA institutions receiving CTSAs during 2006-2009.
Methods
Definitions
The workgroup used the NIH definitions for clinical and translational research (9) and for inclusion of children as participants in research involving human subjects (10) to define child health research as, “all clinical and translational research involving subjects less than 21 years of age that impacts children’s health or addresses childhood diseases, including maternal-fetal research, as well as studies involving infants, children and adolescents.” Research that primarily focused on adults but in which a few older adolescents were enrolled was excluded.
Development of the Metrics
From 2008-2009, workgroup members representing 12 CTSAs participated in monthly teleconferences during which measurement domains and specific metrics were proposed, discussed, and selected. Decisions were made by consensus during all phases of this process. The workgroup reviewed the CTSA Consortium Strategic Goals (11) and chose the following measurement domains, which aligned with these Strategic Goals: I) funding, infrastructure support, and research activity; II) career development of new investigators; III) collaborative research; and, IV) leadership. Workgroup members were then asked to propose specific metrics in each of these domains and then to rank prospective metrics using the following criteria: 1) capacity to utilize current systems to capture existing data; 2) generalizability across most institutions; and 3) ability to identify “new value” attributable the CTSA infrastructure. Metrics that were ranked highly across these criteria were selected for further development. No metrics were added or deleted thereafter. Several members of the workgroup (S.L.B., C.D.S., W.C.H.) collaborated to draft a written description of the metrics with specific definitions of the numerator and denominator for each metric and instructions for data collection. Other workgroup members provided critique of the draft and comments were incorporated into a revised version which was presented to the workgroup in the spring of 2009. During the summer of 2009, members of the workgroup pilot-tested procedures for collecting the required data in 6 of their own institutions and the metric definitions and data collection procedures were revised based on their recommendations. The proposal to implement the data collection strategy for the metrics was reviewed and approved by the CTSA Consortium Steering Committee, a body that consists of all participating CTSA principal investigators and NIH CTSA representatives.
Collection of Metric Data
In November 2009, the final version of the metrics (Table 1) and a worksheet for data collection with instructions were distributed by e-mail to the CC-CHOC member or other relevant contact at each CTSA institution. For each metric, the institution recorded the numeric data and provided qualitative comments, as needed, about how the metric was interpreted and data were collected, as well as any barriers or difficulties encountered in this process. An open-ended field allowed for additional qualitative descriptions of institutional enhancements to promote the conduct of child health research. Given the need to assess administrative financial and IRB databases, data were collected primarily by administrative personnel under the supervision of the CC-CHOC member and either entered directly by each institution using a confidential access code into a Research Electronic Data Capture (REDCap, Vanderbilt Institute for Clinical and Translational Research, VICTR, Nashville, Tennessee) database (12) or transmitted confidentially to VICTR and subsequently entered into the database. Four follow-up e-mails reminders were sent to CC-CHOC members at all institutions, but non-responding institutions were not contacted individually either by e-mail or telephone.
Table 1.
Measurement Domains and Description of Metrics
| Measurement Domain | Description of Metric a |
|---|---|
| I. Support, infrastructure, and research activity b |
|
| Overall funding | Proportion of CTSA grant dollars allocated in support of CH investigators c, d |
| Pilot grants | Proportion of CTSA-funded pilot grants allocated in support of CH research |
| CRC utilization | Proportion of active inpatient and outpatient studies conducted in the CRC involving pediatric subjects d |
| Pilot grants | Proportion of CTSA-funded pilot grants allocated in support of CH research |
| CRC utilization | Proportion of active inpatient and outpatient studies conducted in the CRC involving pediatric subjects d |
| Research activity | Proportion of active IRB-approved studies that involve pediatric subjects |
| Proportion of active IRB-approved and funded studies that involve pediatric subjects |
|
| II. Career development of new investigators b |
Proportion of mentored clinical and translational research training awards held by pediatric/CH trainees and faculty e |
| III. Collaborative research b | Number of multicenter studies involving pediatric subjects in which more than one CTSA participates f |
| IV. Leadership | Proportion of CSTA leadership positions held by pediatric/CH investigators g |
CH, child health; CRC, clinical research center; CTSA, Clinical and Translational Science Award; IRB, Institutional Review Board; NIH, National Institutes of Health
Numeric data values needed to calculate proportions were requested and reported (see Table 2)
Aligned with CTSA Consortium Strategic Goals 1-3 (11)
Including such services as pilot grants, education/training, research infrastructure, and study design services.
Including the following funding sources: NIH, investigator-initiated industry, industry-initiated, foundation, other
Including NIH KL2, K08, K12, and K23 awards, foundation and academic society career development awards
Including studies with co-investigators at other CTSA sites or with collaborative or subcontract arrangements with other CTSA sites
Individuals with substantial influence on directing strategy and resource allocation for the institutional or national CTSA program (e.g., principal investigator, co-principal investigator, program director, associate director, advisory committee member, administrative leader)
Data and Statistical Analyses
REDCap and SPSS, version 18 (IBM SPSS Statistics, formerly Predictive Analytics SoftWare Statistics, Armonk, New York) were used to generate descriptive statistics. Qualitative data were abstracted and synthesized by members of the workgroup (W.C.H., C.D.S., J.W., S.L.B.) and reviewed by the other members. Fisher exact tests were used to compare institutional characteristics between responding and non-responding CTSA institutions.
Results
All 46 institutions that received CTSAs from 2006-2009 received an invitation to participate; 37 (80%) institutions provided responses. Comparing responding and non-responding institutions, there was no evidence of a difference in geographic distribution by region (P=0.96), year of award (P=0.75), and the amount of the CTSA in dollars by quartiles of percentile rank (P=0.33). A children’s hospital was listed as a partner institution and a pediatrician was the CTSA Principal Investigator for 34 (92%) and 5 (14%) responding institutions, respectively, vs. for 6 (67%, P=0.08) and 0 (P=0.57) of non-responding institutions, respectively.
The numeric data needed to calculate each metric were provided by most of the responding institutions (Table 2). Except for the collaborative research metric, data were provided similarly for child health research, investigators, or pediatric subjects (the numerators) and for the CTSA or institution as a whole (the denominators). Data regarding the number of multicenter studies involving more than one CTSA were the most difficult for institutions to provide, both for studies involving pediatric subjects and for studies involving subjects of any age.
Table 2.
Institutions Providing Data and Descriptive Data Regarding Metrics
| Value relating to CH research/investigators or involving pediatric subjects |
Value relating to the CTSA or institution as a whole |
|||
|---|---|---|---|---|
| Measurement Domain/Metric a | Number providing data (%) |
Median (range) | Number providing data (%) |
Median (range) |
| I. Support, infrastructure, and research activity |
||||
| Overall funding, dollars | 36 (97%) | 643,379 (69,994 – 17,564,022) | 36 (97%) | 6,727,057 (1,365,625 – 52,750,544) |
| Pilot grants, number | 35 (95%) | 4 (0 – 34) | 35 (95%) | 20 (0 – 288) |
| CRC utilization | ||||
| Active CRC inpatient and outpatient studies, number |
35 (95%) | 32 (1 – 263) | 35 (95%) | 159 (21 – 449) |
| Research activity | ||||
| IRB-approved studies, number | 35 (95%) | 335 (2 – 1596) | 31 (84%) | 1483 (55 – 6568) |
| IRB-approved and funded studies, number |
33 (89%) | 295 (4 – 983) | 27 (73%) | 1371 (45 – 5228) |
| II. Career development of new investigators |
||||
| Mentored clinical and translational research training awards, number |
33 (89%) | 2 (0 – 170) | 28 (76%) | 48 (0 – 625) |
| III. Collaborative research | ||||
| Multicenter studies with ≥1 CTSA institution participating, number |
29 (78%) | 3 (0 – 208) | -- b | -- b |
| IV. Leadership | ||||
| Leadership positions, number | 37 (100%) | 4 (1 – 13) | 36 (97%) | 23 (5 – 75) |
CH, child health; CRC, clinical research center; CTSA, Clinical and Translational Science Award; IRB, Institutional Review Board
See Table 1 for description of each metric
Few institutions were able to provide data for the institution as a whole
The numeric data values reported by individual institutions ranged widely (Table 2). Likewise, there was wide variation in the calculated metrics (Figure 1), most notably in the metrics related to active CRC studies, mentored research training awards, and CTSA leadership positions involving child health researchers. There was diversity among the institutions that represented the upper bounds of the ranges, with 6 different institutions represented (the upper bound was represented by 1 institution for the metrics related to overall funding, active CRC studies, and mentor research training awards, 1 institution for both metrics related to research activity, and 4 different institutions for the other metrics). Nonetheless, despite the overall variation, the medians of the metrics were generally consistent (Figure 1, range of medians, 0.11-0.24) and the central portions of their distributions, as represented by the interquartile ranges, were relatively narrow.
Figure 1.
Distribution of Metric Proportions across CTSA Institutions
CRC, clinical research center; CTSA, Clinical and Translational Science Award; IRB, Institutional Review Board
Boxplots show the distribution of metric proportions reported by the 37 responding institutions (minimum, 25%, 50%, 75%, maximum). The number and percent of institutions reporting data for each metric is indicated below the metric label. Table 1 provides the full description of each metric. Metric III. Collaborative research (multicenter studies with ≥1 CTSA institution participating) was not included because few institutions were able to provide data for the institution as a whole (see Table 3).
The number of institutions included in each distribution is: overall funding 36, pilot grants 33, active CRC studies 35, IRB-approved studies 31, IRB-approved and funded studies 27, mentored clinical/translational research training awards 27, leadership positions 36.
Challenges in collecting the numeric data were due primarily to issues related to the interpretation of the definitions and the data sources of the metrics (Table 3). Differences between NIH and IRB definitions of child health research and whether to include other sources of support (e.g., supplemental awards, institutional matching funds) were identified commonly. Less common, but more fundamental, were issues of whether studies that relate to child health but do not involve children as research subjects directly should be viewed as child health research, such as healthcare services research and maternal health studies. Inadequate sources of data were a major challenge due to the lack of variables that clearly identify child health research studies and to difficulties synchronizing the records of separate institutional databases, such as institutional review board (IRB) and grant funding databases, to perform queries using the combined data fields from each database.
Table 3.
Description of Issues Identified in Collecting Data Regarding the Metrics a
| Measurement domain / type of issue |
Description of Issue a |
|---|---|
| I. Support, infrastructure, and research activity |
|
| Definitions b | NIH definition defines a child as a person less than 21 years of age, whereas many IRBs define a child as less than 18 years of age |
| Support may also include supplemental awards and institutional matching funds Studies that do not utilize the CRC may benefit from use of other CTSA resources (e.g., study design/biostatistics consultation) |
|
| Studies that do not involve subject contact (e.g. health services research) or contact with children (e.g., maternal health research) may be viewed as CH research |
|
| Data Sources c | Support for infrastructure (e.g., facilities, core laboratories) is difficult to track Data fields in grants management databases are insufficient to identify all CH studies IRB, grants management, and CRC databases are rarely synchronized |
| II. Career development of new investigators |
|
| Definitions b | CH investigators in non-medical schools (e.g., public health, pharmacy) may also receive these awards |
| Data Sources c | Non-federally funded mentored research training awards are not recorded reliably in existing databases |
| III. Collaborative research | |
| Definitions b | Sub-contracts and studies funded by non-federal sources may represent additional evidence of collaboration |
| Data Sources c | Data fields in grants management databases are insufficient to identify the participating sites in multicenter studies, to apportion fractional credit to sites that are identified, and to ascertain whether participating sites are CTSA institutions |
| IV. Leadership | |
| Definitions b | Positions on external and internal advisory committees and CTSA Consortium-related committees and directorships of core laboratories and training grants may also represent important leadership roles |
CH, child health; CRC, clinical research center; CTSA, Clinical and Translational Science Award; IRB, Institutional Review Board; NIH, National Institutes of Health
Variability in how individual institutions interpreted and dealt with the issues identified with respect to definitions (**) and data sources (†) may have resulted in underestimates in individual institutions and greater variability among institutions.
May not be all-inclusive or may be interpreted differently among institutions
May not capture all relevant activity
Institutions provided additional qualitative statements of how the CTSA prompted the development of new institutional strategies and initiatives to enhance child health research, including: establishing multidisciplinary child health research teams or committees to better integrate child health research into the CTSA, enhancing linkages with basic science researchers, and developing specific foci of expertise in child health research; using evidence of CTSA support to obtain additional matching funds to support child health research and new child health investigators; using indirect funds to increase investment in child health research; recruiting child health researchers to compete for pilot grants; establishing a community-based child health research network; and inaugurating an annual pediatric research day event.
Discussion
The metrics presented in this cross-sectional survey illustrate that institutions that received CTSAs from 2006-09 provide substantial support for child health research through overall funding and pilot study grants, CRC utilization, and mentored research training awards for child health investigators. In these institutions, roughly a quarter of all IRB-approved studies include pediatric subjects and nearly a fifth of CTSA leadership positions are filled by pediatricians. These findings may be positively skewed by the trend toward a higher proportion of institutions with a children’s hospital listed as a partner institution among responding vs. non-responding institutions.
Although these findings are generally positive, additional investment by CTSAs in child health research will be required to achieve the major breakthroughs necessary to improve the health of children and those children into adulthood. As reported in this study, the proportions of overall funding and pilot study grants devoted to child health research in CTSA institutions are consistent with the proportion of NIH funding targeted toward child health research reported recently (13). However, the proportion of overall NIH funding for child health research has declined by nearly a third over the past two decades (13). Moreover, leaders in the pediatric research community have identified major unmet needs for child health research that must be addressed in order to improve the future health and well-being of children in the United States (13, 14).
These results should also be viewed in the context of the limitations of this first attempt to assess the impact of the CTSA program on child health research. The lack of pre-award data and the cross-sectional design of this study, which occurred 1 to 4 years after the receipt of the CTSA in the cohorts of institutions receiving awards during 2006-09, do not allow us to assess trends in the support for child health research in relation to the CTSA in individual institutions or across the Consortium as a whole. Although we provided instructions to standardize data collection, issues related to metric definitions and data sources (Table 3) may have resulted in underestimates or greater variability with respect to the metric data reported by individual institutions. We did not perform formal assessments of the reliability of data collection at individual CTSA institutions or the validity of the data in relation to externally verified measures, although we plan to do so in the future. The difficulties we encountered with definitions and data sources for the metrics illustrate general challenges inherent in quantifying the impact of the CTSAs on clinical and translational research in general, and on child health research in particular. Some of these barriers can be addressed easily. For instance, we have already refined the definitions of the numerators and denominators to be more detailed and specific. Other barriers, such as those involving data sources, are likely to require new or revised databases with additional data fields and improved linkages. The difficulty we encountered in capturing information about research collaborations across CTSA institutions from existing grants management databases deserves particular attention because this metric was one of the most difficult to quantitate and yet it uniquely reflects the power of the Consortium.
Nonetheless, the metrics presented in this study provide an important starting point for evaluating the impact of CTSAs on child health research. First, the metrics were developed intentionally to align with the strategic goals of the CTSA Consortium. Therefore, over time, they should reflect enhancements to the clinical and translational research infrastructure that occur in individual institutions and throughout the Consortium. Second, they provide an opportunity for CTSA institutions to assess their current support for child health research in relation to the Consortium as a whole. Third, the metrics provide a standardized mechanism for individual institutions to track the effect of their efforts to enhance child health research over time through changes in either the numeric data values (i.e., the numerators for the metrics), the calculated proportions, or both. Institution-specific tracking of these metrics is important since factors that limit the validity of comparisons across institutions—particularly those related to sources of data—are likely to be less variable within institutions over time. Moreover, by creating an institutional process for measuring child health research efforts, CTSA institutions can examine if they are achieving the child health research goals they set. Fourth, the high response rate indicates there is support for this effort across the Consortium.
To better understand the impact of CTSAs on child health research, we also need better approaches for describing child health research portfolios, including the spectrum of translational research (i.e., T1 through T4) conducted in CTSA institutions(15), research that crosses populations (e.g., women’s health research with outcomes relevant to child health, studies of adult diseases that have their antecedents in conditions affecting children), and research that has an important impact on child health, but does not involve pediatric subjects directly.
As the CTSA Consortium matures, its strategic goals evolve, and the definition of child health research is refined, the metrics used to describe the impact of CTSAs on child health research may need to be adapted. We hope these data will also inform a discussion of what may be considered adequate (or optimal) performance with respect to these metrics. Finally, one of the aspirations of this effort is to develop metrics to evaluate the impact of CTSAs on child health, not just child health research. Indeed, the process of developing such a measurement strategy, in and of itself, demonstrates one of the potential strengths of the CTSA Consortium in facilitating child health research (8).
Acknowledgments
The authors acknowledge Tomaso Tempesti for his assistance with statistical analysis, Ashley Harris for her assistance with administering the survey, and the CTSA Consortium Publications Committee for their critical review of the manuscript.
Statement of Financial Support: This publication was supported by National Institutes of Health/National Center for Research Resources Clinical and Translational Science Award Grant Numbers UL1 RR024150 (Mayo Center for Translational Science Activities, Mayo Clinic), UL1 RR024975 (Vanderbilt Institute for Clinical and Translational Research, Vanderbilt University); UL1 RR029884 (Arkansas Center for Clinical and Translational Research, University of Arkansas for Medical Sciences); UL1 RR025744 (The Stanford Center for Clinical and Translational Education and Research, Stanford University; UL1 RR025741 (Northwestern University Clinical and Translational Sciences Institute, Northwestern University); UL1 RR024153 (University of Pittsburgh Clinical and Translational Science Institute, University of Pittsburgh); UL1 RR025758 (Harvard Catalyst: The Harvard Clinical and Translational Science Center, Harvard University); UL1 RR024146 (UC Davis Clinical and Translational Science Center, University of California, Davis); UL1 RR024986 (Michigan Institute for Clinical and Health Research, University of Michigan at Ann Arbor); UL1 RR024160 (University of Rochester Clinical and Translational Sciences Institute, University of Rochester School of Medicine and Dentistry). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIH.
Abbreviations
- CC-CHOC
Clinical and Translational Science Award Consortium-Child Health Oversight Committee
- CH
child heath
- CRC
Clinical Research Center
- CTSA
Clinical and Translational Science Award
- IRB
Institutional Review Board
- NCRR
National Center for Research Resources
- NIH
National Institutes of Health
- RFA
Request for Applications
- VICTR
Vanderbilt Institute for Clinical and Translational Research
Footnotes
The authors have no relevant financial disclosures or other potential conflicts of interest to report.
Contributor Information
W. Charles Huskins, Division of Pediatric Infectious Diseases, Mayo Clinic, Rochester, Minnesota 55905.
Clare D. Sullivan, Vanderbilt Institute for Clinical and Translational Research Nashville, Tennessee 37203
Janey Wang, Vanderbilt Institute for Clinical and Translational Research Nashville, Tennessee 37203.
Mary Aitken, Arkansas Children’s Hospital Research Institute, Little Rock, Arkansas 72202.
Steven R. Alexander, Department of Pediatrics, Stanford University School of Medicine, Stanford, California 94305
Leon Epstein, Northwestern University Feinberg School of Medicine, Chicago, Illinois 60611.
Alejandro Hoberman, Department of Pediatrics, University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania 15224.
Ellis Neufeld, Children’s Hospital Boston and Harvard Medical School, Boston, Massachusetts 02115.
Anthony Philipps, Department of Pediatrics, University of California Davis School of Medicine, Sacramento, California 95817.
Thomas P. Shanley, University of Michigan Medical School, Ann Arbor, Michigan 48103
Peter Szilagyi, Department of Pediatrics, University of Rochester School of Medicine and Dentistry; Rochester, New York 14642.
Mary Purucker, National Center for Research Resources, National Institutes of Health, Bethesda, Maryland 20892.
Shari L. Barkin, Department of Pediatrics, Vanderbilt University School of Medicine, Nashville, Tennessee 37232
References
- 1.Winer KK, Levin E, Grave GD, Alexander DF. The Child Health Research Centers program: 1990-2000. J Pediatr. 2001;139:757–8. doi: 10.1067/mpd.2001.120130. [DOI] [PubMed] [Google Scholar]
- 2.McCabe LL. National Institutes of Health support for research and training: future of pediatrician scientists. Arch Pediatr Adolesc Med. 1998;152:839–42. doi: 10.1001/archpedi.152.9.839. [DOI] [PubMed] [Google Scholar]
- 3.Zerhouni EA. Translational and clinical science--time for a new vision. N Engl J Med. 2005;353:1621–3. doi: 10.1056/NEJMsb053723. [DOI] [PubMed] [Google Scholar]
- 4.Zerhouni EA. US biomedical research: basic, translational, and clinical sciences. JAMA. 2005;294:1352–8. doi: 10.1001/jama.294.11.1352. [DOI] [PubMed] [Google Scholar]
- 5.Office of Legislative Policy and Analysis [21 April 2011];Legislative updates: National Institutes of Health Reform Act of 2006. http://olpa.od.nih.gov/legislation/109/publiclaws/reformact06.asp.
- 6.CTSA Consortium [29 November 2011];NCRR Fact Sheet: Clinical and Translational Science Awards. https://www.ctsacentral.org/documents/Communication_Toolkit/CTSA_FactSheet.pdf.
- 7.CTSA Consortium [29 November 2011];CTSA Consortium Child Health Oversight Committee. https://www.ctsacentral.org/committee/ctsa-consortium-child-health-oversight-committee.
- 8.Heubi JE. Child health research and the Clinical Translational Science Awards: where have we been and where are we going? Clin Transl Sci. 2010;3:67–68. doi: 10.1111/j.1752-8062.2010.00187.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Office of Extramural Research, National Insitutes of Health [14 May 14 2011];Glossary & acronym list. http://grants.nih.gov/grants/glossary.htm.
- 10.NIH Guide [14 May 2011];NIH policy and guideines on the inclusion of children as participants in research involving human subjects. http://grants.nih.gov/grants/guide/notice-files/not98-024.html.
- 11.Clinical and Translational Science Awards [29 November 2011];Strategic Goal Committees. https://www.ctsacentral.org/content/strategic-goal-committees.
- 12.Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42:377–81. doi: 10.1016/j.jbi.2008.08.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Hay WW, Jr, Gitterman DP, Williams DA, Dover GJ, Sectish TC, Schleiss MR. Child health research funding and policy: imperatives and investments for a healthier world. Pediatrics. 2010;125:1259–65. doi: 10.1542/peds.2009-2635. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Genel M, McCaffree MA, Hendricks K, Dennery PA, Hay WW, Jr., Stanton B, et al. Pediatrics; A National Agenda for America’s Children and Adolescents in 2008: recommendations from the 15th Annual Public Policy Plenary Symposium, Annual Meeting of the Pediatric Academic Societies; May 3, 2008; 2008. pp. 843–9. [DOI] [PubMed] [Google Scholar]
- 15.Szilagyi PG. Translational research and pediatrics. Acad Pediatr. 2009;9:71–80. doi: 10.1016/j.acap.2008.11.002. [DOI] [PubMed] [Google Scholar]