Geographically-Related Outcomes of U.S. Funding for Small Business Research and Development: Results of the Research Grant Programs of a Component of the National Institutes of Health

Abstract

This article examines the geographic distribution of funding for the U.S. Small Business Innovation Research (SBIR) and Small Business Technology Transfer (STTR) programs sponsored by the National Institute of General Medical Sciences (NIGMS). Despite a significant investment in SBIR/STTR and an interest in increasing geographic diversity in the institute’s research portfolio, there has not been an assessment of the distribution of NIGMS’s SBIR/STTR funding, outcomes associated with that investment, and relationships between the two. The geographic distribution of NIGMS’ SBIR/STTR funding was highly concentrated in a small number of states, with a high correlation between each state’s funding and its number of small scientific research and development businesses. Affiliation with a major research university was correlated with several measures of innovation and firm success. Our findings are consistent with earlier research showing that economic activity in research and development and research output tend to cluster in geographic regions where knowledge can be generated and shared more efficiently. These findings lend support to an investment strategy for small business research and development that creates networks between major research universities and small businesses.

1.0. Introduction

This paper reports key outcomes from the National Institute of General Medical Sciences (NIGMS)¹ Small Business Innovation Research (SBIR) and Small Business Technology Transfer (STTR) grant programs. Unlike previous evaluations of SBIR and STTR programs that employed surveys for data collection, this work utilized administrative data from existing large-scale databases to perform the first in-depth evaluation of the outcomes of NIGMS’s investment, commensurate with the stewardship function of NIGMS.

The United States government’s SBIR and STTR programs were established by the Small Business Innovation Development Act of 1982 (P.L. 97-219) and the Small Business Technology Transfer Act of 1992 (P.L. 102-564), respectively, to strengthen the role of small, innovative firms in federally-supported research and development (R&D). The programs have been reauthorized several times since 1982, the latest in 2016 (Section 1834 of P.L. 114-840).

The SBIR program’s mission is to support scientific excellence and technological innovation through the investment of federal research funds in critical American priorities to build a strong national economy. The four major goals of the program are to: meet federal R&D needs; stimulate technological innovation; increase private-sector commercialization of innovations derived from federal R&D; and foster and encourage participation in innovation and entrepreneurship by socially and economically disadvantaged persons (U.S. Small Business Administration, 2017a).

STTR program goals are similar but include an additional goal to “foster technology transfer through cooperative R&D between small businesses and research institutions” (U.S. Small Business Administration, 2017b). There are two unique features of the STTR program that distinguish it from the SBIR program. One is a requirement that the small business formally collaborate with a non-profit research institution (typically, an academic institution). The small business must perform a minimum of 40 percent of the research effort and the research institution must perform at least 30 percent. Secondly, unlike the SBIR program, the STTR program does not require the principal investigator (PI) to be primarily employed by the small business concern. The PI may be primarily employed by either the small business concern or the collaborating non-profit research institution.

Both SBIR and STTR are three-phase programs consisting of a first phase to establish the technical merit, feasibility, and commercial potential of an R&D effort. If feasibility and commercial potential are demonstrated in Phase I, a second phase may be funded to continue the R&D initiated in Phase I. Phase III, not funded by the SBIR/STTR program, is for the small business to pursue commercialization of the results of Phase I and Phase II R&D activities. A percentage of each federal agency’s extramural R&D budget is allocated to funding for the SBIR/STTR programs. This percentage has grown over time, from 0.2 percent in fiscal year (FY) 1983, to 3.65 percent in FY 2017 (3.2 percent for SBIR and 0.45 percent for STTR).

To enhance the federal program, 14 states have created programs to provide matching funds to successful SBIR/STTR applicants within their state (Lanahan & Feldman, 2015). Further stimulating small business R&D within each state, the SBIR/STTR Reauthorization Act of 2011² sought to increase funding to states with historically low levels of SBIR/STTR funding by requiring coordination of these programs with other federal research grant programs designed to increase the geographic distribution of federal funds for R&D. One such program is the Institutional Development Award (IDeA) program administered by the National Institute of General Medical Sciences (NIGMS), a component of the National Institutes of Health (NIH). The IDeA program “broadens the geographic distribution of NIH funding for biomedical research… [and] fosters health-related research and enhances the competitiveness of investigators at institutions located in states in which the aggregate success rate for applications to NIH has historically been low” (National Institute of General Medical Sciences, 2017b).

Despite a significant level of investment in the SBIR/STTR programs and an interest in increasing geographic diversity in the institute’s research portfolio, there has not been an in-depth assessment of the geographic distribution of NIGMS’s SBIR/STTR funding, outcomes associated with that investment, and relationships between the two. To assess the extent to which NIGMS is meeting its goal of investing in a broad and diverse portfolio of highly meritorious research (National Institute of General Medical Sciences, 2015), we examine the geographic distribution of funding through NIGMS’ SBIR and STTR programs, comparing this to other NIH research funding, and we examine evidence that such funding is related to local research capacity and proximity to academic institutions, the recipients of a large majority of NIH’s research grant support. We focus on academic institutions as they are the major recipients of NIGMS’s research programs. We also examine any changes in the distribution of NIGMS funding following the Congressional directive to increase funding to states with historically low levels of SBIR/STTR funding.

In previous studies of SBIR and STTR programs, the primary source of outcome information has been surveys of past recipients of SBIR/STTR funding (e.g., National Research Council Committee for Capitalizing on Science, Technology, and Innovation, 2009). Two national surveys of the NIH SBIR program have been conducted within the past 15 years to assess the success of that program in meeting one or more of the program’s goals (National Institutes of Health, Office of Extramural Research, 2003; National Academies of Sciences, Engineering, and Medicine, 2015). Such surveys have typically suffered from low response rates, raising the possibility of non-response bias in the results. For example, of a total population of 3,375 funded investigators, only 21.5 percent (726 respondents) were located and responded in the 2014 survey of NIH awardees (National Academies of Sciences, Engineering, and Medicine, 2015).

In this paper, we rely on readily-available existing databases rather than surveys of SBIR grantees as the primary source of information to examine the outcomes of NIGMS SBIR/STTR funding to focus on two goals of the SBIR program: to stimulate technological innovation and to increase private-sector commercialization derived from federal R&D. The greater availability of public datasets, and improvements in our ability to merge them, have contributed to the development of useful tools to support evidence-based decision making (Morrel-Samuels, Francis, & Shucard, 2009) and linked datasets are a critical component of current efforts to strengthen evidence-based decision making across the U.S. federal government (Commission on Evidence-Based Policymaking, 2017). Finally, we discuss the synergy between our findings and the establishment of NIGMS-funded Regional Technology Transfer Accelerator Hubs to support IDeA states.

1.1. The Geographic Distribution of Federal Funding for R&D

Programs such as IDeA address a long-standing disparity among states in the amount of federal support they receive in the form of grants and contracts to academic institutions and other research organizations. The geographic distribution of this “extramural” funding is not uniform; in FY 2015, over fifty percent of federal obligations for extramural R&D were concentrated in seven states (NSF National Center for Science and Engineering Statistics, 2017). The distribution of extramural funding from NIH is distributed similarly, with six states (California, Massachusetts, New York, Pennsylvania, Texas, and North Carolina) receiving over 50 percent of funding for research grants and contracts in FY 2017 (National Institutes of Health, 2017b).

Almost all of NIH’s extramural funding is awarded based on the results of peer review of research proposals, and there is evidence that the geographic distribution of federal funding is highly correlated with states’ research capacity. The Government Accountability Office, for instance, citing research by the Small Business Administration, noted that the distribution of all science-related awards—both through the SBIR program and other federal support—tends to follow the distribution of scientists and engineers in the workforce, with geographic clusters of science and engineering talent generating proportionately more awards (U.S. Government Accountability Office, 1999). A large percentage of NIH extramural funding goes to medical schools and research doctorate-granting institutions, and the distribution of funding is highly correlated with the geographic distribution of doctorate recipients (Wu, 2013).

Inequitable state funding for research has created pressure for federal agencies to provide programs to create greater geographic equity in federal support of R&D. Promoting geographic equity was the motivation behind the creation the National Science Foundation’s (NSF) Established Program to Stimulate Competitive Research (EPSCoR) in 1979 (Wu, 2013). EPSCoR “enhances research competitiveness of targeted jurisdictions (states, territories, commonwealth) by strengthening STEM capacity and capability” (National Science Foundation, 2017). Along with the NSF EPSCoR and NIH IDeA programs, three other federal agencies have their own active EPSCoR programs: the Departments of Energy and Agriculture and the National Aeronautics and Space Administration (EPSCoR/IDeA Foundation, 2018).

Geographic differences in federal research funding have implications for small business development, which is closely linked to local research capacity. Economic activity in R&D tends to cluster in geographic regions where knowledge can be generated and shared more efficiently. Geographic proximity to research universities and other producers of scientific knowledge has been shown to contribute to both the development and innovativeness of small firms (Zucker, Darby, & Armstrong, 2001; Zucker, Darby, & Brewer, 1994). R&D firms are attracted to regions that provide access to university laboratories and staff, cutting edge discoveries, and pathways for hiring top graduates (Chatterjee & DeVol, 2012). Increasing the geographic diversity of federal support for small business R&D was addressed directly when Congress required coordination of the SBIR and STTR programs with the EPSCoR and IDeA programs in the SBIR/STTR Reauthorization Act of 2011.

1.2. NIGMS Funding for the SBIR and STTR Programs

Along with the IDeA program, NIGMS administers a substantial SBIR/STTR program. Among civilian agencies, NIH has the largest SBIR/STTR program (U.S. Small Business Administration, 2017c), and NIGMS has the fourth-largest SBIR/STTR grant program among NIH’s institutes and centers (NIGMS does not fund SBIR/STTR contracts, which are used to support research in specific areas defined by an agency and entail federal programmatic involvement during performance of the research).³ In FY 2017, as seen in Table 1, NIGMS funding for the SBIR and STTR programs totaled $72 million and $10 million, respectively. Since the programs’ inception through FY 2017 NIGMS SBIR/STTR funding totaled more than $1 billion.⁴

Table 1.

History of NIGMS funding for the SBIR/STTR Program.

FY	SBIR	STTR	Total
1983	$639,165	$0	$639,165
1984	$2,211,414	$0	$2,211,414
1985	$4,203,739	$0	$4,203,739
1986	$5,502,976	$0	$5,502,976
1987	$6,424,802	$0	$6,424,802
1988	$7,363,881	$0	$7,363,881
1989	$7,569,478	$0	$7,569,478
1990	$7,310,000	$0	$7,310,000
1991	$7,771,000	$0	$7,771,000
1992	$8,418,246	$0	$8,418,246
1993	$10,063,972	$0	$10,063,972
1994	$10,636,886	$394,596	$11,031,482
1995	$14,978,567	$691,854	$15,670,421
1996	$15,832,554	$1,175,000	$17,007,554
1997	$20,665,756	$1,354,000	$22,019,756
1998	$22,331,621	$1,344,609	$23,676,230
1999	$25,034,361	$1,482,228	$26,516,589
2000	$28,110,080	$1,728,984	$29,839,064
2001	$32,522,743	$1,952,000	$34,474,743
2002	$36,137,450	$2,163,000	$38,300,450
2003	$38,528,289	$2,318,000	$40,846,289
2004	$39,774,425	$4,680,661	$44,455,086
2005	$39,646,995	$4,761,913	$44,408,908
2006	$40,940,027	$4,742,333	$45,682,360
2007	$40,872,452	$5,824,630	$46,697,082
2008	$40,996,039	$4,923,964	$45,920,003
2009	$48,577,853	$6,971,938	$55,549,791
2010	$45,969,198	$5,834,470	$51,803,668
2011	$43,225,174	$5,174,645	$48,399,819
2012	$54,722,622	$7,420,000	$62,142,622
2013	$51,486,371	$6,987,929	$58,474,300
2014	$56,123,062	$7,392,145	$63,515,207
2015	$56,696,406	$8,043,856	$64,740,262
2016	$65,306,301	$9,600,810	$74,907,111
2017	$72,393,745	$10,067,404	$82,461,149
Total	$1,002,340,064	$104,563,561	$1,106,903,625

1.3. Measures of Success

Many previous studies of the SBIR program have been limited by their use of sales data as the primary measure of success (Cooper, 2003). We sought alternative measures to assess the outcomes of NIGMS’ investment, using several measures of innovation and firm success, including:

Awarded Patents:

A patent is often an early step in the path to commercialization of a product, and the ready availability of patent data has contributed to their widespread use to measure innovation. Although there are issues in the use of patent activity as a measure of productivity or innovation, some authors have shown patent activity to be an adequate surrogate for more nuanced measures of innovation (Acs & Audretsch, 1989), and patents are frequently a good predictor of technological and economic performance measured in other ways (Keller & Holland, 1982; Griliches, 1998; Hagedoorn & Cloodt, 2003; De Rassenfosse & Van Pottelsberghe de la Potterieb, 2009).

Patent Citations:

A second-order effect of SBIR/STTR funding can be observed in the “forward citation” of patents: the citation of SBIR/STTR-funded patents in subsequent downstream patent applications. Previous research has found this measure of the value of an innovation to be related to a variety of long-term outcomes (Trajtenberg, 1990; Lanjouw & Schankerman, 1999).

Firm Survival:

We determine each NIGMS SBIR/STTR-supported firm’s current business status: active, merged (or acquired), or inactive. While firm survival is but one of several potential measures of a firm’s success (e.g., ability to attract venture capital, financial performance, sales), we were unable to locate historical data for such measures that would span the 35-year history of the SBIR/STTR program. In this study, we use firm survival as a broad, multi-dimensional measure of success, a proxy for different finer-grain measures such as venture capital investment, financial performance and continued demand for products/services.

2.0. Data Sources

2.1 NIH Grants

The history of NIGMS support for the SBIR and STTR programs was drawn from NIH’s Information for Management Planning Analysis and Coordination (IMPAC) II database, an internal database of grant applications and awards maintained by NIH’s office of Electronic Research Administration. The IMPAC II database provided information on grantee institutions—including name, address, and geolocation information—and PIs. Additional publicly-available data were retrieved from NIH’s Research Portfolio Online Reporting Tools website (National Institutes of Health, 2018).

2.2. Patent Awards and Citations

Awarded patents were collected from two sources:

1. The U.S. Patent and Trademark Office (USPTO) Full-Text and Image Database (United States Patent and Trademark Office, 2018) was searched for references to NIGMS SBIR/STTR grants in the Government Interests section of each patent award. References to grants are found in the Government Interests section of awarded patents. An example of such a reference is shown in Figure 1.

2. Awarded patents reported by NIGMS SBIR/STTR grantees through the federal Interagency Edison (iEdison) reporting system. iEdison helps federal government grantees and contractors comply with the Bayh-Dole Act regulations requiring government-funded inventions be reported to the federal agency who made the grant or contract award. NIH has made its iEdison records for patent awards publicly available in its RePORTER database of grant awards (National Institutes of Health, 2016b) and as a bulk download through ExPORTER (National Institutes of Health, 2016a).

Figure 1.

The Government Interests section of patent no. 9,550,998, DNA plasmids with improved expression, citing support from an NIGMS SBIR grant.

We obtained information on forward citations of patents using the NIH’s Portfolio Analysis and Reporting Data Infrastructure, a non-public NIH database that combines NIH IMPAC II grants data with publication and patent records, including the Clarivate Analytics Derwent World Patents Index®.

Finally, we obtained ancillary information on NIGMS-supported patents from the U.S. Patent and Trademark Office’s Patents View database (United States Patent and Trademark Office, 2016). PatentsView provided useful information such as disambiguated patent assignees.

2.3. Firm Status (Mergers, Acquisitions, and Bankruptcies)

We used both publicly-available and proprietary databases to determine each NIGMS SBIR/STTR-supported firm’s current status: active, merged (or acquired), or inactive. Three sources were used for firm status: OpenCorporates (OpenCorporates, 2018), Crunchbase (Crunchbase, 2018) and a unique dataset purchased from Dun & Bradstreet (D&B) (Dun & Bradstreet, 2018). OpenCorporates is a database aggregator that combines information about companies from a variety of sources including government websites and APIs, publicly available datasets, and through Freedom of Information requests. Crunchbase is a crowd-sourced service that provides information on companies, investments, industry trends, and news about public and private companies. D&B is a well-known provider of commercial data and analytics. A combination of firm name, address, and DUNS numbers were used to ensure accurate matches among these data sources and the NIH grant records.

3.0. Results

3.1. The Geographic Distribution of NIGMS SBIR/STTR Funding

Figure 2 shows the geographic distribution of all NIGMS funding for SBIR/STTR through FY 2017. Awards were made in 48 states, the District of Columbia, and Puerto Rico. Alaska and South Dakota were the only states not to receive at least one award. The geographic concentration of NIGMS SBIR/STTR funding is similar to the Institute’s funding of traditional R01 research grants, and funding by state is highly correlated with composite scores of the Milken Institute 2018 State Technology and Science Index (Milken Institute, 2018); R=0.572, p < 0.01.

Figure 2.

Map showing the distribution of NIGMS SBIR/STTR funding through FY 2017.

The cumulative distribution of NIGMS’ SBIR/STTR and R01 funding for FY 1983-2017 by state is shown in Figure 3. Approximately one-half of NIGMS’ SBIR/STTR funding has gone to five states: California, Massachusetts, Wisconsin, Pennsylvania, and Texas.

Figure 3.

Cumulative distribution function of NIGMS SBIR/STTR and R01 funding in 1983-2017, by state, and the five states with the most NIGMS SBIR/STTR funding.

3.2. NIGMS SBIR/STTR Funding to IDeA States

Table 2 shows the number of Phase I applications submitted to NIGMS by businesses in IDeA states⁵ and the percentage of those applications that were funded in the five years prior to and after the most recent SBIR/STTR reauthorization in 2011, which required coordination of the SBIR/STTR and IDeA programs. Since the reauthorization, the success rate of applications (the percentage of reviewed grant applications that receive funding) from IDeA states has increased and been equal to that of applications from other states (about 16 percent, z=0.12, n.s.). As a result, the proportion of NIGMS SBIR/STTR awards going to IDeA states increased 23 percent, from 7.2 percent to 8.8 percent of all awards. A Mann-Whitney comparison of the peer review scores of applications received in 2006-2010 (Mdn = 42) and 2011-2015 (Mdn = 46) indicates that scores were not statistically significantly different (U = 4566.5, p > 0.05). This suggested that the increase in funding to IDeA states was driven more by increased programmatic emphasis on these underserved states than increased competitiveness of applications.

Table 2.

Applications, awards, and percent of applications funded from IDeA states and other states, FYs 2006-2010 (two-tailed z=3.66, p < 0.05) and 2011-2015 (two-tailed z=0.12, p=0.90).

IDeA State	Applications	Awards	% Funded
FY 2006-2010
Y	200	22	11.0%
N	1845	394	21.4%
FY 2011-2015
Y	218	35	16.1%
N	2207	361	16.4%

3.3. Correlates of the Geographic Distribution of SBIR/STTR Funding

As noted previously, several studies have found that the distribution of science-related awards tends to follow the geographic distribution of research capacity. We found the same to be true of NIGMS SBIR/STTR funding. Figure 4 shows a high correlation (0.93) between SBIR/STTR funding over the period 1983-2015 and the number of small firms (those with fewer than 500 employees) engaged in scientific R&D services (North American Industry Classification System code 5417) that were established in each state in 1999-2000, a two-year midpoint of this period (U.S. Census Bureau, 2002).

Figure 4.

Scatterplot of NIGMS 1983-2015 SBIR/STTR funding and number of small firms involved in scientific R&D services established in 1999-2000, by state (r = 0.93).

To measure the association between SBIR/STTR funding and local academic research activity supported with other grant funding from NIH, we determined a) the geographic proximity of SBIR/STTR grantees to institutions of higher education (IHEs) that are major recipients of NIH funding and b) whether the PI of an SBIR/STTR grant was affiliated with a nearby university and had applied for other types of NIH research grants. For the former, we identified 105 “NIH IHEs” that received 80% of NIH research grant funding in 2017. (See Appendix A for a list of these institutions.)

1. To determine the geographic proximity of SBIR/STTR firms to one of the NIH IHEs, addresses of the firms and the NIH IHEs were converted to latitude and longitude and the great circle distances between these points were calculated. SBIR/STTR firms were highly likely to be located near an IHE with support from the NIH. Over 90 percent of firms were located within 50 miles of one of these major recipients of NIH funding (Figure 5). Furthermore, SBIR/STTR firms were more likely to be concentrated near one of the 105 major IHEs than were other IHEs that are not among the 105 major IHEs. Only 60 percent of NIH-supported IHEs not among the group of major recipients were located within 50 miles of one of the 105 NIH IHEs. NIH-supported SBIR/STTR firms in IDeA states were even less likely to be near one of these major IHEs; only 25 percent were within 50 miles of an NIH IHE.

2. A total of 1,758 PIs have received NIGMS SBIR/STTR grants. Of these investigators, 148 also were PIs on other NIH research grant applications submitted from nearby IHEs within three years of the SBIR/STTR award. As a result, of the 1,196 SBIR/STTR firms, 154 (12.9 percent) could be linked to a university through the PIs’ other research activities.

Figure 5.

Distribution of SBIR/STTR firms by distance to nearest of 105 IHEs receiving major support from NIH. Distribution of distances of other NIH-supported IHEs to these 105 major recipient institutions provided for comparison.

As described in section 1.0, relationships between small businesses and non-profit research organizations (typically universities) also are critical components of STTR grants which require such partnerships. While SBIR/STTR grantees with at least one STTR award were more likely than SBIR grantees with no STTR awards to have an academically-affiliated PI (35.6% vs. 9.3%, see Table 3) or be in close proximity to an NIH IHE, there were exceptions. Most STTR firms (105 out of 163) did not have any academically-affiliated PI (as defined in this study), and Figure 6 highlights STTR grantees on the distance distribution from Figure 5 to show that many of these firms were located considerable distances from the nearest NIH IHE.

Table 3.

STTR grants with and without an academically-affiliated principal investigator (two-tailed z=9.31, p < 0.05).

Academic PI	STTR Grant Recipient
	Y	N (SBIR Only)	Total
Y	58 (35.6%)	96 (9.3%)	154
N	105 (64.4%)	937 (90.7%)	1,042
Total	163	1033	1,196

Figure 6.

From Figure 5, distribution of all SBIR/STTR firms by distance to nearest institution of higher education receiving major support from NIH (see Appendix A), highlighting those receiving STTR support.

3.4. Correlates of SBIR/STTR Outcomes

We investigated the associations between geographic location and several measures of innovation and a broad measure of firm success: the number of patents citing NIGMS SBIR/STTR support each firm received, the number of downstream patent citations these patents received, and the long-term survival of the firm. Based on previous research finding spillover effects of local universities on private sector innovation, we tested hypotheses that firms co-located with academic institutions or having academic affiliations would have more positive outcomes than firms lacking such relationships.

Awarded Patents

The number of patents supported by NIGMS SBIR/STTR grants, as a function of the firms’ first year of Phase I NIGMS support is shown in Figure 7. There is a lag between first NIGMS support and the award of any patents resulting from that support. Not enough time has elapsed for grant cohorts after FY 2008 to have complete patent data, so they were excluded from subsequent analyses.

Figure 7.

Number of patents citing NIGMS SBIR/STTR support, by firm’s first year of support, and FY 2007 cutoff for cohorts used in the analysis.

The percentage of firms having received an SBIR/STTR-supported patent was higher among firms having a PI affiliated with an IHE that receives major NIH support (20.6 percent) than it was among other firms (11.3 percent) (one-sided z=2.641, p < 0.05). Firms with at least one STTR award also were more likely to have received a patent: 21.0 percent of STTR-funded firms received at least one patent vs. 11.1 percent of firms with only an SBIR award (one-sided z=2.873, p < 0.05). Patenting activity was unrelated to geographic distance from an NIH IHE and there were no differences between 66 firms located in IDeA states and 833 firms located in other states.

The higher rate of patenting by STTR recipients and firms with academic-affiliated PIs may be a result, in part, of the availability of an established university technology transfer infrastructure to facilitate and support the patent process. To support this, universities were found to be much more prevalent among the assignees of patents associated with these firms, as shown in tables 4a and 4b.

Table 4a.

Patents generated by firms with academic-affiliated principal investigators and the presence of an academic institution among the patent assignees (two-sided z=10.09, p < 0.05).

Academic PI	Academic Assignee
	Y	N	Total
Y	27 (49.1%)	28 (50.9%)	55 (100%)
N	9 (3.2%)	274 (96.8%)	283 (100%)
Total	36	302	338

Table 4b.

Patents generated by firms with an STTR award and the presence of an academic institution among the patent assignees (two-sided z=13.95, p < 0.05).

STTR Grantee	Academic Assignee
	Y	N	Total
Y	29 (76.3%)	9 (23.7%)	38 (100%)
N (SBIR Only)	7 (2.3%)	293 (97.7%)	300 (100%)
Total	36	302	338

Patent Citations

The distribution of SBIR/STTR-generated patents by number of citations is shown in Figure 8. Self-citations (citations in which at least one of the patent assignees of the citing and cited patents are the same) have been removed to provide a more valid indicator of a patent’s value to outside firms.

Figure 8.

Distribution of NIGMS SBIR/STTR-generated patents, by number of times cited (excludes self-citations).

Just as there is a lag between initial NIGMS funding and patent awards, shown in Figure 7, there is a lag between patent award and downstream citations. To allow time for patent citations to accrue, it was necessary to exclude firms whose first year of NIGMS support was later than 2002. Unlike the analysis of number of patents awarded, patents arising from firms having an NIH IHE-affiliated PI or from STTR grants did not have more citations than patents from other firms. Also, there were no differences between firms in IDeA states and other states. All subgroups analyzed had the same median number of patent citations (2.0 citations). There also was no association with distance from an NIH IHE, as seen in Figure 9.

Figure 9.

Scatterplot of firms’ patent citations and distance to nearest institution of higher education with major NIH support (excludes self-citations).

Firm Survival

Analyses of firm status were performed on 313 firms receiving their first NIGMS support in FY 1983-1997 whose current status could be determined. (Analyses showed that most firms funded in cohorts since 1998 are still active [data not shown].)

To examine the association between firm status and geographic proximity to one of the 105 NIH IHEs, we first defined a population of firms to be considered “proximal.” Geographic distance to an NIH IHE was converted to a dichotomous variable by establishing a distance cutoff within which firms were considered “proximal” to an NIH IHE. Firm status also was converted into a dichotomous survival measure by collapsing firms that were either active, had merged with another firm, or been acquired into a single group and compared this group to firms who were no longer active. To establish a cutoff with the greatest discriminatory power, chi-square statistics were used to compare the strength of the relationship between firm survival and proximity using several different cutoffs for proximity. For example, as shown in Figure 10, when proximal firms were defined as those within 10 miles of an IHE, there was only a weak relationship between proximity and firm survival (the chi-square statistic was close to zero). The chi-square statistic reached a maximum with a cutoff of 50 miles. That is, firm survival was predicted best by proximity to an NIH IHE when “proximal” was defined as a firm located within 50 miles of an NIH IHE. Using this definition of proximity, the distributions of firm status for proximal and non-proximal firms are shown in Figure 11.

Figure 10.

Value of chi-square statistic using several different cutoffs to identify firms as “proximal” to NIH-funded institutions of higher education.

Figure 11.

Distribution of firm status for firms within and greater than 50 miles of an institution of higher education with major NIH research grant funding (one-sided z=1.78, p<0.05).

The associations of firm status to two other indicators of academic affiliation are shown in Figures 12 and 13. As hypothesized based on previous findings, the presence of a PI who applied for NIH research grant funding and was affiliated with an academic institution was associated with a more positive firm outcome in the population of former grantees (Figure 12), as was receipt of an STTR award, which requires involvement of a non-profit research organization (Figure 13).

Figure 12.

Distribution of firm status for firms with and without a principal investigator who applied for NIH funding and was associated with an academic institution (one-sided z=1.77, p<0.05).

Figure 13.

Distribution of firm status for firms with at least one STTR award and those with only an SBIR grant (one-sided z=2.22, p < 0.05).

We also examined the association between firm status and two state-level variables: the existence of an SBIR/STTR matching program, as reported in Lanahan and Feldman (2015), and location in an IDeA state. SBIR matching programs are state-level policies that provide matching funds for successful SBIR recipients to support technical aspects of their projects. No a priori hypotheses were made about the direction of these associations. We found the existence of matching programs to be related to firm status, with firms located in states having matching programs more likely to be inactive (Figure 14). There were no significant differences in firm status for firms located in IDeA states and other states.

Figure 14.

Distribution of firm status for firms in states with and without state-sponsored SBIR matching programs (two-sided z=2.48, p < 0.05).

4.0. Discussion

The findings described in this paper constitute the first comprehensive analysis undertaken by NIGMS to examine the geographic distribution of its SBIR/STTR funding and assess its relationship to outcomes associated with investment in the SBIR/STTR portfolio. Such an assessment, including the quantification of outcomes, is an important part of the Institute’s stewardship function, given the considerable investment in the program over the time periods examined and described in this paper.

We found the geographic distribution of NIGMS SBIR/STTR funding to be highly concentrated in a few states, and to a degree similar to the concentration of all NIH extramural funding for R01 research grants. There was a high correlation between each state’s NIGMS SBIR/STTR funding and the state’s number of small scientific research and development businesses. This result suggests that funding is driven by demand, consistent with previous findings that state funding for SBIR is highly correlated with the number of proposals submitted (U.S. Government Accountability Office, 1999).

Our findings are consistent with earlier analyses showing that research output and economic activity in R&D, particularly in knowledge-based fields, tends to cluster in geographic regions where knowledge can be generated and shared more efficiently (Audrestsch, 2003, Audretsch & Feldman, 1996; Chatterjee & DeVol, 2012; Elsevier and the Council of State Governments, 2015; Zucker, Darby, & Armstrong, 2001; Zucker, Darby, & Brewer, 1994), especially when such knowledge is being shared through informal networks (Audretsch & Stephan, 1996).

SBIR/STTR firms were likely to be located in close geographic proximity to one of 105 major NIH-supported universities and, as shown in Figure 5, they were closer to these institutions than other recipients of NIH funding.

Among the population of all NIGMS-supported SBIR/STTR grantees examined in this study, affiliation with a major research university through mere geographic proximity, or more directly through the role of PIs who are university faculty or through STTR partnerships, was associated with several measures of innovation and firm success. (Such associations within smaller subsets of firms, such as the subset of firms from IDeA states, did not reach statistical significance.) Firms with academic PIs were nearly twice as likely to patent the results of their SBIR/STTR research, and firms employing these PIs were more likely to remain active as entities or merged with other R&D firms. The same was true of firms engaged in STTR partnerships with academic institutions. Geographic proximity to major NIH-supported universities also was associated with firm survival. We found these associations to be strongest for firms located within 50 miles of a major NIH-supported university, consistent with other research based on commuting patterns that found significant spillover effects of university research extended within a 50-mile radius, but not a larger 75-mile radius (Acs, Anselin, & Varga, 2002) and a relationship between life science firms’ SBIR funding and the presence of universities conducting life sciences research within the same metropolitan statistical area (Kolympiris & Kalaitzandonakes, 2013).

We found firm survival to be lower within states identified by Lanahan and Feldman (2015) as having an SBIR/STTR matching program designed to provide additional support for SBIR/STTR grantees. There are several possible reasons for this counter-intuitive finding. Lanahan and Feldman identified states based on the existence of matching programs as of 2013 and it’s not known whether these matching programs existed when the firms included in the survival analysis were funded by NIGMS. (The survival analysis was restricted to cohorts whose first support began prior to 1998.) The firms included in the survival analysis would not have been able to benefit from any state matching program that began after their NIGMS support ended. If matching programs had been available to these cohorts of grantees, and while matching programs may have a positive impact within states that have implemented them, lower rates of survival relative to firms in other states would be consistent with one of the findings of Lanahan and Feldman; states with matching programs lag other states in high tech employment. These authors suggested that some states may have implemented their program “as a proactive, catch-up effort” motivated, in part, by a desire to compensate for and overcome an already weak technology and innovation environment within the state.

4.1. Lessons Learned

The use of existing datasets to evaluate a limited number of grantees presented some unique challenges. Matching records from diverse sources without common identifiers required a high level of data quality control and in-depth review of the assembled data prior to analysis. In this study, reliable matches—for example, between a firm’s grant records and its records in business intelligence databases—were ensured by using multiple criteria to ensure a successful match. Any ambiguous cases were flagged and in-depth research targeted to resolving them.

Working with a limited number of programs, there may be insufficient statistical power to identify differences between small subsets of grantees. For example, the geographic variables related to firm survival were not found to be related to forward patent citations, a measure sometimes used to represent the quality of an invention. To allow adequate time for follow-up, the forward citation analysis was limited to firms whose first NIGMS award was in FY 2002 or earlier. This included only 85 firms with at least one patent and, of these, only 74 had at least one forward citation. This may deserve study in the future with a larger population of SBIR/STTR grantees. Data from a larger population may reveal effects obscured in this limited dataset.

Finally, the existing data sources used in this study did not provide the granularity required for a deeper analysis of the multiple factors that contribute to the success of small R&D firms and the variety of ways success might be measured. More information on the history of these firms and the economic climate in which they existed might have allowed us to more clearly attribute outcomes to such factors as the investment in these firms made by NIGMS and their proximity to academic institutions. While partial information was available on a small subset of the firms studied (e.g., we found several existing sources of information on venture capital investments), it was not available on a large enough number of firms for the 35-year period over which these firms were studied to permit a more rigorous analysis. As a result, we were limited to a broad measure of firm success—firm survival—which served as a proxy for a number of fine-grained outcome measures such as venture capital investment, financial performance and continued demand for their products/services. Even so, we believe the ability to quantify this one measure represents a significant improvement over survey data, which has been the major data source in previous evaluations of the SBIR/STTR programs.

Thus, these analyses have brought to light potential policy interventions that the NIGMS and potentially other institutes and centers at the NIH can implement to address the barriers discovered in this work. More extensive performance data collection would provide information on business performance (e.g., sales figures, for the firm and specific to the NIGMS-supported inventions; employment numbers; venture capital investments and other resources the firm is able to attract) and intellectual property (inventions, patents, and licensing). Some of this information, including descriptions of new technologies and products created, sales, licensing revenue, third party investments, and commercialization activities, is now being collected from SBIR/STTR grantees using the most recent version of NIH’s annual Research Performance Progress Report (RPPR; see https://grants.nih.gov/grants/guide/notice-files/NOT-OD-14-092.html). This new RPPR was implemented in FY 2014, too late to be used in the current study. While there remains no mechanism for collecting such data after NIGMS grant support ends, the partial information collected in the new RPPR should facilitate future evaluations of the SBIR/STTR program.

4.2. Conclusion

The SBIR and STTR programs are important in supporting small businesses, stimulating innovation, and catalyzing regional economic growth. Empirical evidence suggests that the NIH programs have a positive net economic benefit (Allen, Layson, & Link, 2012). By focusing on the transfer of technologies within the biomedical research community, they complement NIGMS’s other research programs. NIGMS continuously monitors outcomes of its SBIR/STTR investments. This work has elucidated two major points. The first is that policy interventions are needed to ensure more rigorous and conclusive studies. We have identified key standard reporting requirements from SBIR/STTR grantees that will be needed to generate better outcome analyses. Using the new RPPR, we will be collecting some of the more detailed information on the firms supported, their interactions with academic institutions and other research partners, and the temporal relationships among NIGMS’s funding, technology development, and business performance. Data of this nature will allow us to perform more rigorous and conclusive studies of impact, thus allowing for more robust administration of the program and maximization of taxpayer investments.

Secondly, this study reveals the association of successful business outcomes to academic affiliations and proximity to one of 105 major NIH-supported universities. In recent years, SBIR/STTR applicants from IDeA and non-IDeA states have been equally successful in obtaining funding and this has increased the share of NIGMS’s SBIR/STTR funding going to IDeA states. Yet a majority of IDeA states SBIR/STTR firms were greater than 50 miles from one of 105 major NIH IHEs. While many of these small businesses cannot relocate to be closer to a major NIH IHE, the institute is more confident that programs in IDeA states will be enhanced by new Regional Technology Transfer Accelerator Hubs for IDeA States funded through the NIGMS STTR program in September, 2018 (National Institute of General Medical Sciences, 2017a). The small business concerns selected to lead all four of these regional hubs (see https://www.nigms.nih.gov/Research/mechanisms/Pages/STTR-Regional-Technology-Transfer.aspx) are located within 13 miles of an NIH IHE, and all have established formal relationships with at least one NIH IHE in their region. Each is partnering with several other academic institutions in their region to develop a tech transfer infrastructure and promote an entrepreneurial culture within each region. Follow-up over an extended period can determine whether R&D supported in these states is equally successful with respect to outcomes.

Highlights:

• NIGMS SBIR/STTR funding is highly concentrated in a few states

• Each state’s funding is correlated with its number of small R&D businesses

• R&D and innovation occurs in geographic regions where knowledge can be shared

• Affiliations with research universities drive innovation and firm success

Biographies

Author Biographies:

James Onken is President of Research Enterprise Analytics (REA), specializing in the evaluation of research programs and the analysis of federal research policies. Prior to founding REA, he worked for the National Institutes of Health for 27 years, holding senior positions performing program analysis and developing the NIH data infrastructure for research portfolio analysis.

Richard Aragon is chief of the OPAE, where he advises on the Institute’s strategic planning activities, analyzes and evaluates its research and training programs, and serves as its legislative liaison. Prior to joining NIGMS, Richard held two positions in the Office of the Assistant Secretary for Financial Resources within the Office of the HHS Secretary: directing the Division of Program Integrity Assessment, Integration, and Oversight and the Division of Outreach, Communications, and Training.

Anna Maria Calcagno is the deputy chief of the OPAE. Prior to joining the NIGMS, she served as the data analytics team lead within the Office of Program Integrity Coordination at Department of Health and Human Services where she worked on one of the HHS Secretary’s Strategic Priorities. Dr. Calcagno’s over decade-long career of federal service began as a health science administrator at the National Cancer Institute (NCI).

Appendix A:

Institutions of Higher Education Among Organizations Receiving 80 Percent of NIH Research Grant Funding in 2017. (Source: NIH Awards by Location and Organization. Retrieved November 2, 2017, from https://report.nih.gov/award/index.cfm.)

Institution of Higher Education	City	State	2017 Research Grant Funding
JOHNS HOPKINS UNIVERSITY	BALTIMORE	MD	$611,624,029
UNIVERSITY OF CALIFORNIA, SAN FRANCISCO	SAN FRANCISCO	CA	$563,373,054
UNIVERSITY OF MICHIGAN	ANN ARBOR	MI	$486,978,225
UNIVERSITY OF PITTSBURGH AT PITTSBURGH	PITTSBURGH	PA	$461,143,961
UNIVERSITY OF PENNSYLVANIA	PHILADELPHIA	PA	$457,252,376
STANFORD UNIVERSITY	STANFORD	CA	$441,233,697
WASHINGTON UNIVERSITY	SAINT LOUIS	MO	$414,663,179
YALE UNIVERSITY	NEW HAVEN	CT	$404,614,472
UNIVERSITY OF CALIFORNIA SAN DIEGO	LA JOLLA	CA	$401,785,592
DUKE UNIVERSITY	DURHAM	NC	$399,520,947
UNIVERSITY OF WASHINGTON	SEATTLE	WA	$399,397,746
UNIV OF NORTH CAROLINA CHAPEL HILL	CHAPEL HILL	NC	$384,726,185
COLUMBIA UNIVERSITY HEALTH SCIENCES	NEW YORK	NY	$382,513,159
UNIVERSITY OF CALIFORNIA LOS ANGELES	LOS ANGELES	CA	$375,821,596
ICAHN SCHOOL OF MEDICINE AT MOUNT SINAI	NEW YORK	NY	$297,840,958
EMORY UNIVERSITY	ATLANTA	GA	$295,311,672
UNIVERSITY OF WISCONSIN-MADISON	MADISON	WI	$277,394,669
UNIVERSITY OF SOUTHERN CALIFORNIA	LOS ANGELES	CA	$250,785,902
NORTHWESTERN UNIVERSITY AT CHICAGO	CHICAGO	IL	$238,970,334
UNIVERSITY OF CALIFORNIA AT DAVIS	DAVIS	CA	$227,631,491
UNIVERSITY OF MINNESOTA	MINNEAPOLIS	MN	$226,519,641
NEW YORK UNIVERSITY SCHOOL OF MEDICINE	NEW YORK	NY	$224,371,082
UNIVERSITY OF ALABAMA AT BIRMINGHAM	BIRMINGHAM	AL	$223,572,439
BAYLOR COLLEGE OF MEDICINE	HOUSTON	TX	$216,557,657
OREGON HEALTH & SCIENCE UNIVERSITY	PORTLAND	OR	$210,207,221
UNIVERSITY OF COLORADO DENVER	AURORA	CO	$204,078,872
HARVARD MEDICAL SCHOOL	BOSTON	MA	$196,983,424
UT SOUTHWESTERN MEDICAL CENTER	DALLAS	TX	$171,142,189
CASE WESTERN RESERVE UNIVERSITY	CLEVELAND	OH	$166,384,689
OHIO STATE UNIVERSITY	COLUMBUS	OH	$165,559,918
UNIVERSITY OF UTAH	SALT LAKE CITY	UT	$156,897,313
UNIVERSITY OF CHICAGO	CHICAGO	IL	$155,826,951
UNIVERSITY OF FLORIDA	GAINESVILLE	FL	$153,732,547
UNIVERSITY OF MARYLAND BALTIMORE	BALTIMORE	MD	$151,573,993
UNIV OF MASSACHUSETTS MED SCH WORCESTER	WORCESTER	MA	$148,429,960
UNIVERSITY OF ROCHESTER	ROCHESTER	NY	$142,398,381
UNIVERSITY OF TX MD ANDERSON CAN CTR	HOUSTON	TX	$141,360,384
INDIANA UNIV-PURDUE UNIV AT INDIANAPOLIS	INDIANAPOLIS	IN	$141,197,642
UNIVERSITY OF IOWA	IOWA CITY	IA	$132,859,305
UNIVERSITY OF VIRGINIA	CHARLOTTESVILLE	VA	$132,157,106
WEILL MEDICAL COLL OF CORNELL UNIV	NEW YORK	NY	$125,793,089
BOSTON UNIVERSITY MEDICAL CAMPUS	BOSTON	MA	$122,842,783
UNIVERSITY OF CALIFORNIA BERKELEY	BERKELEY	CA	$118,702,011
HARVARD SCHOOL OF PUBLIC HEALTH	BOSTON	MA	$118,452,749
MEDICAL UNIVERSITY OF SOUTH CAROLINA	CHARLESTON	SC	$115,297,716
UNIVERSITY OF MIAMI SCHOOL OF MEDICINE	CORAL GABLES	FL	$115,064,118
UNIVERSITY OF ILLINOIS AT CHICAGO	CHICAGO	IL	$113,129,810
UNIVERSITY OF SOUTH FLORIDA	TAMPA	FL	$112,773,906
UNIVERSITY OF CALIFORNIA-IRVINE	IRVINE	CA	$111,608,119
UNIVERSITY OF KENTUCKY	LEXINGTON	KY	$105,471,272
GEORGE WASHINGTON UNIVERSITY	WASHINGTON	DC	$104,090,173
MASSACHUSETTS INSTITUTE OF TECHNOLOGY	CAMBRIDGE	MA	$103,811,355
CLEVELAND CLINIC LERNER COM-CWRU	CLEVELAND	OH	$102,569,349
WAKE FOREST UNIVERSITY HEALTH SCIENCES	WINSTON-SALEM	NC	$100,847,115
UNIVERSITY OF TEXAS HLTH SCI CTR HOUSTON	HOUSTON	TX	$90,946,585
UNIVERSITY OF ARIZONA	TUCSON	AZ	$90,710,247
DARTMOUTH COLLEGE	HANOVER	NH	$88,754,340
UNIVERSITY OF TEXAS, AUSTIN	AUSTIN	TX	$86,897,792
VANDERBILT UNIVERSITY	NASHVILLE	TN	$83,921,690
BROWN UNIVERSITY	PROVIDENCE	RI	$77,754,388
UNIVERSITY OF TEXAS MED BR GALVESTON	GALVESTON	TX	$75,909,419
MEDICAL COLLEGE OF WISCONSIN	MILWAUKEE	WI	$73,955,406
VIRGINIA COMMONWEALTH UNIVERSITY	RICHMOND	VA	$73,302,602
ROCKEFELLER UNIVERSITY	NEW YORK	NY	$72,763,129
TEMPLE UNIV OF THE COMMONWEALTH	PHILADELPHIA	PA	$72,610,624
UNIVERSITY OF NEBRASKA MEDICAL CENTER	OMAHA	NE	$72,600,702
CORNELL UNIVERSITY	ITHACA	NY	$71,630,896
HARVARD UNIVERSITY	CAMBRIDGE	MA	$66,050,580
TULANE UNIVERSITY OF LOUISIANA	NEW ORLEANS	LA	$62,180,449
STATE UNIVERSITY NEW YORK STONY BROOK	STONY BROOK	NY	$62,128,731
PENNSYLVANIA STATE UNIVERSITY-UNIV PARK	UNIVERSITY PARK	PA	$61,553,037
CALIFORNIA INSTITUTE OF TECHNOLOGY	PASADENA	CA	$60,470,771
NEW YORK UNIVERSITY	NEW YORK	NY	$60,222,582
UNIVERSITY OF CINCINNATI	CINCINNATI	OH	$58,833,071
UNIVERSITY OF TEXAS HLTH SCIENCE CENTER	SAN ANTONIO	TX	$58,140,588
UNIVERSITY OF LOUISVILLE	LOUISVILLE	KY	$57,788,441
UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN	CHAMPAIGN	IL	$57,117,745
MICHIGAN STATE UNIVERSITY	EAST LANSING	MI	$56,872,808
STATE UNIVERSITY OF NEW YORK AT BUFFALO	AMHERST	NY	$55,298,965
UNIVERSITY OF NEW MEXICO HEALTH SCIS CTR	ALBUQUERQUE	NM	$55,188,513
WAYNE STATE UNIVERSITY	DETROIT	MI	$53,871,253
RUTGERS, THE STATE UNIV OF N.J.	PISCATAWAY	NJ	$52,472,134
UNIVERSITY OF GEORGIA	ATHENS	GA	$52,177,314
TUFTS UNIVERSITY BOSTON	BOSTON	MA	$52,054,881
UNIVERSITY OF KANSAS MEDICAL CENTER	KANSAS CITY	KS	$51,819,911
THOMAS JEFFERSON UNIVERSITY	PHILADELPHIA	PA	$50,986,388
RUSH UNIVERSITY MEDICAL CENTER	CHICAGO	IL	$50,072,197
AUGUSTA UNIVERSITY	AUGUSTA	GA	$48,645,331
ARIZONA STATE UNIVERSITY-TEMPE CAMPUS	TEMPE	AZ	$48,313,946
GEORGETOWN UNIVERSITY	WASHINGTON	DC	$47,600,345
UNIV OF MARYLAND, COLLEGE PARK	COLLEGE PARK	MD	$45,848,921
PENNSYLVANIA STATE UNIV HERSHEY MED CTR	HERSHEY	PA	$45,283,284
UNIVERSITY OF VERMONT & ST AGRIC COLLEGE	BURLINGTON	VT	$44,735,071
UNIVERSITY OF CONNECTICUT SCH OF MED/DNT	FARMINGTON	CT	$44,287,984
PURDUE UNIVERSITY	WEST LAFAYETTE	IN	$41,776,208
UNIVERSITY OF MISSOURI-COLUMBIA	COLUMBIA	MO	$41,214,040
RBHS-NEW JERSEY MEDICAL SCHOOL	NEWARK	NJ	$40,933,482
UNIVERSITY OF OKLAHOMA HLTH SCIENCES CTR	OKLAHOMA CITY	OK	$40,706,837
UNIVERSITY OF HAWAII AT MANOA	HONOLULU	HI	$40,469,049
UNIVERSITY OF COLORADO	BOULDER	CO	$40,304,768
PRINCETON UNIVERSITY	PRINCETON	NJ	$39,639,150
NORTHWESTERN UNIVERSITY	EVANSTON	IL	$37,024,006
UNIV OF ARKANSAS FOR MED SCIS	LITTLE ROCK	AR	$36,687,339
BOSTON UNIVERSITY (CHARLES RIVER CAMPUS)	BOSTON	MA	$36,532,690
DREXEL UNIVERSITY	PHILADELPHIA	PA	$35,545,285