Abstract
Objective
To evaluate the comparability of commercially available practice site data from SK&A with survey data to understand the implications of using SK&A data for health services research.
Data sources
Responses to the Comprehensive Primary Care Plus (CPC+) Practice Survey and SK&A data.
Study design
Comparison of CPC + Practice Survey responses to SK&A information for 2698 primary care practice sites.
Data collection
CPC + Practice Survey data collected through a web‐only survey from April through September 2017, and SK&A data purchased in November 2016.
Principal findings
Information was similar across data sources, although some discrepancies were common. For example, 56% of practice sites had differences in the reported number of practitioners, and larger sites tended to have larger differences. Among practice sites with 1 practitioner in the survey, only 1.3% had a difference of 3 or more practitioners between the data sources, whereas 63% of practice sites with 11 or more practitioners had a difference of 3 or more practitioners.
Conclusions
Discrepancies between data sources could reflect differences of interpretation when defining practice site characteristics, changes over time in those characteristics, or data errors in either SK&A or the survey. Researchers using SK&A data should consider possible ramifications for their studies.
Keywords: data accuracy, data collection, primary health care, research design, surveys and questionnaires
What is Already Known on this Topic
SK&A data were designed as a source of contact information for healthcare marketing, but they have been increasingly used for research purposes.
Researchers have used SK&A to analyze a range of topics, including changes in practice composition, hospital‐physician consolidation, and adoption of electronic health records (EHRs).
A previous study has shown that physician contact information in SK&A is reasonably up to date.
What this Study Adds
No known studies have compared SK&A information about practice site characteristics to a primary data source.
This study shows that information was similar between the data sources, but some discrepancies were common.
This study describes possible advantages and limitations of using SK&A data in health services research.
1. INTRODUCTION
SK&A databases, which are managed by IQVIA—a health care data vendor—contain data on health care providers, including practitioners, hospitals, and accountable care organizations, and their relationships. The SK&A databases were primarily designed for marketing purposes, but they have been a key source for health services research that requires information on practice sites, or a group of practitioners that are part of the same practice and work at the same physician location, as this information typically is not available in administrative data such as insurance claims. Researchers have used SK&A data to examine changes in practice composition, 1 hospital‐physician consolidation, 2 , 3 medical group acquisitions, 4 and adoption of electronic health records (EHRs). 5 Researchers have also used SK&A data to identify comparison groups for evaluations of practice transformation initiatives. 6 , 7 IQVIA recently introduced a new database called OneKey that integrates data from multiple sources, including SK&A. Although OneKey largely replaces SK&A as a standalone data source, historical SK&A data are still available and researchers may use SK&A data for historical analyses.
Using SK&A data offers researchers advantages by reducing time and money spent on primary data collection. However, while a previous study has shown that physician contact information in SK&A is reasonably up to date, 8 a study comparing SK&A data on practice ownership patterns to survey data from the American Hospital Association found some disagreement between the sources. 3 To the best of our knowledge, no studies have evaluated the comparability of SK&A data about practice site characteristics to researcher‐collected primary data.
This study fills that gap by comparing SK&A data and survey data collected from practice sites for the evaluation of Comprehensive Primary Care Plus (CPC+). The Centers for Medicare & Medicaid Services (CMS), alongside 79 other payers, launched CPC+ to improve the comprehensiveness and quality of US primary care and improve patient outcomes (the supplemental materials provide additional information on CPC+). 9
Both data sources may have errors, as both rely on responses from office staff. Nevertheless, comparing these data sources is useful for understanding the implications of using SK&A data in health services research. In particular—even though concordant responses are not guaranteed to be accurate—discordant responses indicate that practice site characteristics are not measured uniformly across data sources or reflect changes over time. Even seemingly simple concepts, such as the number of practitioners, may be more slippery than researchers realize.
2. METHODS
2.1. Data/sample
This analysis examines data from an SK&A file purchased in November 2016, and from the 2017 CPC+ Practice Survey. The SK&A file covered 360 440 practitioners (physicians, nurse practitioners [NP], and physician assistants [PA], regardless of specialty) working at practice sites in the United States that employed at least one practitioner with a primary care specialty, which we defined to include family medicine, general medicine, geriatric medicine, or internal medicine. An SK&A ID identifies practice sites and their practitioners. SK&A states that data are telephone‐verified by practice site office staff and may be up to six months old at the time of purchase. 10 (ie, the data from November 2016 were verified between May and November 2016, depending on the practice site.) The CPC+ Practice Survey was designed as a web‐only survey by Mathematica, to collect information on site characteristics and care delivery approaches. It was fielded April through September 2017, meaning that there is a 4‐month (November 2016 to April 2017) to a 15‐month (May 2016 through September 2017) difference in the timing of data collection compared to SK&A. CMS required CPC+ practice sites to respond, and the response rate was over 99 percent. Survey responses reflect practice site characteristics at the time of response.
The analysis examines 2698 practice sites that began CPC+ in 2017 and (a) were part of the population used in the independent evaluation of CPC+, (b) could be linked to a unique SK&A ID in the November 2016 SK&A file based on the name, address, and National Provider Identifiers (NPIs) listed in CPC+ application data, and (c) responded to the 2017 CPC+ Practice Survey. This represents 93.4 percent of the practice sites that started CPC+ in 2017. We describe the characteristics of practice sites included in the analysis, exclusions (Figure S1), and data sources in more detail in the supplemental materials. The analysis represents approximately 4 percent of the 360 440 practitioners in the SK&A file (practitioners of any specialty in practice sites that provide primary care).
2.2. Outcomes
We analyzed whether survey responses aligned with SK&A information about: (a) number of practitioners at the practice site; (b) practitioner composition, including the percentage of practitioners with a primary care specialty and whether the practice site provided any specialty services; and (c) use of EHRs. Although both data sources cover additional information, we limited the analysis to these three areas where data were directly comparable. Practitioners who worked at multiple practice sites were considered part of all affiliated sites in both data sources. We defined key terms for the analysis in Table 1. Table S2 shows the survey questions.
TABLE 1.
Definitions of terms
| Term | SK&A definition | CPC+ Practice survey definition |
|---|---|---|
| Practitioners | Physicians, NPs, or PAs, regardless of specialty | Physicians, NPs, or PAs, regardless of specialty |
| Primary care practitioners | Practitioners with a primary specialty flag for family medicine, general medicine, geriatric medicine, or internal medicine | Primary care practitioners with a primary specialty of family medicine, internal medicine, or geriatric medicine, practicing under their own NPI |
| Percentage of practitioners with a primary care specialty | Number of PCPs at the practice site divided by the total number of practitioners | Number of PCPs at the practice site divided by the total number of practitioners |
| Multispecialty |
Definition 1—Practice site had more total practitioners than PCPs Definition 2—Practice site has a specialty flag for “multispecialty” |
Practice site reports more practitioners than PCPs |
| EHR use | Practice site reports using an EHR | Practice site reports using an EHR |
Abbreviations: EHR, electronic health record; NP, nurse practitioner; NPI, National Provider Identifier; PA, physician assistant; PCP, primary care practitioner.
2.3. Analysis
For each outcome, we assessed the proportion of responses that were concordant between the two data sources, and where conceptually relevant, the magnitude and direction of discrepancies, based on the size and percentage difference relative to the mean of the SK&A and survey responses. For example, if a practice site reported 4 practitioners on the survey and 2 in SK&A, the difference is −67 percent: ((2‐4)/([2 + 4]/2)).
3. RESULTS
3.1. Number of practitioners
For over half of the practice sites, the SK&A and survey responses differed for the total number of practitioners and for the number of PCPs, although differences were generally small (Table 2). Differences in the number of practitioners by type—physicians and NPs or PAs—were also common.
TABLE 2.
Number of practice sites with differences in practitioner counts between SK&A and CPC+ Practice Survey
| Type of practitioner | All practice sites (N = 2698) | Practice sites with up to 7 practitioners, according to survey (N = 2018) | Practice sites with more than 7 practitioners, according to survey (N = 680) | ||
|---|---|---|---|---|---|
| Any difference between SK&A and survey response (%) | Absolute difference more than 2 practitioners (%) | Absolute difference more than 25% a of total (%) | Absolute difference more than 2 practitioners (%) | Absolute difference more than 25% a of total (%) | |
| Number of practitioners of any specialty | 1505 (55.8) | 281 (13.9) | 698 (34.6) | 432 (63.5) | 311 (45.7) |
| Number of physicians of any specialty | 1175 (43.6) | 374 (8.6) | 576 (28.5) | 365 (53.7) | 335 (49.3) |
| Number of NPs/PAs of any specialty | 997 (37.0) | 90 (4.5) | 577 (28.6) | 203 (29.9) | 381 (56.0) |
| Number of primary care practitioners b | 1449 (53.7) | 229 (11.3) | 664 (32.9) | 409 (60.1) | 339 (49.9) |
Analysis of the November 2016 SK&A data and 2017 CPC+ Practice Survey fielded in April through September 2017.
Abbreviations: NP, nurse practitioner; NPI, National Provider Identifier; PA, physician assistant; PCP, primary care practitioner
We calculated percentage differences relative to the mean of the SK&A and survey responses. For example, if a practice site reported 4 providers on the survey and had 2 recorded in SK&A, that was calculated as a difference of 67 percent (|2‐4|/([2 + 4]/2)).
The survey asks for the number of practitioners who are primary care practitioners, defined as full‐ and part‐time physicians, NPs, or PAs with a specialty designation of family medicine, internal medicine, or geriatric medicine, and operating under their own NPI. From the SK&A data, in contrast, we defined PCPs based on specialty flags for family medicine practitioners, generalists, geriatricians, and internists.
For counts of all practitioners, the distribution of the magnitude of differences is roughly normal (Figure S2). However, more practice sites fall into the left side of the distribution, suggesting that practice sites were more likely to report a greater number of practitioners in the survey than was reflected in SK&A.
Practice sites with no difference in SK&A and survey responses were generally smaller, whereas the sites with the largest differences were generally larger. For example, of practice sites with 1 practitioner in the survey, 80.7 percent had no difference between SK&A and survey responses and 1.3 percent had a difference of 3 or more practitioners (Table S3). In contrast, only 5.6 percent of practice sites with 11 or more practitioners had no difference in responses, whereas 63.1 percent had a difference of 3 or more practitioners.
3.2. Practitioner composition
3.2.1. Percentage of primary care practitioners
About 23 percent of practice sites had a different percentage of PCPs reported in SK&A than in the survey; however, only 13.3 percent of the differences exceeded 25 percent (Table S4).
3.2.2. Multispecialty status
When defining multispecialty status on the basis of having more total practitioners than PCPs, 15 percent of practice sites had discordant responses (Table 3, top panel). Comparing SK&A multispecialty flags to the survey, about 26 percent of practice sites had discordant responses. In both cases, more than half of practice sites with discordant responses were multispecialty according to SK&A but not the survey.
TABLE 3.
Difference in multispecialty status and EHR use between SK&A and CPC + Practice Survey
| Characteristic and comparison | Concordant response (%) | Discordant response (%) |
|---|---|---|
| Multispecialty status | ||
| Comparing survey data to SK&A presence of non‐PCP practitioners (definition 1) | 2306 (85.5) | 392 (14.5) |
| More practitioners than PCPs, according to SK&A | 127 (4.7) | 236 (8.7) |
| Only PCPs, according to SK&A | 2179 (80.8) | 156 (5.8) |
| Comparing survey data to SK&A multispecialty flag (definition 2) | 2004 (74.3) | 694 (25.7) |
| Multispecialty flag in SK&A | 190 (7.0) | 601 (22.3) |
| No multispecialty flag in SK&A | 1814 (67.2) | 93 (3.4) |
| EHR use | ||
| EHR according to SK&A | 2416 (89.8) | 1 (<0.1) |
| No EHR according to SK&A | 0 (0.0) | 274 (10.2) |
Analysis of the November 2016 SK&A data and 2017 CPC+ Practice Survey fielded in April through September 2017.
Abbreviations: EHR, electronic health record; PCP, primary care practitioner
3.3. EHR use
All but 1 practice site used an EHR according to the survey (Table 3, bottom panel), but 10.2 percent did not use an EHR according to SK&A. Overall, 90 percent of practice sites had concordant responses between SK&A and the survey.
4. DISCUSSION
Overall, discrepancies are very common between the SK&A data intended to be current as of May through November 2016, and the survey responses to the CPC+ Practice Survey, fielded April through September 2017. Although most discrepancies are modest in magnitude, a few are substantial. For example, more than one‐third of all practice sites analyzed had a discrepancy greater than 25 percent in the number of practitioners working at the practice site; this was especially the case for sites with 8 or more practitioners according to the survey. The smaller number of these big practice sites with perfect concordance could mean that (a) staff in larger practice sites have a harder time providing accurate rosters or survey responses, or (b) larger practice sites experience more changes in staff over time.
This analysis also found that practice sites were more likely to report a greater number of practitioners in the survey than was reflected in SK&A. One possible explanation is that the survey data are more recent than the SK&A data, and practice sites may tend to increase their number of practitioners over time—especially given the trend of primary care practices more generally to consolidate into larger practice organizations. 11 CPC+ practices do appear to be growing over time, with the average number of practitioners per practice that remained in CPC+ increasing by approximately 6 percent by the end of the second program year. 12 This suggests that some differences in practitioner counts between SK&A and the survey may be due to differences in the timing of data collection. Another explanation is that practitioner counts may be open to interpretation in that when staff provide information to SK&A or respond to a survey, they might exercise judgment about what counts as “a practitioner” or what counts as “working at” the practice site. For example, one advantage of the survey is that we know that it explicitly asks respondents for the numbers of part‐time practitioners, residents, or fellows, which provides more insight into the practitioners included in the counts.
Concordance between the data sources was highest for the measure of EHR use. Still, 10 percent of practice sites reporting “yes” to EHR use on the survey did not have this information reflected in SK&A. These practice sites may be more recent EHR adopters that started using an EHR between the date of SK&A data collection and the survey in 2017. However, because CPC+ practice sites are required to use an EHR as a condition of CPC+ participation, it is likely that all CPC+ practice sites use an EHR. The survey data closely match the expected results for EHR use, with only one practice site reporting “no.” It is possible—perhaps even likely—that some discrepancies between the SK&A data and survey data for this variable are SK&A errors.
The discrepancies in this study have implications for health services research that uses SK&A data. First, ambiguity in how concepts are measured and interpreted by respondents may lead to analysis errors in studies that need to know the characteristics of individual practice sites with certainty. For example, if researchers use SK&A or survey data to exclude ineligible practice sites from an analytic population, such as a subgroup defined by number of practitioners, there will be misclassifications. In addition, using different data sources to define characteristics in an evaluation may introduce errors. For example, an evaluation might use the intervention group's self‐reported data, but the SK&A data to identify a comparison group. If SK&A is more likely than the participant data to identify a practice site as multispecialty, for example, as in this study (see Table 3), then we would expect differences in this “true” characteristic between the intervention and comparison groups, even when the two groups appear similar on the measured characteristic. The extent and direction of bias in the these scenarios will depend on the true relationship between the characteristic and impacts of the initiative.
Fortunately, discordant data are less likely to affect our understanding of the similarity of two groups of practice sites on average, as long as all characteristics are measured uniformly for both groups—for example, when using SK&A data to select a comparison group, but with practice site characteristics defined for both intervention and comparison sites using SK&A. For example, although we may not know with certainty how many practitioners a given practice had on, say, January 1, 2017, this study shows that average discrepancies in practitioner counts are very close to zero. Therefore, two groups of practice sites that appear in aggregate to have a roughly similar distribution of practitioner counts are, indeed, very likely to have a similar distribution. This means that SK&A data are likely suitable for selecting comparison groups—as we did for the evaluation of CPC+.
Different data sources have advantages and disadvantages for determining practice sites’ characteristics. SK&A data have the advantage of providing practice site‐level information for practices across the United States. One potential disadvantage of SK&A is that the database was updated on a rolling basis. Even if a researcher had access to the most recent SK&A data when conducting a study, it was likely for data to be a few months out of date for some practice sites, and there is uncertainly about the exact timing of data collection for each site. This lag in data readiness for SK&A is true of other data sources typically used in research and creates the potential for data errors whenever practice site characteristics are changing. Directly surveying practice sites has the advantage of allowing researchers to tailor survey questions to capture information of interest and may offer more insight on the exact timing of data collection. However, surveying practice sites across the country is burdensome to practice sites and cost prohibitive, and may offer incomplete information if response rates are low. Other administrative data, such as the Medicare Provider Enrollment, Chain, and Ownership System (PECOS), include some publicly available information on healthcare practitioners. However, they generally do not include information at the practice site‐level, and may be limited to practitioners enrolled in Medicare, meaning they exclude some practitioners who do not serve Medicare beneficiaries, such as pediatricians. 10 Researchers should consider these implications when selecting a data source for practice site characteristics.
Our analysis has three main limitations. First, this analysis covers only three concepts that were measured comparably between the SK&A and survey data. Second, despite a thorough search, it is possible we incorrectly identified some CPC+ practice sites when linking to the SK&A database using CPC+ application data for practice name, address, and NPIs—meaning that discrepancies noted here would not reflect discrepancies between SK&A and the survey. Third, it was not possible for us to identify the source of discrepancies when SK&A and the survey data were discordant. Despite limiting the analysis to concepts that were measured comparably between the data sources, differences in how outcomes are measured may explain some discrepancies. For example, using specialty flags to identify PCPs in SK&A data may misidentify practitioners as PCPs and may not be exactly comparable to using practice site reported information on the number of PCPs working at a practice site. In addition, as noted, discrepancies could reflect errors in either SK&A data or the survey, or simply changes that occurred at the practice sites between when the data were collected in 2016 (for SK&A) and 2017 (for the survey).
Finally, OneKey—introduced by IQVIA—largely replaces SK&A. Although SK&A data are an input for OneKey, the database relies on other sources and the findings of this analysis do not necessarily apply to OneKey. Nevertheless, this analysis is relevant to previous studies that use SK&A data, and it may inform future studies using historical SK&A data. Future studies should examine the implications of using OneKey in health services research.
CONFLICT OF INTEREST
The statements contained herein are those of the authors and do not necessarily reflect the views or policies of the Centers for Medicare and Medicaid Services.
Supporting information
Supplementary Material
Supplementary Material
ACKNOWLEDGMENT
Joint Acknowledgment/Disclosure Statement: The authors thank Eugene Rich (Mathematica), Kate Stewart (Mathematica), and Timothy Day (Centers for Medicare & Medicaid Services) for their comments and suggestions on earlier versions of the manuscript. We also thank Mathematica employees Elizabeth Holland and Maya Palakal for programming; Nancy Duda, Sarah Forrestal, and Brianna Sullivan for leading the collection of survey data; and Cindy George for editing.
Maurer KA, Blue L, Orzol S, Morrison Hensleigh N, Peikes D. Measuring physician practice site characteristics: A comparison of data from SK&A and a practice site survey. Health Serv Res.2021;56:334–340. 10.1111/1475-6773.13601
Funding informationThis study was funded by the Centers for Medicare and Medicaid Services, contract No. HHSM‐500‐2014‐00034I/HHSM‐500‐T0010.
REFERENCES
- 1. Barnes H, Richards MR, McHugh MD, Martsolf G. Rural and nonrural primary care physician practices increasingly rely on nurse practitioners. Health Aff. 2018;37(6):908‐914. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Nikpay SS, Richards MR, Penson D. Hospital‐physician consolidation accelerated in the past decade in cardiology, oncology. Health Aff. 2018;37(7):1123‐1127. [DOI] [PubMed] [Google Scholar]
- 3. Baker LC, Burndorf K, Devlin AM, Kessler DP. Hospital ownership of physicians: hospital versus physician perspectives. Med Care Res Rev. 2018;75(1):88‐99. [DOI] [PubMed] [Google Scholar]
- 4. Zhu JM, Hua LM, Polsky D. Private equity acquisitions of physician medical groups across specialties, 2013–2016. JAMA. 2020;323(7):663‐665. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Lammers EJ, McLaughlin CG, Barna M. Physician EHR adoption and potentially preventable hospital admissions among Medicare beneficiaries: Panel data evidence, 2010–2013. Health Serv Res. 2016;51(6):2056‐2075. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Dale SB, Ghosh A, Peikes DN, et al. Two‐year costs and quality in the comprehensive primary care initiative. NEJM. 2016;374(24):2345‐2356. [DOI] [PubMed] [Google Scholar]
- 7. Orzol S, Keith R, Hossain M, et al. The impact of a health information technology–focused patient‐centered medical neighborhood program among Medicare beneficiaries in primary care practices. Med Care. 2018;56(4):299‐307. [DOI] [PubMed] [Google Scholar]
- 8. DesRoches CM, Barrett KA, Harvey BE, et al. The results are only as good as the sample: assessing three national physician sampling frames. J Gen Intern Med. 2015;30(3):595‐601. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Peikes D, Anglin G, Harrington M, Ghosh A, Geonnotti K, O’Malley A, et al.Independent Evalutation of Comprehensive Primary Care Plus (CPC+). 2019. https://www.mathematica.org/download-media?MediaItemId={79F2BFBD-0993-41FF-A0D8-7EC4B42E32EF}. Accessed April 26, 2020
- 10. Cohen GR, Jones DJ, Heeringa J, et al. Leveraging diverse data sources to indentify and describe U.S. health care delivery systems. eGEMS. 2017;5(3): 9. 1‐12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Muhlestein DB, Smith NJ. Physician consolidation: rapid movement from small to large group practices, 2013–15. Health Aff. 2016;35(9):1638‐1642. [DOI] [PubMed] [Google Scholar]
- 12. Petersen D, O’Malley A, Ghosh A, Geonnotti K, Singh P, Tu H, et al.Independent Evalutation of Comprehensive Primary Care Plus (CPC+): Second Annual Report. 2020. https://innovation.cms.gov/data-and-reports/2020/cpc-evaluation-annual-report-2. Accessed October 14, 2020.
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplementary Material
Supplementary Material