Beyond Demographics: Missing Sociodemographics in Surgical Research

Abstract

Introduction:

Reporting sociodemographic data in research is critical to describe participants, and to identify contributing factors for patient experience, outcomes and heterogeneity of treatment effect (HTE). Social determinants of health and clinical health characteristics are important drivers of outcomes, and prospective studies collecting participant-reported data offer an opportunity to report these sociodemographics and evaluate for associations with outcomes. Clinical trials have underreported these factors previously, but reporting has not been examined in surgical research.

Methods:

We reviewed prospective studies collecting participant-reported sociodemographic data from four surgical journals in 2016. The proportion of studies reporting variables of interest in “Table 1” is described. Variables included information on patient identity (e.g., age, sex), clinical health (e.g., disease-specific characteristics, BMI), individual-level (e.g., education, income) and interpersonal-level (e.g., marital status, support) risk factors.

Results:

Forty-one publications met inclusion criteria. All reported ≥1 patient identity variable, 93% reported ≥1 clinical characteristic, 63% reported ≥1 individual-level risk factor, and 7% reported an interpersonal-level risk factor. Age, sex, and disease-specific characteristics were reported most commonly (98%, 98%, 88% respectively). 40% of studies reported comorbidities, though <15% reported on mental health. 50% reported race, 27% reported ethnicity, 24% reported education level, and 22% reported functional status. Other examined factors were reported in <20% of publications.

Discussion:

Sociodemographics reported in these surgical journals may be insufficient to describe the participants studied. This highlights an opportunity for the surgical research community to develop consensus on reporting of important sociodemographics that may be drivers of patient experience, outcomes and HTE.

INTRODUCTION

In healthcare research, understanding the characteristics of the population studied, as well as those excluded from a study, is critical to the identification and estimation of bias and the generalizability of findings.¹ The Consolidated Standards of Reporting Trials (CONSORT) statement supports complete, clear, and transparent descriptions of the study population, stating “this information allows readers, especially clinicians, to judge how relevant the results of a trial might be to an individual patient.”² The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement,³ Agency for Healthcare Research and Quality (AHRQ)⁴ and the Patient-Centered Research Outcomes Institute (PCORI)⁵ also recommend clear reporting and systematic comparison of participant characteristics across research studies. Additionally, the description of sociodemographic characteristics provides information on the inclusion and representation of participants of different genders, races, and ethnicities⁶ in accordance with the NIH Revitalization Act of 1993.⁷ The CONSORT statement specifically recommends displaying baseline sociodemographics in a table,² and authors typically present this information in the first table (“Table 1”) of a manuscript. These “Table 1” characteristics facilitate description of participants, allow for comparison between studies and help readers understand how a given study relates to a population of interest.

Table 1.

Reporting Rates of sociodemographic factors

Sociodemographic Category	Sociodemographic Variable	Reported N= 41 articles
Identity	≥1 Identity Variable	41 (100%)
	Age	40 (97.6%)
	Biologic sex	40 (97.6%)
	Race (any information)	20 (48.8%)
	Race: White	19 (46.3%)
	Race: Black, African American	14 (34.1%)
	Race: American Indian or Alaska Native	2 (4.9%)
	Race: Asian	2 (4.9%)
	Race: Native Hawaiian or other Pacific Islander	1 (2.4%)
	Race: Some other race or origin	8 (19.5%)
	Race: More than 2 races (multiracial)	1 (2.4%)
	Ethnicity (Hispanic or Latino)	11 (26.8%)
	Language	2 (4.9%)
	Gender identity	0
	Sexual orientation	0
Individual-Level	≥1 Individual Variable	26 (63.4%)
	Educational attainment	10 (24.4%)
	Employment status	6 (14.6%)
	Income	5 (12.2%)
	Geographic information (rurality/zip code/region)	1 (2.4%)
	Insurance	4 (9.8%)
	Functional status	9 (22%)
	Quality of life	1 (2.4%)
	Health literacy	0
	Pain catastrophizing	0
Interpersonal-Level	≥1 Interpersonal Variable	3 (7.3%)
	Partner/Marital status	3 (7.3%)
	Household size	0
	Children (≤18 years)	0
	Caregiver status	0
	Instrumental support	0
	Emotional support	0
	Trust in the medical profession	0
Clinical Health	≥1 Clinical health variable	38 (92.7%)
	Disease characteristics of condition studied	36 (87.8%)
	Comorbidities	19 (46.3%)
	BMI	18 (43.9%)
	Prior treatments related to the condition studied	11 (26.8%)
	Smoking	8 (19.5%)
	Alcohol use	2 (4.9%)
	Mental Health (any information)	6 (14.6%)
	Depression	2 (4.9%)
	Anxiety	0
	Other mental health information	6 (14.6%)

Guidelines recommend including demographic, clinical, and social patient descriptors,²,³ and investigators must determine specific information to collect and report for an individual study. However, some studies may lack sociodemographic information needed to describe and understand the experiences of the study population, assess the generalizability of findings, estimate bias, assess inclusivity, systematically compare studies, and understand variability in important clinical and patient-reported outcomes (e.g., experience in healthcare and access to resources).^8–14 For example, many studies have described racial disparities in preventive health services, quality of care, and clinical outcomes for numerous health conditions including cardiac disease, cancers, and surgical diseases.^{10,11,15–19} Differential outcomes have also been associated with other factors including socioeconomic status, education, and language.^{12,13,16,17,20,21} Despite this, there is documented underreporting of sociodemographic factors in clinical trials, and this underreporting has been specifically cited as contributing to a lack of generalizable information in cardiac and cancer trials.^8–11 Surgical research has historically focused separately on biological and non-biological explanations for variability in outcomes, but researchers must examine both together to start to understand their combined impact.^12,13,22

Differences in sociodemographics may be controlled for in randomized studies when assessing for a difference in “average” effect between treatment and non-treatment groups, but these factors may still have an effect on outcomes for a subgroup of patients. Variations in outcomes can be particularly apparent in prospective observational studies and when randomization is not possible.¹⁴ Nonrandom, explainable variability is described as heterogeneity of treatment effect, and identifying contributing sociodemographic factors may be important when considering how to optimize outcomes across all subpopulations. Both randomized and observational prospective studies present an opportunity to explore these relationships.¹⁴ Furthermore, studies gathering patient-reported data can thoughtfully, intentionally, and accurately collect (and report) information hypothesized to impact important outcomes that is often not available in databases or the electronic medical record (EMR). Inclusion of this information is particularly important for assessing and differentiating between biological and non-biological associations with outcomes. The degree to which surgical research captures sociodemographics has not been examined. This study aims to describe reporting rates of patient-level sociodemographic factors known to be associated with patient-reported and clinical outcomes^{16,17,21,24–29} in peer-reviewed surgical research.

MATERIALS AND METHODS

Selection of Sociodemoaraphic Variables

A panel of researchers with experience in health services and patient-centered research collaborated iteratively to develop a list of measurable sociodemographic variables hypothesized to potentially contribute to variation in patient decision-making, patient experience, patient-reported outcomes, and surgical outcomes. Researchers represented clinical training in general surgery, gynecologic oncology, pharmacy, biochemistry, emergency medicine, and internal medicine, as well as research training in epidemiology, health services research, and public health. Initial variables were chosen based upon expert opinion and a review of the literature on the social determinants of health and factors encompassed by the individual and interpersonal levels social-ecological model.^21,25,26,30 The social determinants of health encompass characteristics that have been identified as strong predictors of clinical outcomes and include conditions in which people live and work.²⁵ The social ecological model is a framework used to understand the dynamic intersection of sociodemographic factors with health behaviors on five levels: individual (demographics, knowledge, attitudes), interpersonal (family, friends, social connections, support), organizational (individual organizations, institutions), community (relationships between organizations, social cohesion), and policy (laws and regulations).^26,30 The panel focused on the individual and interpersonal level factors to select items measured for individuals rather than populations. The initial large list of variables was narrowed to those that met the following criteria: 1) association with outcomes supported by prior evidence or published conceptual model; 2) applicable to a variety of surgical outcomes studies; and 3) measurable with established conventions such as categorizations (age, sex, race) or previously validated measures (e.g., pain catastrophizing,³¹ health literacy,³² trust in the medical profession³³).

We grouped variables into four categories: 1) identity (i.e., age, sex, and race), 2) individual-level (per the social-ecological model), 3) interpersonal level (per the social-ecological model) and 4) clinical health (e.g., disease characteristics, comorbidities). All variables and categorizations are listed in Table 1. Sociodemographics included participant descriptors that are historically reported in the literature, often collected in databases or EMRs, and typically used to assess generalizability. Additional patient-reported descriptors included factors that would require information directly from patients/participants beyond what the medical record or quality or clinical databases might capture (e.g., gender identity, quality of life, caregiver status).

Literature Review and Article Selection

To describe reporting rates of the sociodemographics of interest in general surgical literature, articles published from January to December 2016 in four U.S. surgical journals with the highest impact factors in 2016 (Annals of Surgery, Journal of the American Medical Association (JAMA) Surgery, Journal of the American College of Surgeons, and Journal of Trauma and Acute Care Surgery) were reviewed. We limited our review to prospective studies collecting participant-reported data as many of the variables we considered could not be collected from clinical databases, and studies that used only database or EMR abstraction would therefore never have had the opportunity to collect several of these variables. Articles included in our study (detailed in Figure 1) met the following criteria: 1) conducted in the United States; 2) written in English; 3) original investigations; 4) prospective adult studies, and 5) collected participant-reported data. We excluded: 1) research in non-human participants; 2) research focused on health care providers or workers (i.e., surgeons, residents); 3) research conducted in pediatric populations; 4) studies without a table displaying sociodemographic data; and 5) studies that abstracted sociodemographic data solely from the EMR or a database.

Figure 1.

Flow diagram depicting the process of article selection.

Note: Articles were only classified under one exclusion criterion, though many could have been excluded based upon multiple criteria.

Four independent reviewers screened articles, with each reviewing six months of publications from two journals. Methods were reviewed in all studies that appeared to be eligible from the abstract to ensure that sociodemographic data were collected directly from the patient or patient surrogate (e.g., documented survey administration, patient interview, or other description of study procedures). All articles identified underwent a second screening by a different reviewer to ensure eligibility. Discrepancies were resolved by consensus.

Data Abstraction and Analysis

Sociodemographic reporting was a dichotomous outcome, either “reported” or “not reported.” We considered a variable “reported” if it was mentioned in Table 1 (or equivalent table/figure intended to describe the study population). We considered biologic sex reported if a study was sex-specific. We considered composite scores as one variable (e.g., we recorded frailty indexes as reporting of functional status) unless the table/figure further described individual components of the score (e.g., described components of mental health and physical ability). As guidelines recommend displaying baseline demographic and clinical characteristics in tabular form, we did not include sociodemographics reported in the body of the article.²

Two authors independently abstracted data into Microsoft Excel and compared findings. For discrepancies, each author reviewed variable definitions and re-examined articles of interest. After correcting errors in abstraction, all discrepancies resolved. Findings are presented as the proportion of eligible studies that reported each variable.

RESULTS

Of the 1588 articles reviewed, 41 met pre-specified inclusion criteria. The consensus list of sociodemographic variables examined and the proportion of included articles that reported each variable is presented in Table 1. All included prospective studies reported at least one identity variable, with nearly all reporting age (n = 40, 98%) and sex (n = 40, 98%). Additional identity variables were frequently missing. Half of the included articles reported race (N=20, 49%), and information beyond categories of white, black, or “other” was rare (N = 2, 5%). Less than one-third of studies reported ethnicity (N = 11, 27%), and fewer reported language (N=2, 5%). Individual-level factors related to socioeconomic status, including educational attainment (N = 10, 24%), employment status (N = 6, 15%), income (N = 5, 12%), insurance (N = 4, 10%), and geographic information (N = 1, 2%), were also infrequently reported. Behind educational attainment, functional status was the second most commonly reported individual-level factor, appearing in N = 9 (22%) studies.

The only interpersonal-level factor reported was partner/marital status in N=3 (7%) studies. Data on additional individual and interpersonal factors were rare. Baseline quality of life appeared in N = 1 study. Self-identified gender, sexual orientation and health literacy were absent in all studies. Clinical health characteristics were reported in N = 38 (93%) studies, with information on disease characteristics most commonly present (N = 36, 88%). General comorbidities (N = 19, 46%) and BMI (N = 18, 44%) were reported in nearly half of studies while mental health comorbidities were presented in N = 6 (15%) studies.

DISCUSSION

This review of prospective studies published in a subset of surgical journals highlights an opportunity to improve the capture and reporting of sociodemographics in surgical research. Reporting guidelines for observational studies³ and clinical trials² recommend reporting baseline demographic, clinical, and social characteristics. While studies consistently reported sex and age, other important characteristics with implications for healthcare access and health outcomes were frequently not described. The low reporting rates are particularly notable as our study design intentionally limited the analysis to studies with an opportunity to prospectively collect participant-reported data, and therefore those with the highest likelihood of inclusion of these variables. This is a critical gap for describing participants, assessing research generalizability,¹ evaluating bias and heterogeneity of treatment effect,^1,14 identifying risk factors for differential outcomes,^12,13,17,24 and examining inclusivity of surgical research.⁶

The call to improve representation, inclusivity and subsequent sociodemographic reporting in research is not new. For example, in 1993, an NIH-led initiative pushed for the inclusion of women and minorities in research,⁷ and national efforts recommend universal, standardized reporting of racial, ethnic, and language information.³⁴ Despite this, data on race, ethnicity, and language were frequently missing in the reviewed surgical research articles. It is unclear if low reporting rates reflect a lack of inclusion or participation in surgical research, failure to structure data collection to reflect participants, or a lack of adequate final reporting. For clinical health characteristics, less than half of included articles reported BMI despite its importance as a risk factor for surgical complications and implications for overall health.^35,36 Information on mental health was reported substantially less frequently than other comorbidities (<15% vs 46%), though depression specifically has been associated with differential outcomes such as increased risk for mortality after coronary artery bypass graft.³⁷ Given this, mental health information may warrant reporting levels more similar to that of other comorbidities.

Individual-level factors related to socioeconomic status (education, employment status, income, insurance, and geographic information) were also infrequently reported, despite overwhelming evidence that low socioeconomic status is a risk factor for differential outcomes across surgical disciplines. This evidence includes increased mortality after emergency general surgery,²⁰ larger tumor size at diagnosis of breast cancer,¹⁶ and lower rates of stoma closure.¹⁷ These rates of socioeconomic status reporting are similar to those found in a study evaluating demographic reporting in randomized controlled trials, which suggests this may be a general trend in research reporting and not specific to surgery.^8,9 Many additional patient-reported sociodemographics examined (gender identity, sexual orientation, health literacy, pain catastrophizing, household size, children, caregiver status, emotional support, trust in the medical profession) were not reported in any of the studies reviewed. The absence of data on self-identified gender and sexual orientation is notable as its missingness limits surgical health services understanding of the inclusion of this population who may have differential experiences and outcomes.³⁸ Many of the other absent factors were interpersonal-level factors with prior evidence of associations with both patient outcomes and experience (caregiver status, provider trust, instrumental support, and emotional support).^27–29 While the applicability of any one of these factors individually may vary by study, as a group interpersonal factors may contribute to heterogeneity of treatment effect through pathways such as treatment adherence and access to care.

Developing a single optimal “Table 1” presents an ongoing challenge as including all 38 individual variables (including seven racial categories) examined in this review is not feasible or pertinent for all studies. The first practical step toward improved capture and reporting of sociodemographic variables should, therefore, be the standard practice of identifying a conceptual model or framework during study design. A conceptual model should include the interplay of sociodemographic factors with other factors (e.g., patient-level or health system) that may lead to the health outcomes being studied. The use of conceptual models can help investigators carefully consider possible sociodemographic variables, and then narrow to those most important to adequately describe their specific study population, allow the reader to identify possible bias, and investigate factors potentially associated with outcomes. If study constraints preclude investigation of potentially important factors, authors should state this in study limitations. Beyond this, there is also an opportunity for the surgical research community (e.g., editorial boards, national associations) to develop consensus on which sociodemographic factors should be universally captured and reported and which should be routinely considered in conceptual models so that important relationships between these characteristics and outcomes (clinical, patient-reported, and experiential) are examined in surgical literature. Consensus on reporting could also facilitate pooling of data from multiple studies, and Masserwah et al. assert that linkage of two or more data sets from observational studies may be one of the best ways to mitigate bias and translate observational research findings into practice.¹⁴ Universal reporting of age, sex, race, ethnicity, and language would not only help align surgical research with this goal and existing reporting guidelines from the AHRQ,³⁴ but also aid in the transparency of inclusion of minority populations in research.^6,11,22 Similarly, universal reporting of socioeconomic status indicators will be important as socioeconomic status is consistently shown to be associated with differential outcomes in surgery.^16,17,20,24 Additional variables such as baseline functional and mental health status, comorbidities, and measures of interpersonal support should be routinely considered in conceptual models as these known risk factors may be under-recognized, and could represent missed opportunities to develop interventions for those with differential outcomes. Use of validated measures for these variables will be important for validity, consistency, and comparability across studies.

There are several important issues to take into account in the development of either consensus standards or an individual study. Investigators should consider survey fatigue when collecting patient-reported data, limiting questions to those most likely to impact the outcome(s) of interest based upon the conceptual model. When asking for racial and ethnic data, investigators should note that between 2000 and 2010 the US Census Bureau found that “some other race” was the third most common self-selected racial category.³⁹ Subsequent research demonstrated that the addition of “Hispanic” and “Middle Eastern or North African” categories led to higher quality data and fewer people selecting “some other race.”³⁹ Because race is a method of social stratification, self-reported race is more accurate and meaningful than the categories of “white,” “black,” and “other” that were most frequently identified.¹² Furthermore, “other” is not a racial category and is of little utility beyond increasing subgroup size.¹³ Subgroup analyses may not be appropriate for all racial categories identified, but this should not preclude reporting this information in “Table 1” to describe the study population accurately. Similar considerations are important when deciding whether or not to include questions on sexual orientation and gender identity. Concerns regarding privacy and the potential for harm may be particularly important in this setting, and existing literature suggests that participant concerns relate specifically to who will see the information, why it is being collected, and how it will be used.⁴⁰ Stakeholders suggest that explicit explanation of the confidentiality of the information, and the intent (e.g., to include and represent diverse populations in research) will help mitigate this concern. ⁴⁰

A limitation of this study is the small subset of total publications (41/1588) in the reviewed surgical journals that met the inclusion criteria. It is possible that the results would be different if we examined additional journals, and thus our results may not be representative of the entire body of surgical literature published in 2016. Another important limitation is we limited our review only to the information presented in tables within the articles, and it is possible that some variables were collected but not reported in a table. Finally, the relevance of reporting a variable within the context of any individual article was also not evaluated.

CONCLUSIONS

Many sociodemographic factors important for describing participants, assessing heterogeneity of treatment effect and identifying disparities were frequently absent from articles in this selected group of surgical journals. This highlights an opportunity for the surgical research community to define consensus standards for reporting important variables beyond the basic “Table 1” demographics of age, sex, and disease characteristics. Improved reporting will be important to advance our understanding of heterogeneity of treatment effect, generalizability of findings, potential sources of bias, inclusion of underrepresented populations in research, and to expand knowledge regarding drivers of patient outcomes, decision-making, experience and inequities.

Research Highlights:

• There is previous underreporting of sociodemographics in clinical research

• Low reporting in surgical journals may be insufficient to describe participants

• Underreporting limits generalizability and assessment of variability in outcomes

• There is an opportunity for the surgical community to develop reporting standards