Table 1.
Suggested questions to consider and corresponding examples for navigating demographic data use through an ethics and social justice lens
| Research Stage (See Figure 1) |
Questions to contemplate | Examples of practices to consider and resources to leverage |
|---|---|---|
| Research Question | 1. What is the theoretical and empirical justification for asking this question related to demographic data? Or, what is the justification for not asking this question? 2. Who is this research question intended to benefit? Who does it have the potential to harm? 3. What are the possible implications (benefits and harms) of my research question for various communities? How can I attempt to increase benefits and decrease harms? |
Use a diverse team science approach to ensure no one expectation is dominant, no single expertise is prioritized, and to improve the comprehensiveness of the motivating prior research and theory (Ledgerwood et al., 2022; Noroña-Zhou & Bush, 2021) Be intentional about the sample (e.g., avoid defaulting to “easy-to-access populations;” Roberts et al., 2020) Use a data ethics checklist to keep your team accountable throughout the research process (e.g., Lou & Yang, 2020) |
| Capability Set [Community]: How, and should, this data be collected? | 1. Has there been community input on this research question? 2. Has there been community input on the methods I am considering? i. For existing measures: Has prior research utilized community-engaged methods (e.g., focus groups)? Which communities was the measure created and tested in? ii. For new measures: Have I considered community-engaged focus groups, partnering with an institutional community advisory board, or other community-engaged practices? 3. Have I taken steps to recruit a representative sample from the community? How will my sample composition affect generalizability? |
Review the literature to determine if prior studies have collected community input on this or related questions (Pedersen et al., 2022) Consult with community-engaged researchers at your institution or elsewhere (Pedersen et al., 2022) Establish a community advisory board and or partner with community members and advocacy organizations (Brown, K.S. et al., 2019; Collins et al., 2018; Rowley & Camacho, 2015) Hire research staff from within the target community to provide input and help develop rapport with community partners and participants (Rowley & Camacho, 2015) Compensate participants, staff, and community partners appropriately and generously, including forms of compensation in addition to traditional financial payments such as transportation, food, and child care (Brown, K.S. et al., 2019) Use snowball sampling and community partners to help establish trust with participants (Rowley & Camacho, 2015) Regularly review demographic characteristics of the sample to ensure ongoing representation of groups (Pedersen et al., 2022) |
| Demographic Methods | 1. Which demographic variables am I considering including? Which am I considering excluding? What is my justification? 2. What am I trying to ask with these demographic variables? What are the limitations of these variables? 3. How do my choices surrounding demographic methods affect generalizability and interpretability in the context of other research, including future meta-analyses? 4. Who am I helping by including or excluding these variables? Who am I harming? How can I reduce the harm? If harm is possible, what is my justification for proceeding? |
Clearly document the rationale for including and excluding certain demographic variables (e.g., in a protocol) Consider including demographic variables that might be relevant for future research, even if not directly relevant to the current study aims (e.g., demographic characteristics that may be relevant for future meta-analyses) Recognize the sensitivity of demographic data and be explicit and clear with participants about why you need it and how you will use it (Rowley & Camacho, 2015) Consider using evidence-based demographic questionnaires (e.g., PhenX Toolkit; Hamilton et al., 2011) Do not use the label “other” when listing options for demographic categories as it carries a negative connotation of being abnormal (Ford et al., 2021; consider instead “not listed” or “prefer to self-describe”) When asking questions about gender/sex, avoid only including binary male/female options (e.g., include genderqueer as an option; Hyde et al., 2019)) When using a checklist of demographic items, allow participants to check as many as they want; do not force a single selection Moody et al., 2013; Viano & Baker, 2020) Use open-ended demographic questions so participants are not forced to check a box that may not accurately describe their identity (Roberts et al., 2020) For surveys, include demographic questions at the end so participants can choose whether and what demographic information to disclose within the informed context of the other information they’ve already shared (Moody et al., 2013) Provide an easy/accessible way for participants to express any concerns or questions about the methods (Moody et al., 2013) Include questions about cultural assets and strengths, do not only focus on cultural deficits or weaknesses (Castillo & Gillborn, 2022; Sablan, 2019) |
| Ethical Use of Demographic Data | 1. What are the potential benefits and harms of how I plan to statistically examine demographic data? How can I maximize benefits and minimize harms? 2. Have I pre-registered my analytic plan and methods related to demographic data? If not, what is my justification? |
Establish an a priori conceptual framework to support why each demographic factor you include in analysis is relevant to your research question (Chandran, 2021; Noroña-Zhou & Bush, 2021) When analyzing race/ethnicity, avoid defaulting to White as the reference group. This reinforces White as the standard that all other racial/ethnic groups should be normed to (Ionnidis et al., 2021; Noroña-Zhou & Bush, 2021) Examine within-group variability before collapsing groups based on a shared demographic feature for between-group comparisons (Buchanan et al., 2021; Noroña-Zhou & Bush, 2021; Rowley & Camacho, 2015) Try not to collapse different demographic groups with small sample sizes into an “other” or “minority” variable that lacks conceptual meaning (Castillo & Gillborn, 2022; Flanagin et al., 2021; Noroña-Zhou & Bush, 2021); If it is necessary to collapse some groups, justify this decision and describe its limitations |
| Capability Set [Community]: How, and should, this data be applied? | 1. Who will these findings and corresponding interpretations benefit or harm? Has there been community input from those that these findings might affect? 2. Are there communities that are noticeably absent from my research sample? If so, have I reviewed and enacted suggestions for increasing representation (see Capability Set[Community]: How, and should, this data be collected?)? 2. How can I disseminate results back to the community? How does the community want these results to be utilized moving forward? 3. Am I planning to share the demographic data publicly? What is my justification? Have I considered the benefits and harms of sharing demographic data? Have I received community input about this? |
Re-review the literature to understand how community input has or has not been applied to similar research before (Pedersen et al., 2022) Discuss findings, their implications, and if/how to disseminate both the original data (i.e., through public data sharing) and the findings with community partners and/or community-engaged consultants (Collins et al., 2018) Be intentional about which “broader audiences” you are trying to engage. “The audience outside of academia” is not a monolith, so strive to understand the intended audience and craft dissemination materials specifically for them (Lewis Jr. & Wai. 2021) Host community data walks (Brown, K.S. et al., 2019) |
| Transformative Functioning | 1. How am I choosing to report demographic data? Which intersectional identities have I reported? What is my justification based on my research question, community input, and the position of my research within the broader context of my field (e.g., facilitating comparisons with other work)? 2. How am I interpreting findings from demographic data? What theoretical or empirical justification do I have for this interpretation? Could my interpretation reinforce harmful or inaccurate biases? 3. Have I carefully described the limitations of the data and what they cannot be used to describe? 4. How can I partner with the community to use these findings to address root causes of inequity and restore well-being? |
Consider reporting the full sample demographics even for demographic factors not included in analyses either in text or as a supplement (Roberts et al., 2020). Report intersectional identities in text and/or as a supplement. While it may be impossible to report all intersectional identities, consider reporting those that are particularly relevant to your research question or that the community has asked you to center. Consider including a positionality statement in manuscripts to enhance transparency and to better contextualize the work (Castillo & Gillborn, 2022; Roberts et al., 2020) Be clear about generalizability and limitations (e.g., include a constraints on generalizability statement in manuscripts; Castillo & Gillborn, 2022; Pedersen et al., 2022; Simons et al., 2017) Situate socially constructed demographic characteristics properly within historical and sociopolitical contexts (e.g. do not ascribe racial/ethnic differences to biological differences; Cole, 2009; Noroña-Zhou & Bush, 2021) Disseminate the research process and methods along with the findings (e.g., be explicit about who was/wasn't included; (Lewis Jr. & Wai, 2021) Exercise scientific humility when contributing to public discourse (Lewis Jr. & Wai, 2021) |
Note. Our suggestions related to demographic data draw from theories of social justice (Fraser, 2009; Sen, 1985) and the American Psychological Association’s General Principles of Ethics (APA, 2016): (1) Beneficence/maleficence: Maximizing benefits and minimizing harms to research participants and the broader community; (2) Fidelity and responsibility: Justifying decisions related to demographic data by remaining up-to-date on empirical and theoretical knowledge; (3) Integrity: Ensuring that demographic data accurately capture identities and clearly communicating the limits of the data; (4) Justice: Attending to who is included and excluded from the research, who is affected by research findings, and how we can utilize research findings to address root causes of inequity and restore wellbeing; (5) Respect for people’s rights and dignity: Partnering with individuals from the community to center their voices in the research process in order to affirm identities, communicate respect, and promote wellbeing.