Validity of energy social research during and after COVID-19: challenges, considerations, and responses

Abstract

Measures to control the spread of coronavirus disease 2019 (COVID-19) are having unprecedented impacts on people’s lives around the world. In this paper, we argue that those conducting social research in the energy domain should give special consideration to the internal and external validity of their work conducted during this pandemic period. We set out a number of principles that researchers can consider to give themselves and research users greater confidence that findings and recommendations will still be applicable in years to come. Largely grounded in existing good practice guidance, our recommendations include collecting and reporting additional supporting contextual data, reviewing aspects of research design for vulnerability to validity challenges, and building in longitudinal elements where feasible. We suggest that these approaches also bring a number of opportunities to generate new insights. However, we caution that a more systemic challenge to validity of knowledge produced during this period may result from changes in the kinds of social research that it is practicable to pursue.

1. Introduction

The coronavirus disease 2019 (COVID-19) pandemic is having tragic health consequences around the world, and measures to combat it are impacting people’s lives in unprecedented ways. There is, as yet, no clarity on when and how measures such as suspension of certain businesses and physical distancing might end completely, or need to be reintroduced. The timescales required to develop a vaccine and deploy it globally suggest this could be well into 2021 and possibly later.

During this time, the validity of energy social science research faces additional threats. Validity generally refers to the truth of a knowledge claim or inference [1]. National and global events continuously shape social worlds. But the magnitude, speed, and reach of the changes to our lives are of a different order to anything that most people alive today have experienced. Given the scale and rapidity of change, how can we ensure that conclusions drawn from data collected during the pandemic are valid, representative, generalisable to a post-pandemic world, and comparable to the pre-pandemic one?

While the answer is inherently unknowable [2], our aim in writing this paper is to highlight principles that we believe energy social science researchers can take to help mitigate this uncertainty, and ease future interpretation of research findings in the context of the progressing pandemic. Broadly speaking, these principles involve giving consideration to possible impacts of the pandemic and associated response measures on findings; adjusting research design and data collection to reflect this; and reporting extra contextual detail. We argue that researchers who take reasonable steps in these areas will be able to ensure greater confidence in the validity of the work they conduct during this period. Through consciously enhanced transparency for the ‘extended peer community’ that post-normal science scholars have long espoused [2], [3], their contributions will be better positioned to help address future challenges on the validity of findings by reviewers and users.

We co-produced these principles as energy researchers who represent a variety of relevant disciplinary perspectives and subject interests, and are based in a range of institutions and countries. This allowed us to balance the will to draw on a breadth of input across the field, with the need to share these principles in a timely fashion. We set them out in the hope that researchers will find them helpful, but recognise that applicability will vary across energy social science research. Our recommendations are likely to be most applicable to researchers employing quantitative research methods that are often restricted in the amount of contextual data they are able to collect [4]. However, we hope that as a set of considerations they will be helpful to a broad range of energy social science scholars to employ as they see fit.

The next section of this paper sets out the key challenges we identify for validity during the COVID-19 pandemic, and justifies our focus on social science research in energy. We then expand on steps that researchers can take to address these challenges, and provide a simple checklist that can be applied by scholars in order to address the impacts of the pandemic on their research. We finally highlight additional opportunities these steps can yield, but also point out important potential implications for the nature of knowledge generated by contemporary research.

2. Challenges to validity

Decisions about validity inherently concern tradeoffs and priorities of a given research study [1]. For instance, a researcher might prioritize internal validity (or “the degree to which a study establishes the cause-and-effect relationship” [5]) by conducting a randomized controlled laboratory experiment. Artificial laboratory conditions enable strong experimental control, but limit generalizability across diverse, complex real-world situations. A field study, in contrast, might prioritize external validity (or “the generalization of research findings […] to settings and populations other than those studied” [6]), but surrenders some ability to control and measure variables.

Both of these forms of validity are important. If we cannot trust the findings of a study because of methodological problems or unaccounted-for variables, generalisability is irrelevant. And findings that only apply in exceptionally narrow circumstances offer very limited value in applied research settings. External validity tends to be given special weight in applied research, including most energy research, where the generalisability of findings, and therefore any resulting conclusions and recommendations for action, often has primacy.

Our point of departure here is the impact of the COVID-19 pandemic and associated response measures on internal, external, and subsidiary forms of validity. We argue that the pandemic merits explicit consideration for validity for a number of reasons. First, the response to COVID-19 represents a departure from ordinary circumstances that is unprecedented in terms of its global nature, rapidity, diversity, and severity of impacts. At the time of writing in May 2020, over 100 countries and several billion people were under some form of lockdown, with restricted rights to movement and public assembly. In many cases, schools, non-essential businesses and hospitality venues were closed. Evidence of prevalent psychological distress and anxiety had begun to emerge [7], [8]. Such a situation is far removed from the conditions under which knowledge is ordinarily produced and applied, and questions around the validity of findings generated during this circumstance are inevitable.

Second, an important consideration for external validity is how stable findings are over time. While there is always uncertainty about how closely the future will resemble the present, we argue that this uncertainty is now especially high. Movement restrictions have already left millions of people unemployed, with millions more at risk of losing their jobs as businesses contract or close [9]. Governmental support packages are building up unprecedented levels of national debt that will have to be paid for, with little clarity around the effect this will have for public services and taxes. While some effects such as quarantine measures will be shorter-term, it is unknown whether the pandemic itself and associated consequences will result in long-term effects on the individual and societal level. Realistic and symbolic threats induced by the pandemic are likely to affect individuals’ values, identity, and worldviews and thus could exert long-term effects on various dimensions [10]. Moreover, research on past societal crises has shown that pandemic-related effects such as large-scale unemployment can lead to long-term effects on mental health [11]). Taken together with the scale of current impacts, we believe this increased uncertainty in the short, medium, and long term justifies special consideration of validity of social research and, furthermore, a higher burden of proof on claims to such validity.

Why is a particular focus on energy studies important? Energy use plays a prominent role in many aspects of human life. Any changes on the scale being experienced during the pandemic have significant impacts not only on patterns of interaction with energy systems, but also on how people relate to and prioritise those systems. Much energy research conducted today aims to inform transitions to clean, low-carbon energy systems that work for people and society. Although research conducted now can shed light on how the extraordinary measures in place might impact energy use (such as evidence of reduced weekday electricity use [12] and changing usage patterns [13]), it is challenging to disentangle these impacts from those that result from measures deployable absent a pandemic. Moreover, the impact of such a drastic, globally shared experience impacts discursive and normative registers, with undetermined implications for public commitment to low-carbon energy transitions that become interwoven with other drivers of change pathways.

Although many of the principles we set out next could simply be viewed as good research practice, we think that they merit explicit attention during this pandemic and its aftermath. We argue that they are especially important for those domains of energy social research that claim broad generalisability to their findings and insights, with limited focus on context. For example, we think the points raised here are generally more applicable to survey-based than ethnographic research. By bringing these recommendations together here, we hope to stimulate a more consistent response by social researchers, allowing greater commensurability and comparability across studies in the future. Furthermore, we recognise that scholars using social research approaches in energy have a wide variety of backgrounds and levels of experience. What we suggest may be self-evident to some, although for these we hope it will be helpful to have a checklist to compare their own responses against. To others, we hope it will provide both a prompt to consider challenges to validity, and a handy set of responses to consider.

3. Principles to consider for validity

We have argued that challenges to the validity of social science energy research presented by the COVID-19 pandemic warrant special recognition. We now lay out a set of principles for researchers to consider bringing to their practice for the duration of the pandemic period and its aftermath to help bolster the validity of their work, and to ensure that future use of their findings and recommendations is facilitated by requisite information to aid correct interpretation. Our recommendations address data collection and the reporting of study conditions and context, as well as considerations for study designs in order to ensure high validity of energy social science research conducted during and after the pandemic. Given the large number of possible new factors to be taken into account, we propose a ‘core and consider’ approach, allowing researchers in the field to prioritise and justify the measures they want to take to account for potential pandemic-related influences.

Where possible, we have drawn on existing good practice guidance, which itself has developed through conventional processes of cross-field engagement [14]. While we think the validity challenges we have raised here are important, we also recognise that any responses to them must fit within existing research plans, budgets, timelines, labour constraints, and the heightened need for affective care, including researchers’ own well-being under personal stress-inducing conditions. Any response must be both proportionate to the anticipated vulnerability to validity challenges of the kind set out in the previous section.

Ethical and data protection concerns, while not directly related to validity, must be borne prominently in mind. Any changes to planned research should not, unless it is explicitly justified, introduce collection of categories of data that are more sensitive than those that were originally (or would ordinarily be) planned and/or approved. This means, for example, that researchers should not (without careful thought and justification) begin to collect data on physical or mental health unless this was intended anyway. Researchers should be mindful of the extra burden to participants that introducing additional data collection could bring. Extra sensitivity is called for on the part of researchers to the potential impacts of collecting data on topics which may be more upsetting now than would ordinarily be the case.

3.1. Capture and report on extra relevant data

We suggest that additional and/or modified variables may need to be collected and reported for studies carried out during or after the COVID-19 pandemic in order to account for the impact of the pandemic on research validity. Already, researchers should – and many do – report contextual factors of any study, and consider how these might impact the study findings [15], [16], [17]. Given the large number of possible new factors to be taken into account, we suggest researchers take a ‘core and consider’ approach. Government restrictions and relevant demographic variables at the level of the unit of analysis (e.g., individuals or households) are core additional variables that should be reported and discussed. Other factors should be considered for additional reporting depending on the precise topic of research.

As in all studies, reporting of contextual factors should encompass date(s), place(s), and duration of data collection. As a core concern, we suggest that this should now be supplemented with information on pandemic-related national and local policies that were in force at the time and place of data collection. This could include factors such as levels of restriction of people’s freedom to move around outside the home, including self-imposed precautionary behaviour, and the open/closed status of specific relevant services such as schools and certain businesses. Significant changes in any of these measures during data collection should also be reported. Researchers may consider it to be important for context to give a sense of the severity of the pandemic (including health, social and economic impacts, as relevant). We suggest using official government references for a description of such policies and impacts where possible, in ways that are cognizant of their rapid temporal evolution.

A further core consideration is that local and national pandemic response measures affect individuals and households in diverse ways; specifying the national policies during data collection alone does not explain effects at the individual (or other analytical) level. More specific effects can be captured by measuring application of and compliance with response measures on the respective analytical level, and/or through collection of additional demographic variables from which application could be inferred. The nature and detail of measures will differ by locality, but could include whether someone is considered a ‘key worker’ (and hence still regularly leaves the home during lockdown) or comes under a high-risk category and has to observe stricter measures. Other standard demographic variables may need amendment depending on the study aims. For instance, employment status can include categories such as being placed on government-subsidised furlough, working reduced hours, or working fully from home.

Other variables that might ordinarily have been judged as having limited importance, might gain relevance. Impacts of the COVID-19 pandemic are thought to be exacerbating existing inequalities in many societies, such as energy poverty issues [18]. A key variable in many studies will likely be the financial situation of the individual, household, or other unit of interest. Capturing information on recent (and risk of future) changes in factors such as income (including transfer payments), changes in employment status, increased receipt of benefits, or self-reported financial satisfaction may take on greater importance. Unexpected deprivation from work income may have differential effects on energy-related measures, relative to foreseeable prolonged unemployment periods; while this is a consideration in samples at any time, it is likely to be especially common now. Differentiated impacts on variables such as health, income or employment situation are already evident across individuals, notably across ethnicities, gender and income groups [19], [20]. Disaggregating on the basis of such variables, while always beneficial, may now be of more acute importance given heightened inequalities.

At the individual level, we anticipate that COVID-19 response measures will be associated with important changes in behaviour, as well as cognitive, affective, and other social and material dimensions [21]. Changes in energy-related behaviours and decision-making due to changes in daily routines, work and mobility might be more apparent and measurable, but changes in decisions and actions triggered by pandemic-related shifts in energy-related beliefs, attitudes, emotions, and judgments may be just as important to apprehend. For energy social science research focusing on the aforementioned dimensions it is important to assess to what extent these variables are different from a “normal” scenario and whether potential changes are durable or ephemeral. Epidemiological research demonstrates the effect that pandemic response measures and consequences such as unemployment exert over time on personal well-being [11]). While empirical research on specific COVID-19 measures is emergent, existing theoretical research on the psychological consequences of the crisis indicates that the fallout on current generations will linger in complex ways over time [10].The inability to accurately predict how such changes might be associated with energy-relevant outcomes, or which changes might be more or less enduring [22], makes it all the more important to capture and consider them in the long run.

Where additional measures are included, we suggest the use of standardized approaches to the extent possible, such as widely used and validated scales employed in regular national surveys. This will allow commensurability with pre-COVID-19 levels, while minimising construct and instrument validity challenges and the resource-intensive efforts associated with developing new measures (which require substantial testing to ensure scale reliability and validity).

We show an initial mapping of variables as ‘core’ and ‘consider’, in Table 1 . We also provide a checklist (see Appendix) suggesting where and how to report those additional variables (and other considerations) in studies.

Table 1.

Core dimensions affected by the pandemic and measures to address them, distinguished between core and consider variables.

Dimension	Related concepts	Research design measures	Core or Consider (and possible field(s) of application)
Contextual: Response measures and impacts	-Details of of national and local pandemic-related policies (e.g. measures of lockdown at the time of data collection) -Changes in national and local context (e.g. economic, social, health)	-Add objective data from media, governmental sources, databases	Core for all research.
Demographic	-Level of restrictions applying to individual participant (eg. categories such as key worker) -Changes in household situation (e.g., income, employment, household size) -Differentiated effects (e.g. gender, ethnicity)	-Collect and report additional demographic variables if needed -Add subjective measure of level of restrictions applying to individual -Adapt or add in questions to capture self-reported changes in household situation -Add control variables to capture differentiated effects	-Some likely core for all research.
Behavioral	-Changes in (energy-relevant) behaviours and daily routines as a result of COVID-19 restrictions. -Changes in appliance use, travel behaviours, energy related purchase behaviours, etc. -Changes in other activities such as caring, volunteering, etc.	-Add standardized control questions to assess self-reported changes in behaviour on the individual level -Add objective measures such as energy consumption and mobility -Add qualitative measures to assess changes in daily routines on the individual level -Compare, if possible, data with pre-pandemic data	Consider, especially for: -Research on household, work and mobility energy consumption -Research on energy investment decisions
Cognitive and Affective	-Perceived personal impact of pandemic and measures -Perceived personal constraints vs. benefits due to the pandemic and measures -Perceived uncertainty -Changes in goal and priorities -Positive vs. negative affect towards personal and societal impacts of the pandemic/measures -Distinct emotions toward personal and societal impacts (e.g. worried, hopeful, sad, scared, guilty, stressed, relaxed)	-Add standardized control questions to assess subjective cognitive and affective experiences of the crisis on the individual level -Compare, if possible, data with pre-pandemic data	Consider, especially for: -Research on energy-related judgements and decisions
Social	-Changes in social connection/identity/norms -Perceived need for social proximity -Changes in energy burden, energy technology accessibility for underserved community -Changes in community environmental impact or climate change concerns -Perceived fairness of social distancing policy -Changes in clean energy industry & market	-Add standardised control questions to assess subjective changes in social relationships of the crisis on the individual and group level -Add qualitative measures to assess changes in social relationships on the group level -Compare, if possible, data with pre-pandemic data	Consider, especially for: -Research on social networks and community schemes and energy behavior -Research on energy burden, technology accessibility, and affordability. -Research on impacts on under-served communities (e.g., seniors, low-income, minority, differently-abled people)
Material/ technical	-Material changes in homes, workplaces connected with the pandemic (e.g. IT equipment for home working, clothing) -Digital changes such as service subscriptions (e.g. for video conferencing), cyber security issues	-Add standardised questions	Consider, especially for: -Research on household, work and mobility energy consumption -Research on energy investment decisions

3.2. Consider implications for design, conduct and interpretation of research

When thinking about the potential effects of the COVID-19 pandemic on the validity of research findings, it is also important to consider how it might affect research design. In this section, we briefly introduce issues relating to study design, sample selection and recruitment, and data collection methodology, as well as implications for interpretation of findings. Suggestions on ways to report such considerations are also provided in the checklist (see Appendix).

3.2.1. Study design

It is likely that the pandemic will affect non-experimental research in different ways than it will affect experimental research. Research focused on identifying associations might be especially vulnerable to threats to internal and external validity. More specifically, if the pandemic affects both the independent and dependent variables of interest, it can induce a spurious correlation (confounding; [23]). For instance, the pandemic might harm mental health and increase energy usage, making it appear as though the variables are related when they might not be, absent the pandemic. Researchers can address this concern in the way that is typically recommended for addressing confounding: anticipate how the pandemic might affect your variables of interest, measure this set of variables, and test whether they affect the study’s primary results [24], [25]. The idea that ‘correlation is not causation’ is well-known – but worth keeping salient especially at times when non-experimental research is being planned or altered at short notice.

Experimental designs are still potentially vulnerable to other pandemic-induced issues. Experiments, by design, manipulate a specific variable of interest. For example, an experiment aiming to improve people’s motivation to purchase or support renewable energy by means of messaging strategies might focus on the harm caused by fossil fuels to increase people’s fossil fuel risk perceptions. However, the salience of such risks, and therefore their malleability, may be substantially decreased if people are preoccupied with other worries related to COVID-19. Thus, researchers should consider such influences and, if possible, take measures to ensure that they can indeed manipulate the causal variable of interest in an effective and meaningful way. This is an empirical question for each manipulated variable, but we advise that researchers attempt to anticipate such issues and design their research accordingly.

A clear consequence of the pandemic is that it will make it more difficult to conduct between- and within-country comparisons where COVID-19 impacts and restrictions are different. For example, home energy usage will be higher in places where people are required to stay at home. A useful rule of thumb is, wherever reasonably possible, researchers should contextualize their research by considering how political and cultural circumstances might affect their results (see section “Capture and report on extra relevant data”; [15]). It would be even better to anticipate how such factors might affect results and design the study to mitigate them, such as collecting a sample that is relatively homogeneous in orders to stay at home, limit travel, or any variable that might substantively affect the results. If substantial heterogeneity of restrictions is anticipated within a sample, increasing sample size to maintain statistical power should be considered.

Independent of study type, a powerful way to get a measure of stability and validity of findings over time is to build longitudinal elements into the research design. First, researchers could consider building replications into their research plan. This can be done by intentionally splitting data collection over waves separated by a period of time. This allows for comparison of the variables of interest over the two waves. Variables that remain constant over this period are likely to be relatively less affected by pandemic response measures than those which show variation. This approach lends itself particularly well to collaborations between research groups, which could consider teaming up to add variables of interest reciprocally onto the end of each other’s studies, saving on budget and potentially introducing opportunities for new analyses. Please refer to the section “Capture and report extra relevant data” for more information on variable dimensions to consider.

Another possible approach to demonstrate the robustness of research findings over time could be through attempting to reproduce previous research findings – either of related research by the researcher themselves, or of previously well-reproduced effects. The extent to which previous findings are reproduced, or change, could help ‘calibrate’ the more recent research and give some insight into whether or not the domain of interest is more or less impacted by the pandemic and the corresponding response (also accepting that failure to reproduce findings is not an unusual occurrence even under normal circumstances [26]).

3.2.2. Data collection methodology

Data collection with a given research method could produce different findings now compared to before pandemic-related restriction measures were put into place. For example, research conducted online could be more heavily influenced by distracting factors of the participant’s environment. Where people are confined to their homes, completing a survey or conducting an interview in a standardized way might be more difficult than before. This consideration is especially important, since persisting restrictions of contact might result in a shift towards more research being conducted online versus in person. It is thus recommended to explore the possibility of using more than one method to investigate the same research question, and to record potential limitations specific to a data collection method to account for their influence on the validity of the findings. This is another area where collaboration between research groups with complementary interests could bring significant additional value by allowing testing of the same research question through different approaches and in different settings [27], [28], [29]. While elaborating on additional or alternative methods, it is again important to consider ethical aspects. As mentioned earlier, the extra sensitivity of collected data has to be thought through, and in terms of data collection methods, researchers and analysts should make sure that data privacy and confidentiality is not undermined by new approaches [30].

3.2.3. Interpreting findings and making recommendations

In the previous subsection we already highlighted the importance of giving due consideration to contextual factors. In respect of COVID-19, this means paying particular attention to the extent to which pandemic response measures (and changes in them across time and the sample) might have contributed to the observation of particular results. If possible, researchers should attempt to communicate and justify their best estimate as to the impact such factors could have had on findings. For example, if little systematic difference is observed in an outcome variable across groups who were substantially differently impacted by pandemic response measures, this could be offered in support of a case that the impact of COVID-19 of that particular variable could be small. As in many areas of research, transparency is likely to be key in allowing users to make informed judgements of their own. Any recommendations for policy, practice, or further research should be similarly transparent and include appropriate caveats on the context of the findings to which they relate.

4. Opportunities for research

Employing the principles set out above presents a number of opportunities that go beyond simply mitigating threats to validity, and could help generate new insights or improve research practice in general. The introduction of longitudinal elements can provide important insights on stable and dynamic determinants of energy-relevant outcomes, especially if combined with new contextual, behavioural, and other data that may not previously have been collected. Such longitudinal studies could moreover contribute to the research question whether observed changes on the individual and societal level are caused by the pandemic itself (e.g., due to perceived threats and vulnerability) or by associated measures and consequences (e.g., due to lockdown and job loss) and thus provide insights into short-term and long-term effects of the pandemic. Moreover, where collected data suggests that different groups of people have been (or will be) systematically exposed to different conditions as a result of the pandemic, natural experiments could be possible. Natural experiments provide a powerful opportunity to investigate causal associations which may otherwise be difficult or impossible to control for (for an example see: [31]). These fleeting windows of opportunity can provide novel research opportunities and should be considered by energy researchers. The same window of opportunity will likely extend to policy interventions introduced in the wake of the pandemic to aid economic recovery.

We already highlighted the possible benefits that could accrue from collaboration with other groups to facilitate replication and support validity, but there is also a wider convergence research opportunity in energy social studies during and after COVID-19. Convergence research is a way of addressing complex problems through highly integrated interdisciplinary approaches [32]. Given the range and scale of current and anticipated impacts of the pandemic, such an approach is likely to be especially valuable, and opportunities to build inter- and transdisciplinary collaborations should be proactively sought. Such collaboration may also provide a route to adding in important contextual data, for example through matching datasets.

Finally, we suggest that responding to validity challenges presented by the COVID-19 crisis is an opportunity for the energy research field to step up and embrace practices around transparency and reproducibility that are now seen as standard practice in other areas of research. For reasons likely connected with the multidisciplinary and applied nature of most energy research, tools such as reporting guidelines and pre-registration of analysis plans are still rarely employed [33]. It is possible that the particularly pressing need to demonstrate validity at present will result in familiarity with, and adoption of, tools that subsequently become standard practice for an increasing number of energy researchers, potentially enhancing the overall validity of research in the field.

5. Body of knowledge validity

In much of the social sciences, knowledge on the most severe and pressing problems is often difficult to create and therefore constitutes a smaller proportion of thematic scholarship than its implications merit. The flip side of this is that ‘low-hanging fruits’ can suffer from excessive coverage. This impacts the ‘body of knowledge’ validity, which we define as the representativeness of research in a field relative to the real-world problems the field is concerned with [34]. Energy social science research, with its diverse methodologies, spatial and scalar foci, and associated differences of requisite time and effort, is no stranger to these tendencies. Consider, for instance, the wealth of scholarship on local and urban energy initiatives in the UK, home to many energy research scholars, versus the relatively thin body of work on energy practices in rural Sub-Saharan Africa. Both issues merit attention and are generative for conceptual insight, but the latter affects over a billion people, many of whom experience relatively severe degrees of energy poverty, and yet hardly registers in terms of volume in relevant energy social science research. We detect a risk that curtailment of field-based empirical research, especially in regions that face severe energy challenges and may be heavily impacted by the epidemic, will exacerbate existing biases in representation in terms of volume (more desk study over ethnographic research than usual), methodology (potentially more conceptual work over evidence-based research) and regional coverage (less pandemic-impacted areas over more pandemic-impacted areas).

To some extent, this is a perennial problem in any interdisciplinary or transdisciplinary field of study: ethnographic work in challenging regions with marginalised populations takes time and the classics on such topics that have accumulated over the years (in quite large numbers) consequently receive considerable attention. It is similarly evident in other fields of energy research, such as modelling, and outside of the energy domain. Yet research today is heavily metricised, and most scholars with access to most global peer-reviewed scholarship are based in Global North institutions and typically urban contexts, often with pressure to publish frequently. This leads to the double jeopardy of being pressed for time to focus on short-term impact, and of being far more likely to access highly-cited and high-volume segments of the scholarship one engages with. Since the pace of research outputs has escalated, few scholars are positioned to navigate a body of knowledge with adequate care to balance its in-built biases of representation.

Already, we see moves to run online surveys and study social perceptions; even with all the appropriate caveats and the best of informed intentions, these contribute to a likely disbalance by volume of the sort of concerns that will get platformed in energy research journals in the short- to medium-term. How much coverage of marginalised, hard-to-access concerns – such as migrants cast adrift with little energy access, subsistence farmers with crop loss and inability to pay for fuel costs – will be lost and substituted by low-hanging fruit? Such exacerbation of an existing bias can cloud future accounts and understandings of the true effects of a pandemic on the subject of energy research, i.e., on the global lived experience of energy. But it is not inevitable – it is an artefact of choices we make as an epistemic community. Informed by recognition of likely biases, our choices (and those of funders, who can prioritise research on marginalised research areas) can embody normative commitment to proportionally match research coverage to real-world problems. We can productively draw on approaches such as convergence research highlighted above. This drive captures the essence of our contribution, which is to work toward a reflexive understanding of our role as a scholarly community at this time of crisis and opportunity.

6. Conclusion

In this paper we have set out what we see as important challenges to the validity – internal, external, and of other forms – of social research in energy associated with the COVID-19 pandemic and measures put in place to control it. We have suggested a number of principles we think researchers should consider applying to give themselves and the users of their work confidence that the findings and recommendations they present will still be valid in the years to come. These focus mainly on the collecting and reporting of additional contextual data, and the review of research design elements to ensure they are as robust as possible to pandemic-related impacts.

We think that these principles can be employed with relatively minimal impact on resources and timescales required for research. They even present some opportunities both to enrich insight into social aspects of energy, and draw attention to measures to improve research transparency that are still as-yet under-used in the energy field. However, we also need to be mindful that due to limits on the kind of research approaches that can be employed during the pandemic, there are likely to be important gaps in the knowledge generated during this period. We all hope that the period of direct applicability of this paper will be as short as possible, and that measures to control the spread of COVID-19 will soon no longer be needed. Nonetheless, we also think that the considerations we raise here have enduring relevance for energy social science in general, and the potential to contribute to more widespread use of transparent, contextually aware and valid research practices in the long-term.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix

Checklist of items to report or consider reporting in relation to COVID-19 pandemic validity challenges.

Checklist item	Section	Example
Report:
Main details of COVID-19 response measures in action at the time/place of data collection, at least including: level of freedom to move around in public; degree to which schools and businesses are open.	Methods	“At the time of data collection, public movement in the UK was severely restricted by government measures to combat the COVID-19 pandemic. People were instructed to stay at home at all times, except for doing essential shopping, one period of daily exercise, working outside the home if work at home was impossible, and providing support to vulnerable people. All schools, hospitality venues and non-essential shops were closed.”
Consider reporting:
How COVID-19 restrictions are applying to individual participants.	Results	“In our sample, 65% of participants reported staying at home at all times except for when conducting essential shopping and exercise. A further 25% also reported leaving home to undertake work or volunteering.10% of the sample reported staying at home at all times.”
Consider tailoring of the following aspects of the research:
Research design	Methods	“In response to the rapidly changing circumstances connected with the response to the COVID-19 pandemic, we introduced a longitudinal element to our data collection. The survey was administered over two waves separated by two months, allowing us to check whether any of the key independent variables changed over this time, and whether this was associated with any change in the outcome.”
Sample	Methods	“We anticipated that childcare responsibilities could play a role in [variable of interest]. We therefore selected to draw our sample for [region A], where schools were open as normal.”
Data collected (see Table 1 for suggested dimensions)	Methods	“In addition to employment status, we also collected data on the extent to which those in employment were working from home.”
Consider possible implications for:
Findings	Discussion	“We found a strong association between altruism and stated willingness to participate over both waves of the study. However, the association was weaker in the second wave, which, combined with the change in reported application of COVID-19 response measures (while other variables remained stable), suggests that conditions surrounding the COVID-19 pandemic could have affected this finding.”
Recommendations	Conclusion	“Our findings suggest that policymakers should prioritise energy saving messaging framed in terms of benefits to the local environment. However, our participants reported spending more time in their local area as a result of COVID-19 control measures which could have influenced our result. We therefore recommend that the effectiveness of such messaging be carefully monitored.”