Using temperature sensitivity to estimate shiftable electricity demand

Summary

The growth of intermittent renewable energy and climate change makes it increasingly difficult to manage electricity demand variability. Centralized storage can help but is costly. An alternative is to shift demand. Cooling and heating demands are substantial and can be economically shifted using thermal storage. To estimate what thermal storage, employed at a scale, might due to reshape electricity loads, we pair fine-scale weather data with hourly electricity use to estimate the share of temperature-sensitive demand across 31 regions that span the continental United States. We then show how much variability can be reduced by shifting temperature-sensitive loads, with and without improved transmission between regions. We find that approximately three-quarters of within-day, within-region demand variability can be eliminated by shifting just half of temperature-sensitive demand. The variability-reducing benefits of shifting temperature-sensitive demand complement those gained from the improved interregional transmission, and greatly mitigate the challenge of serving higher peaks under climate change.

Graphical abstract

Highlights

• •Using thermal storage, HVAC-related energy demand can be shifted in time
• •We estimate the large-scale potential of such shifts on the U.S. electricity system
• •HVAC loads are identified by linking hourly electricity demand to fine-scale weather
• •Shifting half of estimated HVAC demand reduces daily variability by 75 percent

Energy resources; Energy policy; Energy management; Energy Modeling.

Introduction

The cost of conventional electric power systems depends on how much demand fluctuates over time. Installed capacity must exceed peak demand, while base load (minimum demand) limits the amount of capacity that can be fully utilized. Thus, the greater the spread between peak and base load, the greater the average cost per kilowatt hour. Conventional system design strives to optimally balance capacity cost, operation costs, and capacity utilization is given demand variability (Luss, 1982; Dangl, 1999; Fripp, 2012). Managing variability is becoming increasingly difficult owing to the growth of intermittent renewable energy and climate change. On a levelized basis, wind and solar photovoltaic (PV) are now the least costly sources of power, but have output that fluctuates with weather and sunlight. As a result, the demand net of renewable supply is more variable than demand itself. At the same time, climate change is increasing the peak demand for cooling on hot summer days (McFarland et al., 2015; Auffhammer et al., 2017). Transmission, batteries, pumped-water hydroelectric systems, hydrogen, and other storage methods can help to smooth net demand, but these are costly.

An alternative to these balancing technologies is to make better use of potentially shiftable demand. Some uses of electricity may be flexible in terms of when they draw power from the grid. Smart systems promise to automate and coordinate shiftable demands in order to improve comfort and convenience for customers while reshaping demand to be less costly to serve (Darby and McKenna, 2012; Paterakis et al., 2015). Such demand-side flexibility could be valuable to the broader power system, but since these resources are distributed and sit “behind the meter” of individual utility customers, incentivizing and enabling them may require a considerable redesign of regulated retail prices and system management. A report, commissioned by the Federal Energy Regulatory Commission, investigated the broader issue of demand response in 2009 (Commission and Group, 2009). That report focuses mainly on the share of peak load that might be curtailed or shifted in each state using a bottom-up approach that examined each kind of use from each class of electricity consumer. The report pays less attention to potential effects on base load, which could also have considerable value (de Mars et al., 2020). A more recent analysis by the National Renewable Energy Laboratory (Mai et al., 2020) takes a similar approach and considers implications of flexibility for a range of uses, but assumes only a limited degree of flexibility (a maximal 1 h shift for HVAC), in conjunction with a single model year (2012). These assumptions were embodied in a recent National Academy of Sciences report on decarbonization pathways (Pacala et al., 2021). As we look forward to planning a future grid with more intermittent renewables, what is needed is a broad assessment of the real-time shiftable demand in each location, time of day, and weather circumstances. Such an assessment can be used in conjunction with synchronized weather data to aid the assessment of the best locations to site renewables, transmission, and storage investments. Flexible demand could ultimately reduce the costs of decarbonization by reducing storage and transmission costs, and might therefore involve very different investment strategies, even in the near term, as compared to systems that pay less attention to such potential.

As an aside, note that demand can also be curtailed through inconvenience or discomfort, which customers willingly do under successful demand response programs (Jessoe and Rapson, 2014; Borenstein and Holland, 2005; Borenstein, 2005; Wolak, 2011). Our focus pertains to demand-side storage or shifting, which may involve costs on the demand side, but not necessarily a loss of utility or discomfort to customers. Location-specific real-time retail pricing and other mechanisms could be used to harness both kinds of demand reshaping.

How much flexible demand exists? While many different kinds of demand may be shiftable, a significant share of electricity demand is used for cooling and heating, and affordable technologies exist, and can likely be improved, to shift these loads. For example, with sufficient insulation, energy used for air conditioning and space and water heating can be stored in ice, hot water, underground materials, or other forms of thermal storage for hours to days, and in some cases seasonally (Arteconi et al., 2012; Darby and McKenna, 2012; Yoon et al., 2014, 2016). The resulting flexibility of demand timing can be used to flatten demand profiles, or in more complex renewable systems, shift demand toward variable supply. Although other flexible uses of electricity exist, heating and cooling demand is substantial and sensitive to ambient temperature fluctuations, and thus estimable over time and space.

In this article, we estimate potentially shiftable heating and cooling demand using a “top down” approach that links hourly electricity demand across the continental United States to fine-grained estimates of hourly temperature over space and time. Our approach bears some resemblance to (Denholm et al., 2012) who identify cooling loads by comparing realized demand in warm months to minimum demand on otherwise similar days. Our approach is more explicit and less susceptible to potentially confounding factors that could be associated with low-demand days and hours; it also accounts for heating-related demand. We use a flexible functional form to account for shifts in demand that arise from seasonal, day of week, and time-of-day effects, as well as geospatial variations in climate, which can factor into temperature sensitivity. Conditional on these factors, weather variation is arguably conditionally exogenous—as if randomly assigned—and thereby constitutes a viable natural experiment to identify temperature-sensitive load that might be shiftable. We cross-validate the model by predicting demand in out-of-sample years and show these predictions to be highly accurate. We then use the model to predict the share of temperature-sensitive demand in each hour and region.

Assuming different shares $\left(\alpha \right)$ of the estimated temperature-sensitive load in each hour can be shifted within each day, we determine alternative feasible “flattened” demand profiles. These calculations are illustrated for one day in one region in the Eastern Interconnect, for $\alpha =0.5$ and $\alpha =1$ (Figure 1). At least in a conventional system, the degree to which demand can be flattened comports with the cost of the overall system, holding total demand fixed. We show how much reshaping demand in this manner can serve to flatten load with and without transmission across regions within and between the three interconnects in the United States: East, West, and ERCOT (Electric Reliability Council of Texas). Finally, we predict demand and estimate shiftable load under a uniform climate change scenario in which all temperatures increase by 2°C.

Figure 1.

Demand, flexible load, hard load, and flattened demand

The graph shows electricity demand for one region in the Eastern Interconnect on December 8, 2016, together with the estimated flexible load (shaded yellow) and remaining hard demand (assumed unshiftable). Flattened demand is constructed by reshaping all (in blue) or half (in green) the flexible load in each hour.

Results

If half the temperature-sensitive load is shiftable $(\alpha =0.5)$, regional daily peak load can be reduced by an average of 10.1%, daily base load can be increased by an average of 22.2%, the daily SD of load can be reduced by an average of 76.9%, and 17.9% of region-days can be flattened completely.

If all temperature-sensitive load is shiftable in practice, these numbers improve: average daily peak load falls by 13.2%, average daily base load increases by 24.3%, the average daily SD falls by 92.2%, and 60.1% of region-days can be flattened completely. Thus, roughly three-quarters of peak-shaving potential and about 90% of the base load increasing potential can be achieved if just half the temperature-sensitive load is actually shiftable within days. If only a quarter of temperature-sensitive load is actually shiftable, then roughly half of the potential flattening is possible. These statistics are summarized in the top row of Table 1. To illustrate the full range of what might be possible from demand-side shifting of temperature-sensitive demand, we show the percent decline in the daily SD for the full range of $\alpha \in [0.01,1]$ (Figure 2). The figure also shows how flattening (SD reduction) varies across regions. See also SD reduction map in Supplemental Information Figure S7. More flattening tends to be possible in eastern regions as compared to western regions, while ERCOT (Texas) has somewhat more potential flattening than the national average, typical of many regions in the Eastern Interconnect.

Table 1.

Changes in daily peak and base load and within-day variability when demand is flattened using different levels of $\alpha$ and regional aggregation

Level of Connectivity	Peak Load Reduction				Base Load Increase				SD Reductionb				Share of Flattenable Days
αa value	0	0.25	0.5	1	0	0.25	0.5	1	0	0.25	0.5	1	0	0.25	0.5	1
All Seasons
Regional BAc	0.0	5.4	10.1	13.2	0.0	16.6	22.2	24.3	0.0	48.6	76.9	92.2	0.0	0.7	17.9	60.1
Interconnectd	0.9	6.4	11.2	13.8	0.6	17.7	23.0	24.3	4.6	55.7	84.9	96.8	0.0	0.4	24.0	68.4
Nationwide	2.5	8.1	12.7	14.1	2.0	18.9	23.3	23.7	12.7	67.1	94.3	99.9	0.0	1.1	42.5	95.7
Winter: Dec/Jan/Feb
Regional BA	0.0	4.4	7.8	10.7	0.0	12.6	16.1	17.7	0.0	52.8	77.2	92.2	0.0	2.5	22.1	59.2
Interconnect	1.1	4.9	8.0	10.9	0.8	13.5	16.7	17.8	7.7	58.3	81.9	96.0	0.0	1.6	20.2	56.9
Nationwide	3.5	7.7	10.5	11.5	2.6	15.3	17.0	17.2	21.1	78.1	95.8	99.8	0.0	4.5	54.7	95.9
Summer: Jun/Jul/Aug
Regional BA	0.0	8.1	15.2	17.8	0.0	22.8	30.2	32.1	0.0	53.5	86.5	97.2	0.0	0.0	30.5	80.8
Interconnect	0.5	9.4	17.1	18.5	0.5	24.0	30.6	31.2	1.5	60.6	95.1	99.9	0.0	0.0	49.7	97.3
Nationwide	1.5	10.3	17.6	18.0	1.8	24.9	30.8	30.9	5.9	65.9	98.9	100.0	0.0	0.0	76.0	100.0

Notes: The table shows the average percent reduction in peak load, average percent increase in base load, average percent reduction in daily SD of load, and percentage of perfectly flattenable days, when all $(\alpha =1)$, half $(\alpha =0.5)$, or a quarter $(\alpha =0.25)$ of temperature-sensitive load in each hour is shiftable to another hour in the same day. These calculations are performed for the individual regions, when regions are pooled within each interconnect, and when all regions across the continental United States are pooled (Nationwide). The $\alpha =0$ column shows how much transmission flattens load. The calculations are also broken out for winter and summer months.

^aShare of temperature-sensitive load that is assumed to be shiftable. α = 0 stands for no load is shiftable, α = 1 stands for all temperature-sensitive load is perfectly shiftable.

^bPercentage reduction in daily SD of load.

^cAggregated Balancing Authority (BA): 15 aggregated BAs in the Eastern Interconnect, 15 aggregated BAs in the West Interconnect, and ERCOT.

^dIncluding Eastern Interconnect, Western Interconnect and ERCOT.

Figure 2.

Proportional reduction in overall load variability for different shares of temperature-sensitive load being flexible

Each line in the graphs shows the reduction in the (A) daily or (B) overall (3-year) SD of demand for a region when raw demand is optimally flattened using $\alpha$ share of temperature-sensitive load within each day. Each line is colored according to the interconnect in which the corresponding region lies. The thicker black line represents the demand-weighted regional average of daily (A) or overall (B) reduction in SD.

Electricity demand is more strongly associated with cooling degree hours (CDH) than with heating degree hours (HDH), while the amount of CDH and HDH varies regionally, with more of both in Eastern regions and ERCOT, which helps to explain why these regions have more temperature-sensitive load and more potential flattenability.

There are clear seasonal and climatic patterns in temperature-sensitive demand, with more concentrated in winter and summer. The summary statistics and regression models show a stronger link between demand and CDH than HDH, but there tends to be more HDH on average, while some southern regions have far more CDH. The greater frequency and response to HDH and CDH in the Eastern Interconnect and ERCOT leads to more potential flattening than in the West, which is milder. We show these relationships for ERCOT in Figure 3; other regions and regression results are reported in supplemental information, including seasonal patterns. Note that Figure 3 does not account for seasonality (hour of year), time of day, or day of week, as do the regression results. Table 1 summarizes flattenability separately for winter and summer months. See also Figure S8 for seasonal SD reduction in supplemental information. Because the overall peak load tends to be in summer, driven mainly by cooling demand, the potential flexibility of this load could be particularly valuable.

Figure 3.

ERCOT Demand in relation to CDH and HDH

The graph shows the ratio of electricity demand to mean demand in relation to average CDH and HDH in the hour, each aggregated over all grids in the region. The graph also shows the (overlaid) frequency distributions of CDH and HDH. Texas has more CDH than HDH, plus a stronger association with CDH than HDH. Graphs for the other regions, 15 in each interconnect, are provided in Supplemental Information (Figures S4 and S5).

The measures reported above pertain to individual days and individual regions. There are two key concerns with these measures: (1) electricity can be traded between regions, using existing transmission within interconnects, or in the future, using new transmission within or between interconnects; (2) seasonal and day-of-week variation also matter for system cost, not just variability across hours within each day. To address these issues, we also report potential flattening statistics by interconnect and nationally in Table 1. These measures pool regional demands and potentially shiftable demands in each hour, synchronizing across time zones, and then flatten all regions simultaneously. This allows us to see how much transmission could aid in smoothing demand, with or without demand shifting. We also break out statistics by season and, separately, calculate how much within-day shiftable load can diminish overall variability—the SD across the whole three-year (2016-2018) time period examined.

When shiftable load is paired with perfect transmission between regions, average daily load variability (SD) can be reduced by 84.9% when all regions within each interconnect are pooled and by 94.3% when all regions are pooled nationwide, assuming just half the temperature-sensitive load is shiftable.

If all temperature-sensitive load is shiftable, the reductions are 96.8 and 99.9%, respectively, and if just a quarter of temperature-sensitive load is shiftable, the reductions are 55.7 and 67.1%, respectively. The size of peak load reductions and base load increases associated with an increasing $\alpha$ are slightly lower when the load is pooled across regions because pooling demand across regions already reduces the peak and increases the base. The share of completely flattenable days increases considerably when the load is pooled and then flattened. All of these statistics are reported in Table 1.

We now consider how much within-day shifting of temperature-sensitive load can reduce overall variability. Demand varies across days, seasons, and years, not just across hours within each day, and variation over this longer timescale is also pertinent to costs. Temperature-sensitive load is likely only shiftable over a short horizon, and our analysis assumes it is shiftable only within each 24-h day. This duration of shiftability is greater than what might be achieved through simple thermostat adjustments, but potentially conservative for some kinds of demand-side thermal storage (Ruan et al., 2016; Heine et al., 2021). Nevertheless, we find that within-day flexibility can do a lot to reduce overall variability.

Assuming half of temperature-sensitive demand can be shifted within days $(\alpha =0.5)$, the overall SD of demand for a typical region can be reduced by roughly 25%.

Larger shares of shiftable temperature-sensitive demand do little to further reduce the overall SD of demand (Figure 2B), indicating that diminishing returns to flexibility arise faster when considering overall variability as opposed to daily variability. We find overall load-flattening potential varies widely across regions, with more generally available in the Eastern Interconnect than in the Western Interconnect.

Because transmission and load shifting both can be used to flatten effective demand profiles, we illustrate the relative importance of each in Figure 4. This figure is used to illustrate a wide range of results. The aqua-colored bars in the graph show the mean peak and mean base load and the whiskers show the 1 and 99% percentiles of peak and base relative to each region’s mean load. The more similar peak and base, the flatter the typical load profile. The lighter colored bars show peak and base relative to the daily mean, while the darker bars show peak and base relative to the overall mean. The choice of baseline mean has little bearing on the size of the bars, but normalizing by the overall mean noticably widens the spread of the whiskers to reflect overall variability across three years not just within-day variability. The panels of the graph stratify calculations across different $\alpha \in \{\mathrm{0,0.5},1\}$ and three levels of transmission: no transmission between regions, perfect transmission within interconnects, and perfect transmission nationally. Thus, the top-left panel shows the raw data, with no flattening, and the bottom right shows the combined effect of full shiftability of temperature-sensitive demand and perfect transmission across all regions.

Figure 4.

The influence of demand flexibility, transmission, and climate change on daily and overall base and peak load

The aqua-colored bars show average values of daily peak and base load divided by the same-day mean (lighter shade) or overall (3-year) mean (darker shade). The red bars indicate the same average values with $+2°C$ change in temperature, normalized by the actual historical load. Whiskers mark the 1st and 99th percentiles of daily peak and base demand. Demand flexibility increases from left to right, where $\alpha =0$ is raw demand (left column), $\alpha =0.5$ is demand optimally flattened using half the temperature-sensitive load, and $\alpha =1$ is demand optimally flattened using all of the temperature-sensitive load. Transmission increases from top to bottom, where the first row assumes no connectivity between regions, the second row assumes perfect transmission within interconnects (Eastern, Western, ERCOT), and the last row assumes perfect transmission across the contiguous United States.

Moving from top-left to bottom right, the bars become nearly equal in height and the whiskers narrow. Although improved transmission does act to reduce the effective peak and increase the effective base (moving from top to bottom panels), we see considerably more flattening arising from shifting of temperature-sensitive load (moving from left to right panels). Although the mean peak and mean base loads quickly converge to the overall mean as $\alpha$ increases, indicating strong smoothing overall days, the whiskers on the darker bars indicate that between-day/between-region variability remains, but is markedly diminished. Transmission does more to diminish the spread of the whiskers than equalize the height of the mean peak and mean base bars, both with and without flattening of temperature-sensitive demand. In this sense, improved transmission can complement demand-side smoothing, as we detail later in discussion.

Shifting of temperature-sensitive load does more to reduce variability than improved transmission.

Whereas perfect transmission between all regions can increase the effective mean base load from 80.7% of the overall mean to about 82.7% of the overall mean, and reduce the effective mean peak load from 118.2% of the overall mean to 113.8% of the overall mean, shifting just half of the temperature-sensitive load $(\alpha =0.5)$ increases the effective mean base load to 97.8% of the overall mean and reduces the effective mean peak load to 105.9% of the overall mean. With perfect transmission and shiftability of all temperature-sensitive load $(\alpha =1)$, the effective mean base load can be increased to 100.0% of the overall mean and the effective mean peak can be reduced to 100.0% greater than the overall mean; i.e., demand can be perfectly flattened within days. If we look at the first percentile of overall base load and the 99^th percentile of the overall peak, we find perfect transmission increases the base from 56.4% to 69.6% of the overall mean while reducing the peak from 179.4% to 152.1% of the overall mean. For comparison, flattening within regions using just half the temperature-sensitive demand increases the base to 70.4% and reduces the peak to 144.9% of overall demand. With perfect nationwide transmission and flattening with half the temperature-sensitive demand, the overall base rises to 79.6% and the overall peak falls to 127.1%. These statistics and many more are reported in the Supplemental Information (Tables S2, S3, S6, and S7), and displayed in Figure 4.

Shifting temperature-sensitive demand can mitigate impacts of climate change.

Our analysis of climate change impacts is straightforward: we shift the historical distribution of temperatures 2°C warmer in each hour, project the change in total demand and share flexible load using the regression model. To illustrate how climate change affects peak load, base load, and overall flattenability of loads over space and time, we include red bars and whiskers in Figure 4 that show climate-shifted base and peak loads, normalized by the historical means instead of the climate-shifted means. This normalization allows us to see how mean, base, and peak loads change with climate, both before and after accounting for changing shiftability. The impact of climate change holding all else the same (without shifting), is illustrated in the top-left by the increase in the peak red-colored bars relative to the aqua-colored bars, and by the increase in the peak top whisker. We find a 2°C increase in temperature causes average daily peak demand to increase from 118.6% to 122.5% of historic daily mean and from 118.2 to 121.9% of the historic overall mean, while the 99^th percentile of peak load increases from 179.4% to 191.0% of the overall historic mean (top-left panel of Figure 4). These predicted changes are similar to those predicted in an earlier study (Auffhammer et al., 2017), which estimated a 3.5% increase in average peak and a 7% increase in the 95^th percentile of peak load for the RCP 4.5 warming scenario. Our estimates imply 3.9 and 11.0% increases to the same measures, but our estimates pertain to a slightly warmer climate change scenario than RCP 4.5, owing mainly to the recent baseline of 2016-2018.

We find that climate change has relatively little influence on base load, increasing from 80.4% to 81.1% of the historic daily mean and from 80.7% to 81.2% of the historic overall mean. When shifting all flexible loads, the climate scenario causes the average daily base load to increase from 99.6% to 102.0% of the historic daily mean and from 99.5% to 101.7% of the historic overall mean.

Climate change also increases the peak of shifted and pooled demands, but by a smaller amount. The average peak of flattened demand increases from 105.5%/105.9% to 107.7%/107.9% of the historic daily/overall mean when $\alpha =0.5$, and from 102.0%/102.2% to 104.2%/104.3% when $\alpha =1.0$. The 99^th percentile of the peak demand increases from 118.0%/144.9% to 121.3%/151.2% of the historic daily/overall mean when $\alpha =0.5$, and from 114.7%/142.1% to 114.9%/149.4% of the historic daily/overall mean when $\alpha =1.0$. Thus, reductions in peak load stemming from improving shiftability are far larger than increases from climate change. Table S3 in the supplement reports the 1%, 99%, and mean values of base and peak loads using daily and overall mean baselines, with and without climate change, for $\alpha \in \{\mathrm{0,0.5},1\}$, and for all levels of transmission (regional, interconnect, and national aggregation).

Discussion

Although there is uncertainty about how much temperature-sensitive electricity load might actually be employed to reshape the timing of demand under appropriate policies, and over what duration of time load might be shifted, plausible use of existing technologies and the timing and volume of temperature-sensitive demand indicate that the potential is large enough to have a profound influence on the cost and resiliency of power systems. This conclusion holds if just a fraction of temperature-sensitive demand is actually shiftable within one day.

It is likely that the temperature-sensitive demand that we identify derives mainly from the use of HVAC systems. The total share of temperature-sensitive load that we estimate (19.0%) roughly matches the share of demand attributed to HVAC systems by others (see STAR methods for how we calculate this share). For example, the Energy Information Administration reports that HVAC systems comprise 26% of use from both residential and commercial sectors while attributing 72% of total demand to these sectors, implying a total HVAC share of about 18.7%. Industrial uses, which comprise most of the remaining demand, appear to have little HVAC. Although our estimates cannot breakout temperature-sensitive demand by user type, we can estimate how it varies by location, season, and hour of day, which is valuable data that can be used to aid in planning the grid of the future. It may be that heating and cooling loads have different shiftability at different costs, such that $\alpha$ or the duration of shiftability may differ by the user and building type, as well as the particular technology employed. Our intent here is to show in a transparent way a plausible range of what is likely possible.

A substantial commercial market for demand-side thermal storage has existed for many years and is anticipated to grow substantially with renewable energy. A report by Allied Market Research estimates a global market for thermal energy storage of $20.8 billion in 2020, about two-thirds of which is demand-side cooling and heating (https://www.alliedmarketresearch.com/thermal-energy-storage-market), and projects it to grow to $50.3 billion by 2030. A substantial academic literature also continues to flourish. For example, Yan and Yang (2021) report 448 articles on thermal storage published in Applied Energy over just ten years, 2009-2018. If thermal storage can shift energy demand relatively cheaply, it raises the question of why these systems have not been adopted more broadly and incorporated into system planning. The answer likely stems from a combination of transactions and retrofit costs, system benefits, and incentives, which vary widely across jurisdictions. Incentives to store energy on the demand side of power systems (“behind the meter”) depend on retail pricing schedules, which are regulated. Retail prices need to vary significantly enough for customers to profit from using storage to buy more when prices are low and less when prices are high. Some utilities offer “time of use” prices, critical peak pricing, compensation for demand reduction during the system peak or to provide regulating reserves, and in rare cases, location-specific real-time marginal cost pricing (LMP) (Albadi and El-Saadany, 2008). Availability of these tariffs, which are regulated by state Public Utilities Commissions, varies widely across utilities and customer classes, and may or may not reflect the full value of storage to the system.

While LMP might instill appropriate incentives, it tends to be resisted by regulators. Part of the resistance to LMP likely comes from wary customers who have no knowledge of the potential savings and reticence of state public utility commissions that may fear public backlash from extreme spikes in prices, like those that occasionally arise in Texas, which embraces markets more than most States (Departo, 2019). At the same time, a current regulatory practice sometimes rewards (implicitly) high-cost, centralized solutions that require more capital expenditure, some of which may be avoided with effective demand-side management. This may help to explain why utilities rarely market LMP pricing to customers, even when such contracts are made available Goldman et al. (2006). Moreover, LMP would only instill appropriate incentives for demand-side storage in an idealized, perfectly competitive market, which the electricity sector is not. Given the regulated nature of electricity systems, even in situations where LMP has been implemented, demand-side investors would need to have reasonable expectations about centralized decisions, which, among other departures from perfect competition, could include subsidized storage (e.g., such as batteries paired with renewable energy installations) and regulated capacity decisions. Of course, demand-side storage systems can also be subsidized to help overcome investment hurdles.

Despite these obstacles, there has been a notable long-term success with LMP for commercial customers in some markets, like Georgia Power (Barbose et al., 2004). Because investments in thermal storage and other devices that could enable demand-side flexibility are long lived, the electricity customers that install them may need assurance about their ongoing value, which involves both technological and regulatory uncertainty. A recent decision by the Supreme Court, however, paired with a new rule by the Federal Energy Regulatory Commission (FERC Order No. 2222), may facilitate greater use of demand-side resources, despite limited availability of LMP and other resisting factors. These rulings should enable independent integrators to harness demand-side resources directly from customers and sell them into wholesale markets as virtual power plants. As state public utilities commissions begin to implement the new rules, system-wide stakeholders and policymakers will need to anticipate (and might beneficially encourage) these impending changes and adjust broader investment decisions accordingly. The estimates we present here might aid this planning.

Invariably, as this study shows, there is a joint dependency between the value of energy storage and other investments in the power system, including the capital cost and flexibility of generation resources, the cost of fuels, transmission, and the cost of centralized storage. Although far from a comprehensive assessment of benefits and costs, this study provides some sense of scope and scale of demand-side resources related to heating and cooling demand as well as calibrated data resources that can be used to better assess this potential value in conjunction with other investments. In particular, because future power systems are likely to have much higher shares of low-cost intermittent renewables, shiftability is likely to be more valuable than suggested here, but will require different and far more complex kinds of shifts. Instead of simply flattening the demand profile, demand will need to be shifted toward variable supply. The value of transmission in such a system will also be greater, but different and more complex, because some regions of the country have more renewable energy potential than others, owing to heterogeneity in available wind, solar radiation, and land resources. Diversity of renewable resource assets will also be valuable as this will serve to even out supply and thereby substitute for storage and demand flexibility. In evaluating the best mix of investments, inclusive of the demand side, it is important to have a synchronized account of supply (wind and solar), demand, and demand flexibility, all of which are all weather dependent and typically associated.

Thus, the nature of shifts that will be most valuable in the system of the future will be closely intertwined with the nature and locations of investments in renewable energy resources and new transmission builds. As a result, the time and location-specific estimates that we have developed for potentially shiftable demand should be paired with other estimable sources of shiftable load, like electric vehicle and hot water demand, and incorporated into state-of-the-art capacity expansion models of the future electric grid. To our knowledge, few capacity expansion models are well-equipped to handle intermittent renewables, much less incorporate a demand side with varying amounts of shiftable load. One exception is GenX, a state-of-the-art capacity expansion model developed by MIT and Princeton, which has been used to co-optimize power-system investments while assuming demand shiftability as estimated by the National Renewable Energy Laboratory (Mai et al., 2020). As we noted above, this study develops “ground up” estimates of electricity demand for a single year (2012) that may not align with actual realized demand. They also assume that HVAC demand can be shifted only 1 h, while thermal storage technology can facilitate considerably larger shifts. Another exception is SWITCH, the open-source software developed by Johnston et al. (2019), which can also account for intermittent renewables, storage, and the full joint-dependency of supply and demand. Imelda et al. (2018) use this model in a capacity expansion model to estimate the high potential value from LMP pricing that would employ both supply and demand sides of the electricity market for balancing. This study indicates a very high value from shiftable demand, much of which is identified as HVAC load, which is assumed to be shiftable within a day, as we do here. This particular application, however, is narrow in geographic scope and therefore cannot consider the full range of possibilities on a continental scale where transmission and larger seasonal variation will present different challenges and opportunities. Nevertheless, it is likely that such flexibility will have a powerful influence on least-cost decarbonization plans, including the best timing, sizing, and locations of near-term investments like transmission, renewable energy installations, and various forms of storage.

Limitations of the study

Although we do not consider the costs of thermal storage here, including thermal loss and capital expense, they do appear modest (Heine et al., 2021). Cost-effective installation of thermal storage systems would likely be paired with the installation of HVAC systems that might also improve efficiency and save costs, like air or ground-sourced heat-pump systems. The incremental cost of adding thermal storage will likely vary widely depending on location, scale, and whether it is a new building, a retrofit of a full HVAC system, or simply addition of storage to an existing system. Other factors affecting costs include the durability of the system and its maintenance costs. Presumably, the costs of such systems and their maintenance would depend on the scale of adoption, potentially enabling economies of scale and learning-by-doing.

Other kinds of flexible loads, besides those that we can identify statistically, are likely shiftable using demand-side thermal storage. Water heating might be the easiest current use of electricity that is shiftable (Buescher et al., 2015; Passenberg et al., 2016; Ali et al., 2019). Although some water-heating demand may be captured by our estimates, the timing of water-heating demand tends to be more closely associated with use, not with ambient temperature, and thus less estimable in this manner. The Energy Information Administration reports that water heating comprises 10% of residential electricity demand and 2% of commercial demand (Energy Information Administration, 2015). Electrification of gas water heaters might grow this share, while the adoption of heat-pump water heaters could improve overall efficiency while maintaining and growing shiftability. In time, electric vehicles (EVs) could be an even more flexible source of demand than water heating. EVs can be charged at strategically appropriate times in an automated fashion, not only to shift loads, but to help regulate frequency and even provide power to the grid under certain conditions (“vehicle to grid”) (Noel et al., 2018; Zhang et al., 2020). Some kinds industrial processes might also be shifted at low cost, while some heavy computing loads have already been shifted. For example, the widely noted success of Georgia Power’s real-time pricing tariffs for commercial customers has been attributed mainly to shifting of industrial loads (Barbose et al., 2004). Also, see Google’s effort to shift server demand in an effort to reduce pollution emissions, https://www.blog.google/inside-google/infrastructure/data-centers-work-harder-sun-shines-wind-blows/. Other techniques would be required to estimate such loads over time and space. In addition, electrification of heat, which has become more economic with improved electric heat pumps, could make winter demand much higher, but potentially more flexible, and will likely be a necessary component of future decarbonization efforts. Indeed, ground-source heat pump systems, which are effective even in the coldest US climates, would embody some degree of thermal storage over a seasonal scale (not considered here). Thus, the response to HDH is likely grow, likely in a manner that is estimable over time and space. A natural extension of this study could use the relationships to estimate how and where demand and demand shiftability will increase with further electrification of heating systems. For all of these reasons, even the high-end $(\alpha =1)$ estimates of potential flexibility estimated here could be conservative for a forward-looking decarbonization plan.

Our analysis cannot distinguish between different types of electricity users, but given commercial and residential demand comprise similar shares of both total demand and HVAC demand, plus the possibility that some thermal loads derive from industrial demand, it is likely that half or more of temperature-sensitive demand derives from large, non-residential users. Specifically, the Energy Information Administration’s Annual Energy Outlook, reports that 37% of electricity demand is residential, 35% is commercial, and 27% industrial, and the rest is mainly transportation (Energy Information Administration, 2015). Also see https://www.epa.gov/energy/electricity-customers). Such users are likely to operate at a scale that would make an investment in power-shifting devices more economic and thus more likely to be adopted under appropriate incentive policies, although we do not have data in this study to decompose commercial, industrial, and residential customers, much less size classes.

STAR★Methods

Key resources table

REAGENT or RESOURCE	SOURCE	IDENTIFIER
Software and algorithms
R	The R Project for Statistical Computing	r-project.org
This paper	Interpolate and integrate data from public sources. Algorithms and analysis.	https://github.com/Sisihawaii/Shiftable_electricity
Other
8 x daily gridded temperature data	NCEP North American Regional Reanalysis (NARR)	psl.noaa.gov/data/gridded/data.narr.html
Hourly electricity demand data	EIA-930 Hourly Electric Grid Monitor	www.eia.gov/electricity/gridmonitor/about
Electricity System Control Areas (shapefiles)	Homeland Infrastructure Foundation Level Database (HIFLD)	hifld-geoplatform.opendata.arcgis.com/datasets/control-areas

Resource availability

Lead contact

Further information and requests for resources should be directed to and will be fulfilled by the lead contact, Michael J. Roberts (mjrobert@hawaii.edu).

Materials availability

This study did not generate new unique reagents.

Method details

Data

All data used in this analysis are publicly available. All code used in data compilation and estimation will be made publicly available upon publication.

Cleaned hourly electricity demand from the Energy Information Administration (EIA) form 930, were obtained from Ruggles et al. (2020) for all balancing authorities for three calendar years, 2016–2018. For both the Eastern and Western Interconnects, we aggregated hourly load into 15 regions. We aggregated balancing authorities based on spatial overlap of coverage areas as indicated in shape files obtained from the Department of Homeland Security. Specifically, we found the overlapping area of each balancing authority with every other balancing authority and merged those with largest area overlapping. This process was repeated until 15 balancing authorities remained in Eastern and Western Interconnects. The idea with aggregating balancing authorities in this manner was to better match geospatial variation in weather to reported demand, while preserving regional areas likely connected via transmission. These aggregated regions are plotted in figures in the online supplement. Note that the aggregated regions still have areas of overlap with other regions.

Gridded air temperature $\left(T\right)$ data at two meters of elevation (“air.2m”) were obtained from the the National Oceanic and Atmospheric Administrations North American Regional Reanalysis data (NARR). These data give three-hour measures on a roughly 30 kilometer grid. We linearly interpolate the three-hour data to convert to hourly data. Two key weather measures are used, cooling degree hours (CDH) and and heating degree hours (HDH), where

$$\text{CDH}=\max \{T-\mathrm{18,0}\}$$ (Equation 1)

and

$$\text{HDH}=\max \{18-T,0\}$$ (Equation 2)

To merge these temperature measures with the regional demand data, we overlaid the gridded weather data with roughly 1 km gridded population data, and calculated population-weighted averages over each region. Gridded population data were obtained from NASA’s Socioeconomic Economic Data and Applications Center. Given the non-linearity of CDH and HDH, we first calculate the measures for each NARR grid cell before aggregating to each region. If we were to instead find the average temperature in a region and then calculate CDH and HDH, estimates would be biased. For example, the mean temperature in a region for a particular hour might be 18°C, while some areas within the region may be cooler and some areas warmer, giving rise to positive CDH and HDH. If we were to first average temperature and then calculate CDH and HDH, both measures would be spuriously measured as zero.

To illustrate the link between demand and temperature in each region, we plot the average demand anomaly (demand divided by mean demand) in each region at each average temperature, breaking out separate measures for each season (Figures S1–S3). The graphs of regions in the Eastern and Western Interconnects are ordered such that the warmest regions are in the top left and the coolest region is in the bottom right of the panel of graphs. This ordering is apparent from the frequency histograms of temperatures plotted below each region’s panel. The graphs generally show a “U” shape of load in relation to temperature, with a minimum near 18°C, which is consistent with convention. The “U” shape, with a lightly flatter slope near the minimum, which may result from spatial averaging as suggested above, as well as from temperature associated time-of-day effects. We also show each region’s relationship with HDH and CDH in Figures S4 and S5 (ERCOT is shown in the body of the paper), which appear slightly more linear. While the slopes in relation to HDH and CDH are similar across regions, they do differ. This may occur because different regions use different sources of heating and cooling, due to differing shares of industrial or other non-temperature sensitive loads, and because different regions experience different climates. For example, in typically temperate climates that rarely experience very hot temperatures, fewer customers may choose to install air conditioning, and such customers would respond less to extreme temperatures when they do arise. In the Eastern Interconnect, we see a stronger link between HDH and relative demand in warmer areas than in colder areas, presumably because colder areas in the Northeast are less likely to use electric heat. Also, homes in the Northeast are presumably better insulated than homes in the South that experience milder winters.

Regression model

The regression models we use for estimating temperature-sensitive load employ a flexible functional form linking demand to hour of day (hod), hour of year (hoy), day of week (dow), and standard weather metrics, heating degree hours (HDH) and cooling degree hours (CDH). Each region was estimated separately. A simplified representation of the functional form is:

$$d_t=s_h\left(ho{d}_t\right)\ast s_d\left(ho{y}_t\right)\ast I\left(do{w}_t\right)+{\alpha }_{h}HD{H}_t+{\alpha }_{c}CD{H}_t+u_t$$ (Equation 3)

where $d_t$ is the natural log of hourly demand, $s_h\left(ho{d}_t\right)$ is a natural spline of the hour of day (1–24) in period $t$, $s_d\left(ho{y}_t\right)$ is a natural spline of the hour of year (1–8760), and $I\left(do{w}_t\right)$ is a vector of day-of-week indicator variables that take on a unique fixed value for each day of week. The ‘$\ast$’s indicates that these explanatory variables are interacted, such that separate $s_h\left(\right)$ and $s_d\left(\right)$ splines are, in effect, estimated for each day of the week, and the interaction between $s_h$ and $s_d$ implies that hour-of-day effects are allowed to change smoothly over seasons. That is, each column of a spline design matrix is multiplied with each column of the other spline design matrix, with an additional coefficient estimated for each interaction.

We can formalize the complete model as follows. Define $D^h$ and $D^d$ as the design or basis matrices for $s_h\left(\right)$ and $s_d\left(\right)$, which have dimensions $N\times K$ and $N\times L$ when $s_h\left(\right)$ has $K+1$ knots and $s_d\left(\right)$ has $L+1$ knots. Denote the $k^{th}$ column of $D^h$ as $D_k^h$, and the $l^{th}$ column of $D^d$ as $D_l^d$. Finally, redefine $I$ as an $N\times 7$ matrix of indicator variables for the day of the week, and $\circ$ as a Hadamard product. The resulting regression model is:

$$d_t=\sum _{k\in 1,K}\sum _{l\in 1,L}\sum _{m\in \mathrm{1,7}}{\beta }_{k,m,l}{D}_k^h\circ D_l^d\circ I_m+{\alpha }_{h}HD{H}_t+{\alpha }_{c}CD{H}_t+u_t.$$

Model selection mainly comes down to a choice of the number of knots in each spline. These were selected by 3-fold cross validation, wherein the model was estimated three times, with two of the three years used in estimation, and one full year of data reserved to assess out-of-sample fit. We select the number of knots based on out-of-sample fit, and select a separate optimized value for each region. For example, region 1 in the Eastern Interconnect, summarized in Table S1 uses 19 knots for the hour-of-day spline and 6 knots for the hour-of-year spline. The number of degrees of freedom for each spline equals the number of knots. Accounting for the interactions, the total number of parameters estimated equals $(19\times 6\times 7+2=800)$, which is relatively parsimonious given we observe over 26,000 observations for each region (3 $\times$ 365 $\times$ 24).

The summary statistics presented above suggest that climate may influence sensitivity of demand to weather across regions, and these differences will be accounted for since we estimate separate models for each region. We also consider a specification that estimates heterogeneous responses to weather within each region. For each 30km weather grid cell we multiply the cell’s 12-year mean CDH by realized hourly CDH, and similarly calculate each cell’s mean HDH and multiply by hourly HDH. We then aggregate these interactions to the region level by taking the population-weighted average, as we did with CDH and HDH. These covariates allow the marginal responses to CDH and HDH to change linearly in relation to climate measured as the mean of CDH and HDH. As with CDH and HDH, it is important to account for this interaction on the smallest geographic temperature measure before aggregating.

The regressions generally show that a large share of the residual variance remaining after flexible spline controls for hour-of-year and time-of-day was explained by CDH and HDH. These two weather variables alone typically reduce the RMSE by about 50%, and increase the out-of-sample R² from about 0.6 to 0.9. Using Newey-West robust standard errors to account for the autocorrelated error, find t-statistics on the order of 250 for these weather variables. When we add the interaction terms to the model, fit tends to improve slightly, increasing the out-of-sample R² by 0.01 or less, but the size and sign of the interaction variables differs greatly across regions. In some cases, for some sub-regions, these interaction terms cause the marginal effect of CDH or HDH to turn negative. We believe some of these results are likely spurious, stemming from the coarseness of our weather measures in relation to population and the rough link between population and demand. Nevertheless, we found inclusion of the CDH and HDH in the regressions have little influence on the estimated share of flexible demand. The interactions did, however, cause some highly implausible projections in a couple regions under climate change, so we opted to use the simpler model without these interactions for the main results. Table S1 summarizes regression results for one region and Figure S6 summarizes the regressions for all regions.

The share of temperature-sensitive demand

Seasonal (hour-of-year), day-of-week, and hour-of-day patterns in electricity demand account for 60% of demand variance $(r\approx 0.75)$ over time for a typical region, but this does vary across regions. This measure is derived from the out-of-sample fit (R²) from a baseline regression of hourly demand on a hour-of-year spline interacted with an hour-of-day spline and day-of-week indicator variables, as described above. The interactions of time-of-day and day-of-week allows the time-of-day effects to be different for each day of the week, and interactions with hour-of-year allow these effects to change smoothly over the seasons. We select the number of knots in each spline using cross-validation, reserving a full year of out-of-sample data to assess fit in each fold. All reported fit metrics pertain to the out-of-sample predictions in comparison to actual demand.

Remaining demand variability is driven mainly by weather. Although weather likely underpins much of the seasonal and hour-of-day variation in demand, we explicitly control for these variations because they also derive from other factors, and could confound the weather link. Weather anomalies, in contrast, isolate as-if random variations from which we are more likely to infer a causal link. Demand variability related to short-term fluctuations in temperatures are also more likely to be shiftable. To estimate this relationship, we find population-weighted exposure of cooling and heating degree day hours (CDH and HDH), as well as overall average heating and cooling degree day hours in each location to account for climate. The weather measures are derived from 3-hour, 30 km resolution climate reconstruction data, interpolated to hourly measures. Because degree-day measures are nonlinear in temperature, we first calculate HDH and CDH for each grid cell and population-weight the grid-cell measures to derive regional aggregates. To allow sensitivity to change with climate, we interact long-run average measures (12 years) with hourly outcomes at the grid level, and aggregate the interactions. Adding HDH and CDH to the specification improves the out-of-sample fit to over 90% in most regions (r $\approx$ 0.95). Adding the interactions with climate can improve fit, but modestly, and has no apparent influence on the results. We therefore focus on results from the simpler model without climate interactions. Tables and illustrations of fit and sensitivity to weather and climate variables are reported in Supplemental Information (Figure S9, Tables S1, S4, and S5).

We then estimate, for each region, what electricity demand would have been with zero heating or cooling degree hours (i.e., a constant temperature of 18°C). We call this estimate hard demand. If we define $Y_{it}$ as the observed demand in region $i$ and hour $t$, $\widehat{Y_{it}}|T=T_{it}$ as the model-predicted demand, and $\widehat{Y_{it}}|T=18°C$ as the predicted value at a constant 18°C, then we estimate the maximum possible share of shiftable load in that region and hour as

$$S{S}_{it}=\alpha \frac{\left(\widehat{Y_{it}}|T=T_{it}\right)-\left(\widehat{Y_{it}}|T=18°C\right)}{\left(\widehat{Y_{it}}|T=T_{it}\right)}\text{,}$$ (Equation 4)

where the parameter $\alpha$ is the share of temperature-sensitive load that is assumed to be actually shiftable.

Flattened demand

To determine “flattened” demand profiles for each region and day, we developed a script that sorts hourly hard demand (as described above) from lowest to highest, and then adds flexible load to hard demand, filling and leveling across hours, until all flexible load has been exhausted. Remaining hours—those with the highest hard demand—are not flattenable. The computational script simply adds a day’s total flexible demand to the cumulative sum of hard demand sorted from lowest to highest, and divides this cumulative sum by integers one through 24. This vector of length 24 will decline until all flexible load has been distributed to the lowest hard-demand hours, such that the value at the minimum equals the flattened minimum. If peak hard demand is less than mean demand, then the day is completely flattenable—a constant load is achievable. We executed this script for each day in each region. All scripts are available on Github.

These flattened demand profiles are not necessarily the most desirable reshaping of demand that would arise from an optimized power system. They are presented as emblematic of the nature and scale of reshaping that is potentially achievable and could significantly reduce costs in the conventional power systems that currently predominate.

For each region and each hour, we calculate the degree to which electricity load can be flattened for $\alpha \in \{0.01,0.02,\dots ,1\}$. For each region and day, we calculate the percent reduction in the standard deviation (SD) of load, the percent reduction in peak load, and the percent increase in base load, when demand is optimally flattened using available shiftable load $\left(S{S}_{it}\right)$. The percent reduction in SD is calculated as $100\%\times \left(1-\frac{\text{SD}\left(\text{flattened demand}\right)}{\text{SD}\left(\text{raw demand}\right)}\right)$. These calculations are made on both daily and overall (years 2016–2018) time scales.

Effects of improved transmission

Demand profiles across regions is imperfectly correlated due to differing time zones and weather. Because transmission can be used to spread supply over larger geographical areas, it effectively flattens the relative variation in total demand while taking advantage of scale economies and regions with comparative advantages in generation at any point in time. We therefore consider load profiles at three geographic scales: (1) regional aggregations of balancing authorities (15 regions in each of the Eastern and Western Interconnects, and one in ERCOT, described above); (2) interconnect-level aggregation (Eastern, Western, and ERCOT), which assumes perfect transmission within each interconnect; and (3) national-level, which assumes prefect transmission between all three interconnects that comprise the lower-48 states. While power flows at the regional level currently face fewer constraints, transmission between the interconnects is extremely limited. Comparison across these different levels of aggregation provides some insight into the potential value of demand-side flexibility relative to improved transmission within and between interconnects. It is important to note that transmission has benefits beyond variability management. It also allows for transfer of power from resource-rich areas, like sunny deserts, rich in solar power but with few alternative land uses, to resource-poor areas, like cloudy and densely populated urban areas.

To find flattened demand curves at interconnect and national levels, we synchronized hard and flexible demand in each hour and aggregated across candidate regions, and then flattened the pooled regions as described above.

Climate shifted demand

We estimate raw demand under simple $+2°C$ climate change as

$$Y_{it}^{+2°C}=\left(\widehat{Y_{it}}|T=T_{it}+2°C\right)+\widehat{u_{it}}$$ (Equation 5)

where $\widehat{u_{it}}$ is the residual from the regression model in Equation 3. We calculate the share of shiftable load under climate change in the same manner as historical demand,

$$S{S}_{it}^{+2°C}=\alpha \frac{\left(\widehat{Y_{it}}|T=T_{it}+2°C\right)-\left(\widehat{Y_{it}}|T=18°C\right)}{\widehat{Y_{it}}|T=T_{it}+2°C}.$$ (Equation 6)

Given the amount of warming that has already taken place, adding $+2°C$ to historical values implies substantial warming relative to the pre-industrial period.

Published: September 16, 2022

Data and code availability

• •This paper analyzes existing, publicly available data. These accession numbers for the datasets are listed in the key resources table.
• •All original code has been deposited at [Shiftable_electricity]: https://github.com/Sisihawaii/Shiftable_electricity and is publicly available as of the date of publication. DOIs are listed in the key resources table.
• •Any additional information required to reanalyze the data reported in this paper is available from the lead contact upon request.