On the impact of increasing penetration of variable renewables on electricity spot price extremes in Australia

Abstract

In energy-only electricity markets, such as Australia’s National Electricity Market (NEM), it has been argued that an increasing penetration of variable renewable energy (VRE) generation is likely to have two effects: (i) more extreme spot prices, with greater instances of both very high and very low prices and (ii) a need to increase the market price cap (MPC) and related price signals for reliability. This article examines the validity of both these effects using spot pricing outcomes in South Australia (SA), which has one of the highest VRE penetrations worldwide. We find partial support for these two effects. While extremely low prices have become more frequent over time, extremely high prices have become less frequent. Spot price volatility has risen, consistent with the hypothesis, but not because prices have become more extreme. Furthermore, these findings are observed for prices in all NEM regions, not just SA. Also, reliability has remained high over the past decade despite the MPC remaining constant in real terms. We provide four reasons why higher VRE penetration need not result in more extreme prices and higher MPCs: (i) greater investment in volatility-dampening, reliability-enhancing technologies like storage and interconnectors; (ii) increased contract cover; (iii) more price-responsive demand; and (iv) emergence of additional ancillary service revenues. These findings have implications for the durability of the NEM’s energy-only design given expected further increases in VRE penetration rates across the NEM.

1. Introduction

The past decade has seen a dramatic increase in the penetration of wind and solar PV – the two mainstream forms of variable renewable energy (VRE) generation in Australia – in Australia’s National Electricity Market (NEM). Over the year to 2007, small- and utility-scale wind and solar PV comprised less than 1 per cent of NEM generation, compared to around 14 per cent over the year to 2019 (AER, 2019).1 Moreover, across regions and nations, South Australia (SA) has one of the highest utility-scale2 VRE penetrations in the world, exceeded only by Denmark (Fig. 1).

Fig. 1.

Penetration of utility-scale VRE generation, 2018.

AER (2019) and International Energy Agency (IEA) (2019).

This surge in VRE penetration has been driven by a combination of declining costs for VRE generation – initially wind, but more recently solar PV – and policies aimed at reducing the emissions intensity of electricity generation. Today, the cheapest form of new generation technology in Australia is wind on a levelised cost of electricity (LCOE) basis, though solar PV is expected to overtake wind as the cheapest form of electricity generation (BNEF, 2019).

In energy-only markets like the NEM, an increasing penetration of negligible short run marginal cost (SRMC) VRE plant has been hypothesised to have the following two impacts on prices:

• 1.Prices become more volatile: the issue with VRE generators is not just that their output is variable; it is also that their output is poorly correlated with demand. This is especially true for wind output in SA, which is typically negatively correlated with demand (Cutler et al., 2011, Rai and Nunn, 2020). Hence, supply becomes harder to equilibrate with demand – with this manifested via higher price volatility and price extremity – as the penetration of variable renewables increases.
• 2.The reliability price settings – chiefly, the market price cap (MPC) and the cumulative price threshold (CPT) – needs to increase.3 Since low-SRMC VRE generators will increasingly be price-setters as VRE penetration increases, prices will be low for an increasing amount of time. In order for all generators to recover their long-run costs, prices would have to be higher during periods where VRE generators are not the price-setters. Non-VRE prices may even need to exceed the existing reliability price settings, thereby requiring these prices to be increased in order to avoid any ‘missing money’.4

  The NEM’s MPC has been estimated to be $60,000–80,000/MWh (Riesz et al., 2016) in order to achieve the reliability standard in a world with 100 per cent VRE penetration, a four- to five-fold increase compared to existing levels ($14,700/MWh for 2020). Were it infeasible to raise the MPC to such a level, a capacity market would be needed in order to meet the reliability standard, a well-established argument in the literature (Besser et al., 2002, Bublitz et al., 2019, Cramton et al., 2013, Hogan, 2005, Simshauser, 2018).

This article examines the validity of these two impacts, using actual spot price outcomes in SA. We find partial support for both hypothesised impacts. In terms of the first, while extremely low prices have become more frequent over time, extremely high prices have become less frequent. Moreover, spot price volatility has risen, consistent with the hypothesis, but not because prices have become more extreme. Instead, as we show, higher volatility has been due to increased instances of spot prices being in the $100–$500/MWh range, a range well below the historic or current MPC. This increased price volatility has also occurred in all NEM regions, not just SA, suggesting VRE penetration is not the sole (or major) contributor to this higher volatility.

In terms of the second hypothesised impact, reliability has remained high over the past decade despite the MPC remaining constant in real terms. The MPC has increased over time, but at a much slower rate than has VRE penetration in SA or the broader NEM. This finding is consistent with the hypothesised impact, but not resoundingly so.

We provide four reasons why higher VRE penetration need not result in more extreme prices and higher MPCs: (i) greater investment in volatility-dampening, reliability-enhancing technologies like storage and interconnectors; (ii) increased contract cover; (iii) more price-responsive demand; and (iv) the emergence of additional ancillary service revenue streams. These findings have implications for the durability of the NEM’s energy-only design in light of expected further increases in VRE penetration rates.

This said, we note upfront that our analysis is largely descriptive, with more sophisticated econometric analyses of the VRE penetration-price volatility link being undertaken in related research (Mwampashi et al., 2020). Furthermore, we readily acknowledge that there is a big difference between a 55 per cent VRE penetration rate in just one region and a 100 per cent VRE penetration rate across the NEM, the latter being the focus of much of the related literature (Bublitz et al., 2019, Riesz et al., 2016, Simshauser, 2018).5

Furthermore, our empirical analysis is based on the generation stock that existed in SA over the chosen sample period (2009–2019). As others have noted, SA thermal plant have typically been poor complements of high and rising VRE output, especially the inflexible brown coal plants in Playford and Northern power stations (Rai et al., 2019, Simshauser, 2018). While these two plants have exited the market, the remaining plants (Torrens Island A and B gas plants) are not much better suited. As such, historical pricing outcomes in the market reflect the sub-optimal nature of the existing generation stock. As that turns over, pricing and reliability outcomes might change, either for the worse or, if the market provides clear signals for ‘dispatchability’ (Rai and Nunn, 2020), for the better.

Furthermore, our discussion is confined to the NEM, and in particular South Australia. We have not undertaken a comparison between the NEM and other energy-only markets internationally (e.g. Texas) to see whether our analysis of VRE-induced impacts on spot prices and system reliability for SA is also observed in other energy-only markets. This analysis would be important and is left for future research.

Therefore, our findings are not, and should not be seen to be, the final word on the debate about whether an energy-only market, or a capacity market, is a better design for a scenario with a 100 per cent VRE penetration rate. As and when VRE penetration rates in the NEM rise, our findings will need to be revisited periodically to check if they remain valid and relevant.

The article is structured as follows. Section 2 presents data on historical VRE penetration and on historical spot price outcomes, in SA Section 3 discusses reliability outcomes in SA and other NEM regions. Section 4 discusses four reasons why extreme spot prices, and the MPC, need not dramatically increase under increasing VRE penetration: (i) the role of volatility-dampening technologies like storage and interconnectors; (ii) the role of hedging on generator bidding behaviour and its effect on spot prices; (iii) the impact of more price-responsive demand; and (iv) the emergence of additional ancillary service revenue streams. Section 5 concludes with a brief discussion of the policy implications of these trends.

2. The South Australian evidence

2.1. VRE penetration

Over the year to 30 June 2019, more than 50 per cent of SA electricity demand was supplied by large-scale VRE generation (Fig. 2). In contrast, 7 per cent of SA demand was met by large-scale VRE generation over the year to 30 June 2007. Virtually all (around 99 per cent) of the increase in large-scale VRE generation in SA during this period was from wind.

Fig. 2.

Penetration of large-scale VRE generation in SA, by quarter.

AEMO.

Going forward, large-scale solar PV is expected to contribute more to future increases in large-scale VRE generation penetration in SA. As at January 2020, 3271 MW of solar PV projects are ‘proposed’ in SA, compared to 3914 MW of wind (AEMO, 2020). It is worth noting not all of this capacity is likely to be installed given minimum and median demand in SA was only 415 MW and 1300 MW, respectively, in 2019. Concerns that COVID-19 could result in demand being lower for a sustained period of time, compared to pre-COVID levels, could also limit entry.

It is also worth noting Fig. 2 excludes small-scale VRE generation (principally, rooftop solar PV), and so underestimates the overall penetration of VRE in SA. Rooftop solar PV comprised around 10 per cent to electricity consumption in SA over 2018 (AEMO, 2018). Hence, utility- and small-scale VRE penetration in SA was in excess of 60 per cent over 2019.

Growing VRE penetration, combined with flat electricity demand growth, has made SA residual electricity demand more volatile, on both intra- and inter-day timescales.6 One way to observe changes in residual demand is to examine various points on the residual demand frequency distribution. Since 2010, the residual demand distribution in SA has (see Fig. 3):

Fig. 3.

Median, 5th and 95th percentiles of monthly SA residual demand.

AEMO.

• •shifted to the left i.e. residual demand has fallen
• •become wider – residual demand at the 5th percentile has fallen by 100 per cent – from 1000 MW to around zero – exceeding the 40 per cent decline in residual demand at the 95th percentile (from 2500 MW to 1000 MW). South Australian residual demand is increasingly zero and even negative, even at the 5th percentile. This means minimum residual demand is even lower, and
• •become peakier – residual demand at the 95th percentile has not fallen by as much as residual demand at the 50th or 5th percentiles.

These differential impacts of wind output on demand frequency distributions and load duration curves has also been observed internationally, from Tamil Nadu (George and Banerjee, 2009) to Iberia (Figueiredo and Da Silva, 2019) and California (Prol et al., 2020).

2.2. Historical pricing outcomes

This article uses the following two broad measures of price extremity/volatility:

• 1.The contribution of extreme spot prices (‘extreme’ defined here to be below $0/MWh or above $300/MWh) to the overall average spot price.
• 2.The number of periods in which spot prices were extremely high or low.

Both measures provide information about the price distribution. The first measure captures the extent to which more extreme prices impacts the overall average; broadly speaking, the average price reflects the price required by generators to cover their long-run average cost.7 A strike price of $300/MWh is the NEM-standard for bifurcating spot prices into ‘volatile’ and ‘non-volatile’ values, as it has historically reflected prices corresponding to the highest-SRMC generators: gas- or liquid fuel-powered (e.g. diesel) peaking plant (Riesz et al., 2016, Simshauser, 2018, Simshauser and Gilmore, 2020).

The second measure is a clearer indicator of price extremity. Since extreme spot prices need not result in a higher overall average, this second measure is a useful supplement to the first measure. Together, these two measures provide information on whether spot prices have become more extreme, and whether this results in higher consumer prices as proxied by average spot prices.

Average prices impact end-consumer electricity prices more than very low or very high prices. While more extreme prices can and do impact end-consumer prices, the effect on end-consumer prices is larger if more extreme spot prices also impact average spot prices.8

2.2.1. The contribution of extreme prices to the overall average

Over 2019, this measure of price extremity was lower than during both 2017 and 2008 (Fig. 4). More generally, an increasing penetration of VRE generation in SA has not been associated with increasing instances of $300+ prices. Over 2019, prices above $300/MWh contributed around 18 per cent to the overall average ($110/MWh), compared to a 41 per cent contribution over 2008 (average price of $74/MWh). In contrast, the penetration of (large-scale) VRE generation in SA in 2008 was 4–8 per cent, compared to 45–50 per cent during 2019 (Fig. 2).

Fig. 4.

Spot price volatility in SA.

AEMO.

Instead of peak prices driving up average prices, average spot prices have instead risen due to an increased incidence of prices in the $100–300/MWh range (Fig. 4). Over 2019, $100–300/MWh prices contributed almost half (47 per cent) to the overall average price, compared to a 7 per cent contribution over 2008.

That said, while the growing share of this price band is positively correlated with an increasing VRE penetration, there are likely to be other drivers behind the rising share of $100–$300 prices: chiefly, higher coal and gas prices. The role for higher fuel costs in driving a higher contribution of $100–$300 prices is seen by the increased contribution of this price band to average prices even in regions of the NEM, such as NSW and Queensland,9 where both the level and change in large-scale VRE penetration rates are significantly lower.

Furthermore, average prices have risen sharply. Between 2012 and 2019, spot prices almost quadrupled, from $30/MWh to $111/MWh. These increases have been attributed to the confluence of (Rai and Nelson, 2019, Simshauser, 2019):

• •Unexpected and sudden exit of large-scale thermal plant. Two coal-fired plant in SA with combined capacity of 786 MW exited in 2016: Playford B (240 MW10 ), and Northern (546 MW). This comprised more than half of SA average demand during 2016. Both plant exited with barely seven months’ notice (Simshauser, 2019). These exits reflected technical factors (i.e. end-of-life) and, to a larger extent, economic drivers, namely the generally low spot prices between 2010 and 2016 (see Fig. 5).

  The resulting upward pressure on SA spot prices was compounded by the unexpected and sudden exit of coal-fired plant in neighbouring Victoria.

• •
  Higher gas and coal prices. Between 2016 and 2018, gas prices doubled, driven by the combination of (Grafton et al., 2018, Simshauser and Nelson, 2015):

  • $\circ$
    expiry of relatively cheap long term gas contracts at a time when new production costs were rising rapidly
  • $\circ$
    linkage of the Eastern Australian gas market with international markets due to the advent of gas exports in 2015, driven by the commissioning and operation of three large LNG plants in Queensland from 2015
  • $\circ$
    supply restrictions on the development of both conventional and unconventional gas in some states since 2010
  • $\circ$
    slower than expected “ramp up” of gas export supply that required the purchase of gas that would otherwise have been available for domestic demand, and
  • $\circ$
    the exercise of market power by gas producers,

  This had a significant impact on SA spot prices as gas-fired plant were the only thermal plant remaining in the market in SA, following the exit of coal-fired plant during 2016. Higher coal prices also raised the price of electricity imports into SA from Victoria.

• •A lack of sufficient entry of new ‘dispatchable’ plant, to replace retiring plant, not just in the immediate aftermath of plant exits, but also over the medium term. Since the coal plant exits in SA, the only thermal capacity added to SA is a 210 MW gas plant (comprising 18 fast-start reciprocating engines) at Barker Inlet, in late 2019. This plant is an ideal complement to VRE output and to 5-min spot settlement (Rai et al., 2019), and it is surprising that more of these types of plant have not been added to SA, even taking into account high gas prices.11

Fig. 5.

Spot price volatility in the other mainland NEM regions.13

AEMO.

The increased instances of $100–300/MWh prices is a relatively recent phenomenon (since 2017). Given the above drivers of these higher prices, namely higher fuel costs, this phenomenon may last for as long as fuel costs remain elevated. Coal and gas prices have declined significantly since February 2020, reflecting both COVID-19 induced concerns about commodity demand and an oil price war between Saudi Arabia and Russia. It remains to be seen whether coal and gas prices regain their pre-COVID levels. If that were to occur, that is likely to result in $100–300/MWh prices maintaining its existing share of overall average prices.

The SA experience also provides only partial support to the argument that higher VRE penetration results in extremely low spot prices. For example, negative spot prices contributed 1 per cent to the overall average price over 2019, a similar proportion to 2008 (Fig. 4). This is despite the six-fold increase in VRE penetration rates between 2008 and 2019 (Fig. 2).12

Fig. 5 reveals a similar trend for the other three mainland NEM regions. Over 2019, $100–300/MWh prices contributed almost 38 per cent to the average in these three regions, compared to a 17 per cent share over 2016. Furthermore, the share of $300+ prices fell from 17 per cent to 6 per cent between 2015 and 2019. This echoes the SA experience with $300+/MWh prices.

There are two key differences between Fig. 4, Fig. 5:

• 1.$100–300 prices comprised a larger share of SA prices (48 per cent) than prices in the other three NEM mainland regions (38 per cent), over 2019.
• 2.Negative prices have comprised a larger share of SA prices than in the other NEM mainland regions.

2.2.2. The frequency of extreme prices

This measure of price extremity shows that $300/MWh+ prices have become more frequent over time, from a total of 642 5-min dispatch intervals during 2009, to 1378 during 2019 (see the first horizontal panel of Fig. 6). However, the incidence of these high prices has fallen since 2017, a year where high price periods were especially prominent following the closure of Hazelwood power station in Victoria.

Fig. 6.

The distribution of spot prices in South Australia. Note: The top panel shows the number of 5-min dispatch intervals that spot prices were in excess of $300/MWh; the bottom panel shows the number of intervals where spot prices were below $0/MWh.

AEMO.

Low prices (i.e. negative prices) have also become more frequent over time, with the number of such rising from 405 periods during 2009, to 1541 during 2019 (see the bottom horizontal panel in Fig. 6). However, this measure is volatile; the incidence of such low-price periods during 2018 (711 time periods) was less than half its 2019 value.

The middle section of Fig. 6 contains a histogram of 5-min spot prices grouped into $10/MWh bins, between $0/MWh and $300/MWh. The histogram reveals the SA price distribution has widened over the past decade: prices have become more volatile within the ‘extreme price’ boundaries (i.e. $0/MWh and $300/MWh).

While this frequency-based measure provides the strongest support for the hypothesis that rising VRE penetration is resulting in higher price volatility, this higher volatility is more due to greater price variation within the $0–300/MWh range, and less due to increased instances of extreme prices. This price range is well below both the historic and current MPC, both of which are in excess of $10,000/MWh (see Section 3.1).

However, as noted above, increased instances of ‘mid-tier’ prices is a finding that is not SA-specific, and has occurred in regions where VRE penetration rates are both lower than, and have risen at a slower rate than, in SA. This suggests that VRE penetration is not the sole (or perhaps even the major) contributor to this higher price volatility.

The preceding analysis is consistent with econometric-based studies of the VRE penetration-price volatility link. Such studies also provide only partial support for the hypothesised price impacts. While there is a lack of econometric analysis for the NEM, higher VRE penetration is found to raise weekly spot price volatility in Denmark and Germany, with differing impacts depending on the type of VRE generation (wind or solar PV). However, wind output decreases daily Danish volatility, and solar PV output decreases daily German price volatility (Rintamäki et al., 2017).

Collectively, the findings in this section provide partial support for the hypothesised impact of rising VRE penetration on price volatility. This said, as we noted upfront in Section 1, there is a big difference between a 55 per cent penetration rate just in one region and a 100 per cent VRE penetration rate across the NEM. Therefore, our findings are not, and should not be seen to be, the final word on the debate about whether an energy-only market or a capacity market is a more appropriate design when VRE penetration rates reach 100 per cent.

2.3. Recent events in Queensland

At 1.15pm on 21 July, 5-min spot prices across the NEM were zero simultaneously, due to the combination of modest demand (due to mild winter conditions across the NEM) and high VRE output, both small-scale (i.e. rooftop PV) and utility-scale. Furthermore, the proportion of time when SA spot prices were below zero increased to 6 per cent during July, from 2 per cent in June, due to the combination of strong winter winds raising VRE supply, and lower demand due to a relatively mild winter (AEMO, 2019a).

Furthermore, low spot prices are increasingly occurring in Queensland, especially during the middle of the day. For example, mid-day spot prices were negative over the week of 18–22 August 2019 (Fig. 7). This is due to the entry of utility-scale solar PV, combined with the inability or unwillingness of incumbent coal-fired plant to reduce their output when PV plant are generating. This has resulted in depressed spot prices during the day.

Fig. 7.

30-min spot prices in Queensland, 18–22 August 2019.

AEMO.

The recent Queensland, and broader NEM, experience suggests greater, VRE-induced, instances of more extreme prices might be emerging first as a seasonal phenomenon, rather than over a year. Focusing on annual-average prices may obscure the seasonal element to VRE-induced price extremes. Moreover, this seasonal phenomenon appears to be greater for solar PV than for wind, reflecting solar PV’s relatively higher correlation and co-incident output (Bell et al., 2017).

2.4. Changes in the peakiness of electricity demand

One possible reason for reduced instances of $300+ prices could be if demand had become less “peaky”. Less peaky demand can reduce periods of supply scarcity and in turn reduce periods of scarcity pricing. However, demand has become more peaky, especially in SA (Fig. 8).

Fig. 8.

Electricity demand in each of the NEM regions.

AEMO.

Between 2010 and 2019, the contribution of $300+ prices to the average price in SA declined sharply (see Fig. 4); in contrast, the peakiness of SA demand rose by one-fifth, from 2.1 to 2.5, over the same period (Fig. 8). A similar finding applies for the other NEM regions. Therefore, changes in the peakiness of demand does not explain the decreased contribution of $300+ prices to average spot prices, in South Australia or in other regions of the NEM.

3. Other potential impacts of rising VRE penetration

3.1. Impact of reliability outcomes in South Australia

A corollary of the hypothesised impacts of higher VRE penetration is that the MPC should increase as VRE penetration increases. The support for this hypothesis is also somewhat mixed. While the MPC has increased over time, it has increased at a slower rate than has VRE penetration. Between 2008 and 2019, South Australia’s VRE penetration rose six-fold (Fig. 2). Yet, over the same period, the MPC increased only 47 per cent (Fig. 9). Moreover, since 2013, the MPC has been increased by only the inflation rate (2.5 per cent p.a.).

Fig. 9.

MPC and observed USE.

AEMO.

This sluggish increase in the MPC suggests reliability outcomes in SA should have deteriorated based on the hypothesised impacts of VRE penetration on reliability. Yet the observed experience in SA, and in other regions of the NEM, is that reliability has remained high, with actual unserved energy (USE) typically below the reliability standard (Fig. 9).14

We acknowledge some of the low USE values reflect instances of the market operator (AEMO) intervening to avoid load shedding: AEMO intervened a total of four times in 2017 and 2018, to avoid USE events in SA, compared to zero previously.15 However, these interventions are not the principal reason for the observed low USE outcomes; as noted, there were no reliability-related interventions in SA between 2010 and 2017.

Moreover, the year where USE in SA exceeded the reliability standard (2009) was due to the effects of extremely hot weather in SA and neighbouring Victoria (VIC), which both increased demand and reduced availability from SA and VIC thermal plant. Exports from VIC to SA were further limited by heat-induced network outages in VIC. There were also short-notice reductions in the availability of Basslink (the VIC-Tasmania interconnector), which limited flows into VIC, further resulting in less VIC plant being available for export (AEMC, 2009).

That is, actual USE in SA exceeded the reliability standard in 2009, due to transitory factors (unexpectedly hot weather in January), rather than a consistent, structural issue in relation to a VRE-induced lack of missing money. Moreover, VRE penetration in SA over 2009 was only 10 per cent (Fig. 2).

Hence, analysing reliability outcomes in SA shows mixed evidence for the argument that higher VRE penetration rates is expected to result in higher MPC values. While the MPC has risen over time, the rate of increase has lagged that of VRE penetration, and this has occurred without exacerbating reliability outcomes in South Australia.

3.2. Impact on frequency control ancillary service markets

While the focus of this article is on the impact of increasing VRE penetration on measures of spot price volatility, increasing VRE penetration can also impact system frequency. Frequency is one of the indicators of the ease with which electricity demand and supply can be equilibrated; frequency that often deviates from its target (50 Hz in Australia) indicates supply and demand are becoming increasingly hard to equilibrate (Stoft, 2002). Therefore, to the extent that increased VRE penetration makes it harder to equilibrate demand and supply, this could be reflected in an increasing amount of frequency control ancillary services (FCAS) being procured.

There are eight FCAS markets in each NEM region (AEMO, 2015):

• •Two “regulation” FCAS markets: (i) a frequency raise service, and (ii) a frequency lower service. Regulation FCAS allows frequency to be monitored and controlled every four seconds via adjusting the output of regulation FCAS-enabled generators.
• •Six “contingency” FCAS markets, consisting of three raise and three lower services. Each raise and lower service is provided within a specific time window, with the fastest being up to six seconds (i.e. dubbed ‘Raise 6 s’, and ‘Lower 6 s’), and the slowest being between sixty seconds and five minutes (‘Raise 5 min’, ‘Lower 5 min’).

There has been an ongoing decline in power system frequency control over the past two years. For example, over January and February 2019, power system frequency remained outside the normal operating frequency band (NOFB) more than 1 per cent of the time in mainland regions of the NEM (AEMO, 2019b, Simshauser, 2019). This is inconsistent with the frequency operating standard, which requires system frequency to remain within the NOFB 99 per cent of the time.

This decline in frequency control was attributed to both increased VRE penetration, as well as a reduction in the primary frequency control capabilities of individual plant which has made the system less responsive to changes in frequency. To improve frequency control in the NEM, on 22 March 2019 AEMO increased the procurement of regulation FCAS across the mainland NEM regions by 50 MW to 180 MW (AEMO, 2019b).

4. Potential reasons why spot price volatility and reliability outcomes are little changed

While actual spot price and reliability outcomes in SA provide only partial support for the hypothesised effects of higher VRE penetration, this does not mean such arguments are invalid: higher price volatility (and need for a significantly higher MPC) could occur as VRE penetration continues to increase especially across the NEM. Moreover, econometric studies for the NEM (Mwampashi et al., 2020) and internationally provides some support for the higher VRE penetration-price volatility hypothesis.

Our analysis of spot price and reliability outcomes in Sections 2, 3 are based on the generation stock that existed in SA over that time period (2009–2019). While Playford and Northern power stations, both poorly-suited to complementing high and rising VRE penetration rates, have exited the market, the other remaining plant are not much better suited (Rai et al., 2019, Simshauser, 2018). As such, historical pricing outcomes in the market reflect the sub-optimal nature of the existing generation stock. As that turns over, pricing and reliability outcomes might change, either for the worse or, if the market is providing clear signals for ‘dispatchability’, for the better (Rai and Nunn, 2020).

What our findings do suggest is there are other intervening variables that have meant the predictions of Riesz et al. (2016) are yet to be fully borne out in practice. There are five potential reasons why higher VRE penetration in SA has not resulted in higher instances of extremely high spot prices or worsened reliability outcomes:

• 1.Increased interconnection between South Australia and Victoria
• 2.Increased storage, both utility- and small-scale
• 3.Role of contract cover to deal with higher risk of spot price volatility
• 4.Role of price-responsive demand, and
• 5.Potential role of additional ancillary service revenue streams going forward.

A sixth consideration is the design of emissions reduction mechanisms. This, and each of the five above-listed factors, is discussed below.

4.1. Increased interconnection between South Australia and Victoria

Riesz et al. (2016) are largely silent on the interaction between interconnector capacity and price volatility. Increased interconnection between regions with imperfectly correlated demand has the potential to reduce the volatility of spot prices in each region, as prices in each region become more equalised. This is because periods of excess supply or low demand in one region can be used to hedge against corresponding periods of tight supply or high demand in another region (due to imperfectly correlated demand in each of the two regions).16

Various researchers have documented this finding in Australia and internationally (e.g. Bell et al., 2015 for the NEM; Lund et al., 2015 and Obersteiner, 2019 for various European and North American markets).

Two indicators of imperfect correlation between South Australia and Victoria are:

• 1.the correlation between spot prices in each region. Over the past decade, the correlation between daily Victorian and SA spot prices was 0.51, and
• 2.the correlation between residual demand in each region. Over the past decade, the correlation between Victorian and South Australian residual demand was 0.74.

Interconnection between Victoria and SA has increased over time (Table 1). The Heywood interconnector was upgraded in July 2016 by 190 MW in both directions, though the transfer limits were subsequently revised downwards slightly in January 2017.17

Table 1.

Transfer limits on the Heywood (VIC–SA) interconnector.

Date	Victoria-to-SA max. transfer (MW)		SA-to-Victoria max. transfer (MW)
	Change to max. transfer limit	New max. transfer limit	Change to max. transfer limit	New max. transfer limit
Mar 2010	+160	460	+160	460
July 2016	+190	650	+190	650
Jan 2017	−50	600	−150	500

AEMO.

To examine the role of the Heywood interconnector on spot prices, Fig. 10 splits Fig. 4 into two distinct periods:

Fig. 10.

SA spot prices based on interconnector constraints.

AEMO.

• 1.periods when the VIC–SA interconnector was not constrained, and
• 2.periods when the interconnector was constrained.

This split reveals the following three findings:

• 1.Prior to 2017, average spot prices at times when the interconnector was not constrained were typically around half the level of prices when there were interconnector constraints. Prior to 2017, unconstrained interconnector flows meant a greater volume of cheap (brown coal) imports from Victoria, pushing down SA prices. In comparison, periods where the interconnector was constrained meant these cheaper imports were less available, in turn pushing up SA spot prices.

  However, from 2017 onwards, these findings reversed: average spot prices were typically higher when the interconnector was not constrained compared to when there were constraints. Furthermore, VIC–SA net flows reversed, with SA becoming a net exporter. The increased SA exports reflected relatively higher spot prices in Victoria, with higher Victorian prices due to higher coal prices and a tightening in its demand–supply balance following the closure of Hazelwood power station. This meant average SA prices in 2018 were higher when the interconnector was unconstrained, as the effect of the interconnector flow was to equate SA prices with the higher Victorian prices (Mountain and Percy, 2019). In contrast, average SA prices were lower when the interconnector was constrained, as this meant less exports and therefore a looser supply–demand balance in South Australia.

• 2.Price volatility is higher during periods when the interconnector is binding than when it is not. For example, $300+ spot prices comprised around 80 per cent of the average 2018 price when the interconnector was constrained (see top panel of Fig. 10); in comparison, $300+ spot prices comprised around $\frac{121}{2}$ per cent of the average 2018 price when the interconnector was not constrained (see bottom panel of Fig. 10).
• 3.In both panels of Fig. 10, spot price volatility had not increased as the penetration of VRE generation increased, consistent with the findings in Section 2.

4.2. Increased utility- and small-scale storage

Similar to the role of increased interconnection, increased storage can also dampen spot price volatility (Khan et al., 2018, Lund et al., 2015, McPherson and Tahseen, 2018). Lund et al. (2015) note increased VRE penetration can increase spot price volatility especially when VRE output is poorly correlated with demand. This increased volatility can in turn be reduced by increased storage and price-responsive demand.

Demand response (DR) and storage contribute to system adequacy and reliability in energy-only markets, with the need for a capacity market significantly lessened when DR and storage are available in the context of increasing rates of VRE penetration (Khan et al., 2018). In contrast, in the absence of DR and energy storage, there is a growing need for capacity markets in order to achieve system adequacy in the presence of high and increasing VRE penetration.18

Storage can do this by:

• •providing frequency response services, reducing both the level and volatility of FCAS prices. This is especially pertinent for short-duration (i.e. 1–2 h discharge capacity), fast-responding battery storage. In turn, this reduces energy price volatility given the link between energy prices and FCAS prices (i.e. the opportunity cost of providing FCAS is the spot value of energy); and/or
• •performing traditional price arbitrage actions, buying and storing electricity during periods of low spot prices, and discharging and selling this electricity during periods of high spot prices (in industry parlance, this price differential is dubbed the “park spread”). This can reduce the extremities of the spot price distribution, by both increasing spot prices during times of high VRE output and reducing prices during times of minimal VRE output, thereby lowering spot price volatility. This is pertinent for longer duration storage such as pumped hydro and 4-to-6-h duration batteries.

We acknowledge upfront that battery storage is currently an immaterial share of the NEM, in both a dollar and volume sense. For example, batteries have had the largest influence on the FCAS market, yet this market is just over 1 per cent of the energy market: $220 million vs. a $19 billion spot market, over 2019 (AER, 2019). As we argue below, we consider battery storage, along with pumped hydro, is likely to become an increasing source of overall energy turnover (i.e. spot market + FCAS) going forward, even though currently batteries are a small share of turnover.

In late 2017, the Hornsdale Power Reserve (HPR) HPR commenced operation in SA HPR is to date the world’s largest lithium-ion battery energy storage system, with a discharge capacity of 100 MW, 80 MW charge capacity, and energy storage capacity of 129 MWh.19 Prior to the entry of HPR, the SA market had virtually no storage, either small- or utility-scale.

HPR’s most significant market impact has been in the Regulation FCAS market, where it captured nearly 10% of raise FCAS volumes, displacing higher-priced gas plant. HPR’s presence led to Regulation FCAS costs more than halving between Q1 2017 and Q1 2018 (Fig. 11).20 While the FCAS market is a small share of the overall energy market – $220 million vs. a $19 billion energy market over 2019 – FCAS prices and spot prices move in lockstep due to the opportunity cost of providing FCAS vis-à-vis energy. Hence, HPR has played an important role in reducing FCAS prices and spot prices, especially spot prices in the $300+/MWh range.

Fig. 11.

Quarterly FCAS costs by type of FCAS.

AEMO.

Storage technologies recorded significant revenues between January 2019 and March 2020, with the bulk of batteries’ revenues coming from providing FCAS as opposed to energy (Fig. 12). The converse was true for pumped hydro, reflecting the technical characteristics of batteries vis-à-vis hydro (i.e. shorter-duration discharge capacity, with a faster response time). For example, FCAS provision constituted almost 98 per cent of batteries’ net revenues in the three months to end-March 2020 (i.e. Q1 2020), compared to just 6 per cent for pumped hydro.

Fig. 12.

Revenue sources by storage technology.

AEMO.

Looking ahead, there is a significant pipeline of utility-scale storage expected to enter the NEM, with 13 GW of pumped hydro and battery storage capacity committed or proposed across the NEM, supplementing the 8.2 GW of existing large-scale storage (AEMO, 2020). Even if only the committed part of this pipeline (2.1 GW, of which 2 GW is pumped hydro capacity in the form of “Snowy 2.0”) is realised, this is likely to reduce the incidence of extreme spot prices in future.

Furthermore, the projected declines in battery storage costs means these price spreads are likely to be smaller than for existing storage; for example, the LCOE of 4-h battery storage is projected to halve between 2018 and 2030 (BNEF, 2019). Projected declines in storage costs mean storage facilities are likely to require lower “park spreads” to break even, in turn potentially limiting spot price volatility and extremity going forward.

Similarly, penetration of residential- and commercial-scale storage is also expected to increase going forward, and expected to reach 5.6 GW by June 2037, up from a capacity close to zero today (Graham et al., 2019).21 This sizable storage capacity may also help dampen price volatility through energy arbitrage activities similar to that of larger-scale storage.

All this said, we note not all of this storage may enter the NEM, especially for battery storage within AEMO’s ‘proposed’ category (which comprises almost half of the 13 GW storage capacity committed and proposed). The business case for short-duration battery storage will depend on the potential FCAS revenues vis-à-vis capital costs. Infigen Energy has recently proposed a fast frequency response service for the NEM. If introduced, this would enhance the business case for short-duration, fast-responding storage. On the other hand, a primary frequency response rule made in March 2020, which increases the potential supply of frequency response providers to all scheduled and semi-scheduled plant, may detract from this business case (AEMC, 2020b).

The business case for longer-duration battery storage (i.e. four- to six-hour continuous discharge capacity) will depend on both expected “park spreads” (i.e. the value of traditional energy arbitrage activity) and the capital costs of this longer form of storage. As shown in Fig. 12, energy arbitrage remains the province of hydro plant, though battery storage may increasingly play this role in the NEM in future if its costs decline sufficiently.

4.3. The role of contract cover

The characterisation of the NEM as an “energy-only” market is somewhat misleading. In particular, a capacity market already exists in the NEM: the forward contracts market (Anderson et al., 2007, Simshauser, 2019). A cap is the classic example of a financial contract functionally similar to a capacity reserve mechanism (CRM) contract: the premium on the cap provides a generator a fixed revenue amount that covers their fixed costs, similar to CRM payments.22

If spot price volatility was to increase as the VRE penetration increases, this is likely to increase parties’ incentives to contract. This in turn is likely to have two impacts:

• 1.As contract prices would have a greater bearing on generators’ revenues, thereby reducing the extent of any increase in the MPC/CPT to resolve any ‘missing money’ problems (Anderson et al., 2007, Riesz et al., 2016, Simshauser, 2018).
• 2.Financial contracts can itself reduce spot price volatility. The level of contract cover by generators influences their spot market bids: generators bid more of their capacity at or below their SRMC when that capacity is contracted. This results in lower spot price volatility compared to prices under strategic bidding and exercise of spot market power (Anderson et al., 2007, Wolak, 2000).

Quiescent spot prices in SA during 2018 partly reflected more competitive spot market bidding by generators, which market liaison has attributed to increased contract cover. Data on the extent to which retailers and generators are contracted are not publicly available; gathering data on generators’ level of contracting is a challenge outside the scope of this paper, with survey-based methods having been used for generators in the NEM (Anderson et al., 2007).

Market liaison and anecdotal evidence provided to the authors is that contract cover in South Australia and other parts of the NEM increased between 2017 and 2018, from around 80 per cent to over 90 per cent. The reasons attributed to this increase included:

• •the reaction by market participants to the high-spot-price events during summer 2017, with such events partly caused by the closure of Hazelwood power station in March 2017. Some market customers were overly exposed to these high prices, and increased their contract cover in response, and
• •the increased attention on contracting in the design of the National Energy Guarantee and the retailer reliability obligation (RRO) (COAG Energy Council, 2019). In particular, the amount of contract cover required of each liable entity under the RRO – namely, the entity’s share of one-in-two-year expected peak demand – is higher than the extent of contract cover held by some participants.

4.4. Role of price-responsive demand

It is well-recognised that issues related to resource adequacy could be eliminated by sufficiently increasing demand elasticity (Cramton et al., 2013, Cramton and Stoft, 2006, Joskow, 2008, Joskow, 2006, Simshauser, 2019, Stoft, 2002). Another well-recognised issue is the amount of DR provided in electricity markets around the world has been sub-optimally low (Batlle and Pérez-Arriaga, 2008, Cramton and Stoft, 2008, Cramton and Stoft, 2006, Hogan, 2005, Stoft, 2002).

If there were comprehensive DR, with a sufficiently high MPC, it would not be necessary to determine an aggregate reliability standard. Instead, each customer could elect to remove load from the system in response to price, reflecting their individual value of reliability (or indeed, values of reliability for their different loads). As individual customers engage more of their load in DR, they can choose the desired reliability level (and characteristics) that suit their preferences. The aggregate reliability standard implied by the MPC can then gradually apply to a diminishing proportion of the system (Cramton and Stoft, 2008, Cramton and Stoft, 2006, Golden et al., 2019, Hogan, 2005, Riesz et al., 2016, Simshauser, 2018).

More DR is seen as especially important to maintaining reliability under high VRE penetration rates (Golden et al., 2019, Khan et al., 2018, Lund et al., 2015). Furthermore, higher DR can also reduce spot price volatility, especially in a high VRE penetration world. More DR can lower the instances of spot prices hitting a particular MPC and CPT. It also means the MPC/CPT can be set at lower levels than would occur under less DR, for a given level of reliability (Khan et al., 2018).

Available estimates of the amount of wholesale DR – that is, DR available in the spot market – suggest around 400 MW is available across the NEM, with NSW and Victoria comprising around 70 per cent of this (AEMC, 2020a).23 The amount of DR available in the spot market is likely to increase going forward due to a decision by the Australian Energy Market Commission (AEMC) to introduce a wholesale DR mechanism (AEMC, 2020a). This would enable parties other than a customer’s retailer to provide DR.

Over the medium-to-longer term, moves to establish a two-sided market could unlock further DR, both for the NEM (AEMC, 2019) and overseas (Rochet and Tirole, 2006).

4.5. Potential role of additional ancillary service revenue streams

The increasing penetration of VRE generation reflects a combination of greater entry of asynchronous, inverter-connected generators, coupled with the exit of synchronous, inertial-responsive, generators. The decreasing penetration of synchronous plant is leading to a consideration of the range of ancillary services that hitherto were a free by-product of synchronous generators’ output, with these services increasingly valuable as asynchronous (VRE) penetration rises. There is therefore a need to value these “missing” ancillary services, which include services such as inertia, fault current, and reactive power (Billimoria and Poudineh, 2018, Gu et al., 2019, Pollitt and Anaya, 2019, Simshauser, 2019, Simshauser, 2018). Identification of the types of ancillary services that may be needed going forward is an ongoing and growing area of work.

This need to value additional ancillary services is likely to impact both price volatility and the MPC and CPT. In terms of the MPC/CPT, new ancillary service revenue streams means the MPC and CPT can be lower than in the absence of these revenue streams. This is because plant capable of providing these ancillary services, which includes both renewables and non-renewables, can recover some or all of their capital costs via revenues earned from providing these services. Additional ancillary service revenues could make these generators less reliant on “black” electricity prices, with these “missing value streams” substituting for any potential ‘missing money’ issues associated with providing “electricity only”.

This said, additional ancillary service revenues may increase the propensity for very low spot prices, thereby increasing price volatility ceteris paribus, precisely because generators would be less reliant on electricity spot prices to recoup some or all of their capital costs.

Similar to our discussion of battery storage (Section 4.2), we acknowledge new ancillary service revenue streams are a more modest share of overall energy market revenues. As we discuss below, the value of new ancillary services in SA like “system strength” is c.$30–50 million p.a. This is barely 2 per cent of the $1.7 billion SA spot market (AER, 2019). This said, we consider the value of system strength, and potentially other new ancillary services, to be a rising share of overall spot market revenues going forward, given projected increases in VRE penetration rates.

Over 2018 and 2019, AEMO intervened around 20 per cent of the time in SA, largely to direct synchronous generators on (or to prevent these generators from taking units offline) so as to maintain sufficient fault current (or “system strength”) (Fig. 13).24 AEMO did this during periods of high wind generation and co-incident low demand; such periods are typically characterised by low spot prices, which in turn leads to decommitment by synchronous units and raises concerns about inadequate system strength. In contrast, AEMO intervened barely 0.5 per cent of the time during 2017.

Fig. 13.

SA demand during intervention and non-intervention periods.

AEMO.

An indication of the magnitude of the “missing ancillary services” value stream can be obtained by considering the differences in two spot prices:

• 1.The spot prices that arose when AEMO intervened in South Australia to maintain system strength (i.e. to maintain appropriate levels of fault current). This is dubbed “intervention prices” as the prices that result from the intervention are higher than would be the case in the absence of the intervention (i.e. prices under less-constrained dispatch).25
• 2.The spot prices that would have arisen if AEMO had not intervened (i.e. prices if AEMO had dispatched generators according to merit order).

The average difference between these two prices has increased over time, from $1.8/MWh over 2017, to $5.4/MWh over 2019 (Table 2). This price differential translated to around $29 million in extra pool payments in South Australia over 2019, and an estimated total of $88 million in extra pool payments between 2017 and 2019.

Table 2.

SA spot prices, intervention and non-intervention prices^a.

Financial year	Average prices with system security interventions included ($/MWh)	Average prices ignoring system security interventions ($/MWh)	Price difference ($/MWh)	Value excluding reliability-related interventions ($m)
2017	103.1	101.3	1.8	9.3
2018	94.6	86.6	8.0	49.8
2019	102.8	97.4	5.4	28.8
Whole period	103.7	98.6	5.1	87.8

^aExcluding costs associated with reliability-related interventions (i.e. reliability directions plus the RERT).

AEMO.

In addition to these costs of system security directions, there are costs associated with payments to those scheduled generators whose dispatch targets are affected as a consequence of the direction.26 There is limited public information about the size of these costs, and we have not attempted to include estimates of these costs in Table 2.

Technologies that can provide this service include synchronous condensers, synchronous generators, and “grid-forming inverters”, and this service may be offered by generators or other market participants including network businesses (Gu et al., 2019, Pollitt and Anaya, 2019).

4.6. The extent to which emissions reduction mechanisms are technology-neutral

A final consideration relates to the design of emissions reduction mechanisms. The entry of VRE generators into SA was driven by the large-scale Renewable Energy Target (LRET27 ), a renewable portfolio standard which provided a “green” revenue stream in the form of large-scale renewable generation certificates (LGCs) bought by energy retailers VRE plant have proved to be the cheapest way to achieve the LRET, with cost declines vastly exceeding expectations (Nelson et al., 2020, Nelson et al., 2015, Rai and Nelson, 2019, Simshauser, 2019).28

The LRET created signals for VRE plant entry even when cheaper forms of emissions abatement, such as coal-to-gas switching, may have been possible. This was especially the case between the late 2000s and mid-2010s, when South Australia’s VRE penetration tripled from 10 to 30 per cent despite wholesale electricity prices in SA being at consistently low levels. For example, average annual dispatch-weighted prices received by VRE generators between 2009 and 2015 ranged from $25/MWh to $55/MWh (Rai and Nunn, 2020). These prices were well below the long-run marginal costs for new-entrant wind generators, which at that time were in the $90–120/MWh range (International Energy Agency (IEA), 2019).29 In this regard, the LRET enabled recovery of VRE plant fixed costs, thereby resolving ‘missing money’ issues for these plant.

Production subsidies like the LRET subsidise specific forms of generation instead of directly pricing the externality (Nelson et al., 2020, Nelson et al., 2019, Rai and Nelson, 2019, Simshauser, 2019, Simshauser, 2018). A technology-neutral mechanism, such as an emissions intensity scheme (EIS), may have resulted in a relatively lower VRE penetration rate as some of the required emissions abatement could have come from existing thermal generators. Coal-to-gas switching may also have been a cheaper form of abatement than installing new VRE capacity, especially over the decade to the early 2010s when gas prices were relatively low.30

As a 100 per cent VRE penetration scenario is more likely to occur under a technology-specific mechanism as opposed to a technology-neutral mechanism, it is therefore possible that a technology-neutral mechanism could result in relatively lower spot price volatility and extremity.

5. Concluding remarks

This article has analysed spot price outcomes in SA, the NEM region with the highest VRE penetration, to evaluate hypotheses that an increasing VRE penetration increases both spot price extremes and requires significant increases in the MPC. The second purpose of this article is to outline some of the conditions that might be needed in order for higher price volatility to occur and in order for the MPC to need to be significantly increased.

While actual spot price and reliability outcomes in SA provide partial support to these two hypotheses, it does not mean the arguments are invalid: more extreme spot prices could increase in future under high and rising VRE penetration. For example, if the correlation between VRE resources across the NEM increases, due to a lack of sufficient geographic and technological diversification and/or increased co-incident weather patterns, then the potential for greater interconnection to dampen volatility may diminish as demand and supply becomes more correlated across regions. Furthermore, there is a big difference between a 55 per cent VRE penetration rate in just one region (SA), and a 100 per cent VRE penetration rate across the NEM.

Our analysis also suggests greater, VRE-induced, instances of more extreme prices might be emerging first as a seasonal phenomenon, rather than over a year. Focusing on annual-average prices may obscure the seasonal element to VRE-induced price extremes. Whether this phenomenon persists, both across seasons in a year and across years, is dependent on whether or not the mitigating factors discussed in Section 4 abate. In turn, these four potential mitigating factors will influence the NEM’s ability to retain its existing decentralised energy-only design under higher VRE penetration rates, such that price volatility is minimised whilst the reliability standard continues to be met. This is consistent with findings of other research on the sustainability of energy-only markets under high and increasing VRE penetration.

The existing literature focuses largely on the short-term impact of VRE penetration on spot prices, the so-called ‘merit-order’ effect (Csereklyei et al., 2019, Cutler et al., 2011, Forrest and MacGill, 2013). Future research should econometrically assess the impact of VRE penetration on price volatility that we have documented descriptively. We are aware of only one econometric study in this area (Mwampashi et al., 2020), with more needed going forward. Furthermore, our discussion is confined to the NEM, and in particular SA. We have not undertaken a comparison between the NEM and other energy-only markets internationally to see if our findings also apply in other energy-only markets. Such an analysis is important and, to the best of our knowledge, there are few studies on this (Ela et al., 2019 is one example). This is left for future research.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.