Predicting the geoeffective properties of coronal mass ejections: current status, open issues and path forward

Abstract

Much progress has been made in the study of coronal mass ejections (CMEs), the main drivers of terrestrial space weather thanks to the deployment of several missions in the last decade. The flow of energy required to power solar eruptions is beginning to be understood. The initiation of CMEs is routinely observed with cadences of tens of seconds with arc-second resolution. Their inner heliospheric evolution can now be imaged and followed routinely. Yet relatively little progress has been made in predicting the geoeffectiveness of a particular CME. Why is that? What are the issues holding back progress in medium-term forecasting of space weather? To answer these questions, we review, here, the measurements, status and open issues on the main CME geoeffective parameters; namely, their entrained magnetic field strength and configuration, their Earth arrival time and speed, and their mass (momentum). We offer strategies for improving the accuracy of the measurements and their forecasting in the near and mid-term future. To spark further discussion, we incorporate our suggestions into a top-level draft action plan that includes suggestions for sensor deployment, technology development and modelling/theory improvements.

This article is part of the theme issue ‘Solar eruptions and their space weather impact’.

1. Introduction

Coronal mass ejections (CMEs) constitute the main sources of variability in the space environment around Earth (and other planets). They are frequently associated, particularly the most energetic events, with flares, filament eruptions, and drive shocks that produce solar energetic particles (SEPs.) The resulting disturbances in geospace, collectively called space weather, affect space systems with consequently broad societal impacts in telecommunications, geolocation and aviation, to name a few. Accurate forecasting is necessary to protect those space assets and minimize the cost impact of space weather. This, in turn, requires reliable knowledge of a CME's arrival and physical properties in the Earth's vicinity. Space missions and much research have focused on increasing this knowledge and on improving space weather (SWx) forecasting.

This paper is a review of the current status of predicting the space weather impact of CMEs and draws on our efforts within the Hellenic National Space Weather Research Network (HNSWR)¹ co-funded by the European Union and Greece [1]. To keep the paper focused and within a reasonable length, we discuss only certain CME impact parameters; namely, their time-of-arrival (ToA), speed-on-arrival (SoA), momentum, length of interaction with the magnetosphere (size), and their magnetic configuration expressed by the southward component, B_z, of the CME-entrained magnetic field. These are the most relevant CME parameters for space weather, particularly the B_z component (magnitude and sign as a function of time). In addition, we restrict ourselves to issues of short- to medium-term forecasting (up to 7 days) and hence do not discuss issues relating to CME occurrence and climatology. Finally, we assume that the reader is familiar with the basic properties of CMEs. There exist several reviews on CMEs [2] and their properties [e.g. [3]].

In addition, we cover pertinent results only from the last 15 years, which witnessed a fundamental shift in solar and heliospheric observational capabilities, in regards to SWx research. The shift is the availability of multi-viewpoint imaging of the solar corona and inner heliosphere thanks to the deployment of the Solar Terrestrial Relations Observatory (STEREO) mission [4] in 2007. STEREO comprises two spacecraft with nearly identical payloads on gradually separating orbits ahead and behind Earth. Each spacecraft carries the Sun–Earth Connections Coronal and Heliospheric Investigation (SECCHI) [5] imaging payload. SECCHI consists of an EUV full-disk imager, two coronagraphs, and two Heliospheric Imagers (HI) that collectively image the inner heliosphere along the Sun–Earth line, from the solar surface to 1 AU and beyond. STEREO and the SECCHI observations enabled the Heliophysics community to address two long-standing observational ‘bottlenecks’: (i) assess the projection effects on CME kinematics and, (ii) investigate the evolution of CMEs in the largely unexplored inner heliosphere.

The paper is structured as follows. In the next five sections, we discuss the current status of the field for each of the five CME parameters, any issues that affect the prediction accuracy for the given parameter, and suggest a path forward. In the last section, we provide a synopsis of our findings and recommendations.

2. Forecasting coronal mass ejection time of arrivals and speeds at 1 AU

To first order, the average SWx end-user is primarily interested in two straightforward questions: will a CME, detected in the corona, hit the Earth? and when will the impact occur? The corresponding forecasting quantities—‘hit/miss’ and the arrival time of the CME (ToA)—have been the subjects of several studies which we proceed to review, starting with the ToA (table 1).

Table 1.

Time-of-arrival predictions from various models.

reference	accuracy (h)	MAEa (h)	SoAa (km s−1)	sample size	comments
empirical
[6]*	10.7	15.1 ± 12.8b	—	47	ESAc
[7]	—	11.5 ± 9.0b	—	75	ESMc
[8]*	—	16.9 ± 11.3	—	31	GCS + ESA
[9]*	—	8.4 ± 5.8	—	19	ESA
[10]*	±6	4.7 ± 4.8b		9	multiple models
[11]*	−3.8 ± 9.6	8.1 ± 6.3	284 ± 288	24	multiple models
[12]	+6.4 ± 5.3	6.8 ± 4.7b	16 ± 53	21	ElEvoHI + DBM
[13]*	—	6.3 ± 8.1	—	31	3D reconstruction
[14]	+3.2 ± 16.3	13.9 ± 11.9b	191 ± 341	171	SSEF30; const. speed
[15]	−0.3 ± 1.5	17.7 ± 0.85	—	214	EAMc
drag-based
[16]*	−0.3 ± 16.9	13.9 ± 9.7b	—	34	WECd
	+1.1 ± 19.1	14.6 ± 10.4b	—	34	DBMd
[17]*	−6.1	12.8 ± 6.8b	—	16	WEC; uses [18] sample
	−9.7	14.3 ± 9.0b	—	16	DBM
[19]*	−7.1	12.9	—	21	GCS + DBM
[20]*	−2.1 ± 3.9	3.5 ± 1.6	38.5 ± 17.9	7	GCS + DBM + Correction
[21]	+0 ± 10.6	9.1 ± 5.3b	—	14	DBM; uses [19] sample
physics-based shock
[22]	+2.2 ± 3.8	3.5 ± 2.4	—	8	Type-II; blast-wave model
[23]	—	6.7 ± 20	95 ± 249	40	piston shock
MHD
[24]*	+0.6 ± 10.2	7.5 ± 10		25	CAT + WEC
[25]	−3.7 ± 17.1	14.1 ± 18.4b	—	518	multiple MHD models
[26]*	−4	10.4 ± 0.9	—	273	StereoCAT + WEC
machine learning
[27]	—	≈ 12	—	153	neural network
[28]	—	5.9 ± 4.3	—	37	support vector machine
Alle	—	9.8 ± 2.0	—		unweighted average

^aMAE = mean absolute error; SoA = speed on arrival.

^bValues calculated by us.

^cESA = empirical shock arrival; ESM = expansion speed model; EAM = effective acceleration model.

^dWEC = WSA + ENLIL + Cone; DBM = drag-based model.

^eExcept [19,27].

*Deprojected CME parameters.

The field of ToA prediction has a long history. A variety of methods and models have been proposed in numerous case and large sample studies (empirical, physics-based, time-dependent MHD). Fortunately, Zhao & Dryer [29] have provided a recent in-depth review of the field focusing primarily on the performance of ToA predictions from single viewpoint observations. While [29] contains a thorough collection of the early work from the STEREO, that work was limited to single event studies at the time of writing their review.

Therefore, we structure this section as a complementary review that focuses on the developments for the last decade or so and attempts to contrast and summarize the ToA performance between the pre-STEREO and STEREO era. We avoid discussing details of the methods or of past literature except where necessary for clarity. We direct the interested reader to [29] for in-depth discussions of the prediction methods and the historical literature.

(a). Empirical models

The difference between the CME speed in the coronagraph FOVs and the speed measured in situ at Earth, along with the CME transit time, provide the average deceleration or acceleration, whether we are dealing with a fast or a slow CME, of the event between the Sun and Earth. With a large sample of events, the data can be fit to provide the ‘effective’ acceleration versus the coronagraph speed. This was the basis for the pre-STEREO-era empirical models, such as the empirical shock arrival (ESA) [6,30]. That work attributed much of the prediction errors to projection effects (since Earth-directed CMEs are strongly projected in the images) and the unknown CME evolution in the inner heliosphere (due to the complete lack of measurements between approx. 30 R_s and 1 AU). The expansion speed model (ESM) [7] was an effort to address this from a single viewpoint using the CME expansion speed to estimate the radial speed (figure 1a).

Figure 1.

(a) CME travel time to Earth against the halo expansion speed V_exp for 80 unique CME-shock correlations. The travel time T_tr is defined by the CME's first appearance in the LASCO/C2 images and the shock arrival at 1 AU. The solid line is a least-squares fit to the 80 data points, the fit function being T_tr = 203 − 20.77 × lnV _exp. The standard deviation from the fit curve is 14 h. The two dotted lines denote a 95% certainty margin of two standard deviations. The thin dashed line marks the calculated travel time for a constant radial propagation speed, V_rad, (kinematic approach) inferred from the observed V_exp, via V _rad = 0.88 × V _exp. The green dots denote CMEs without shock signatures, that is magnetic clouds (M) and plasma blobs (B). These points were not used for the fit. Reproduced with permission from Schwenn et al. [7]. (b) CME interplanetary propagation speeds, V_IPo, towards the in situ observatory, versus CME transit times, TT. The V_IPo from three geometrical models (fixed-Φ (red), harmonic mean (blue) and self-similar expansion (green)) is plotted. The details of power-law fits (solid lines) to each model are shown. The black dashed line is the mean transit time for the employed CME sample, 〈TT〉 = 71.9 h. Reproduced with permission from Möstl et al. [11]. Copyright © AAS. (Online version in colour.)

The two models can be combined of course. The quadrature configuration between STEREO and the Solar and Heliospheric Observatory (SoHO) in 2011 provided a unique opportunity to validate the V_rad − V_exp relation and test the ESA performance with the better-determined V_rad [9]. Using a 19-event sample between 2010 and 2012, the best results (mean absolute error; MAE = 8.4 ± 5.8 h) were obtained for widths determined from the SECCHI coronagraphs. The ‘traditional’ LASCO-based estimates result in MAE of 14 ± 10.4 h. The full ‘Ice-Cream’ Cone assumption was used in both cases (see also table 3 in [9]). The authors conclude that this simple prediction scheme performs as well as the other more complex schemes we discuss later.

A recent variation to the ESA model is the effective acceleration model (EAM) tested against a sample of 214 CMEs observed by LASCO [15]. The authors corrected the projected LASCO speeds assuming radial propagation from the flare site (for the 87 events where the flare site was known) and applying to the full sample. The final MAE (17.7 ± 0.85 h) is relatively high but it may reflect the large event sample.

In the STEREO era, there have been a handful of multi-CME ToA analyses with empirical methods [10–15]. Colaninno et al. [10] reconstructed the three-dimensional (3D) trajectories of nine Earth-directed events in 2010–2011, and used a variety of simple analytical models to calculate their ToA and compare it with the in situ measurements. Their study investigated the performance of linear, quadratic and non-uniform kinematic profiles and assessed the effects of the CME front geometry. They concluded that the best model—a linear fit above 50 R_s—gives the ToA within ± 6 h for 78% of the events and within ± 13 h for the full sample. They concluded that off Sun–Earth line measurements provide a demonstrable improvement in ToA predictions and that the kinematic evolution of the CME, close to the Sun, is not a reliable indicator for its heliospheric profile. These conclusions, however, were based on a rather small sample.

Extending [10] to a larger event sample, [13] performed detailed 3D reconstructions of 31 Earth-directed CMEs throughout the full SECCHI FOV and examined the ToA accuracy against different extrapolation distances (or equivalently, lead forecast times) using an empirical kinematic profile. Rather unsurprisingly, they found that the shorter lead time led to improved ToA up to a distance of 0.3 AU from the Sun. They attributed the (counterintuitive) increase in the ToA error for distances closer to Earth to increased reconstruction uncertainties due to the diminishing brightness of the CMEs at larger solar distances. They also examined the effect of corrections for the Earth-component of the CME speed but these were insignificant.

Adopting a different approach, [11] studied 22 events imaged by SECCHI in 2008–2011 and detected at any of the orbiting in situ payloads at 1 AU. Instead of a 3D reconstruction, they relied on the trajectories recorded by a single spacecraft, which was not along the event's path, and used a variety of analytical models for the shape of the CME front to estimate the 3D trajectory. Their ToA predictions were accurate to 8.1 ± 6.3 h and could be improved to 6.1 ± 5 h, after empirical corrections. Their resulting transit time versus CME speed curve (figure 1b) can be compared directly with the pre-STEREO results and shows the improvement in ToA afforded by the availability of SECCHI observations.

Rollett et al. [12] reanalysed a subset of the [11] CME list using an elliptical model for the CME front, the ellipse evolution model [31]. They introduced a correction (called the elliptical conversion method) to allow event tracing to the larger elongation captured by the SECCHI HI. Their method also included a drag-based model. Their prediction system performed similarly to the commonly used functions, such as fixed-P (FPF), harmonic-mean (HMF) and self-similar expansion (SSEF), at 6 ± 5.3 h (table 1) but had the tendency for later ToA. The predicted speeds-at-arrival, on the other hand, were much better than those predicted from the other models (table 1).

Finally, [14] reports the largest empirically based ToA study. It involves 1137 CMEs detected by the SECCHI HI between 2007–2014 and catalogued by the HELiospheric Cataloguing, Analyses and Techniques Service (HELCATS) project. Of the 1137 CMEs, 641 where detected in situ in the inner heliosphere (between 0.31 and 1.09 AU). The events were modelled as 60°-wide self-similarly expanding bubbles (SSEF model) propagating at constant speed and direction. Under these assumptions, the predicted ToA were accurate to 2.6 ± 16.6 h. In table 1, we report the results for STEREO-A only, which are indicative of the whole sample.

(b). Drag-based models

In addition to [12], there have been several interesting developments in drag-based models (DBM) [see §2.3 in [29]] in the last 4 years [16,17,19–21,32]. Sachdeva et al. [32] applied the viscous drag model [33], a model that accounts for the small-scale physics of the CME-solar wind interaction, against a small CME sample (eight events). The main purpose of the study was to investigate the height where drag forces begin to dominate (15–50 R_s) and to validate the model against the remote sensing observations (the model showed less than 2 h errors). No ToAs to 1 AU were reported.

Most of the DBM studies use coronagraph observations from a single viewpoint (usually STEREO-A or SOHO) and rely on a geometrical assumption about the shape of the CME (i.e. [11] or [12]) to estimate the 3D location and direction of the event, possibly for expedience reasons. Instead, [19] use all three available viewpoints (STEREO and SOHO) to reconstruct 21 CMEs with the Graduated Cylindrical Shell (GCS) model [34], which generally provides a better description of the 3D CME configuration but requires higher time investment than the simpler geometrical approaches. The authors test the CME inputs against three drag models (viscous, aerodynamic and hybrid). They find similar results from all three, with the hybrid drag having a slight edge (table 3 in [19]). We quote the results for the hybrid DBM in table 1. When they consider a subsample of events without low corona interactions (16 out 21 events), the results improve significantly—e.g. the MAE reduces from 12.9 to 6.8 h for the hybrid DBM.

Hess & Zhang [20] take an additional step and add a correction for the propagation direction of the CME based on the curvature of the reconstructed CME front (similar to [10,13]). They confirm the [10] finding that this correction overestimates the curvature at 1 AU and turn to a weighted average between the corrected and uncorrected height for their final analysis. However, this weighted average is based on knowledge of the actual ToAs and hence has little operational value. In the end, the addition of the correction performs better (MAE = 3.5 h) than the standard (aerodynamic) DBM and (MAE = 8.5 h) and much better than the empirical model [6] (MAE = 13.3 h). Of note, [20] assess ToA and SoA for both the shock/sheath and the actual CME. Although rare in the literature, this distinction is useful information to forecasters and should be reported more frequently.

The DBM requires five input parameters: the CME height, angular width and speed when the drag force becomes dominant (usually in the outer corona), the background solar wind speed and the drag parameter. The first three are observation-based and the other two use a priori values. Napoletano et al. [21] suggest a probabilistic framework to obtain these parameters. In this novel approach, the observational parameters are treated as Gaussian probability distribution functions (PDFs), while the solar wind speed and drag parameters are drawn from a priori PDFs. Then the ToA and SoA are obtained via Monte Carlo simulations. The results from this first study are promising. The MAE is 9.1 h while the average ToA error is a Gaussian of 10.6 h width (table 1).

Finally, one of the most interesting results of the recent DBM studies is the finding [16,17] that the DBM performs equally well as MHD models (table 1), at a much lower user cost (since the DBM is an analytical model versus the numerical MHD codes).

(c). Shock-based models

This class of models, reviewed in [29] (§§2d), introduces physics for the propagating CME—the expanding shock (driven or not) is the most common physical subject—that can be computed analytically. Many of them use historical events to constrain some of the input parameters and thus make these models applicable to operational use. For example, [22] used Type-II radio bursts to constrain their piston-shock model with promising results (MAE =3.5 ± 2.4 h) for a rather small shock sample of eight events (table 1). More recently, [23] extended their piston-shock model with MHD shock relations and validated it against 40 fast CMEs over two cycles (1995–2015). To increase the operational utility of their method, [23] formulated an analytical prescription for the piston-shock model. First, they derived a semi-empirical relation for its parameters using a subset of their events (the 20 fastest CMEs). Then they initiated the model with the height of the first CME appearance in the LASCO FOV, and the speed at that height, and used the rise time of the associated flare as a proxy for the acceleration duration. The solar wind speed, 6–12 h prior to the shock arrival, is the remaining input. Then, they checked the predictions against the full sample resulting in MAE of 6.7 ± 20 h and SoA of 95 ± 249 km s⁻¹ (table 1), which are on par with the performance of more sophisticated models. The paper has an extensive discussion of the errors and uncertainty calculations.

(d). Magnetohydrodynamic models

However, empirical methods lack physical sophistication. They cannot, for example, account for the state of the background heliosphere. This is the purvue of magnetohydrodynamic (MHD) models, particularly time-dependent MHD simulations [see §2.5 in [29], for details]. The majority of the models use as input the CME speed, size and direction at around 20 R_s and propagate the event to 1 AU through a predetermined inner heliospheric state. The latter is usually modelled with boundary conditions provided by the solar photospheric magnetic field measured over a solar rotation (approx. 27 days) and relies on empirical profiles for the coronal density and temperature or empirical coronal heating models [e.g. [35], and references therein]. Since this is the approach adopted by the agencies responsible for SWx predictions (e.g. NOAA, UKMetOffice), we focus our review on these models. In particular, NOAA and UKMet have adopted the WSA-ENLIL-Cone (WEC) model [36,37] as their operational model.

Solar wind models suffer from considerable uncertainty, especially during periods of high solar activity, due to the lack of simultaneous 360° photospheric field measurements and our limited physical understanding of the solar wind acceleration mechanism. This uncertainty hinders the accuracy of ToA predictions. Ensemble modelling is a way to address some of these uncertainties. [18] performed the first large sample ensemble modelling study consisting of 35 CMEs during the solar maximum of Cycle 25 (2013–July 2014) using the WSA-Enlil-Cone (WEC) model. They found a mean ToA of −7 h that indicates a tendency to predict early arrival. Recently, the real-time WEC simulations were validated against 1800 events during 2010–2016 [26]. The study involved CMEs arriving in any of the STEREO or Earth-orbiting spacecraft and studied the effects of including measurements from one, two or three viewpoints. The ToA accuracy for the sample was 10.4 ± 0.9 h again with a tendency towards early arrival prediction. The authors compared the performance against the number of viewpoints and found that the ToA uncertainty increased by 1.7 h when one or two viewpoints were available, suggesting that multi-viewpoint CME observations are marginally useful.

The most comprehensive assessment of ToA performance was completed very recently [25] (figure 2). The study compared ToA predictions from 32 different models for 28 events between 2013 and mid-2018 and found that the ToA accuracy was −3.7 ± 17.1 h. The mean absolute error, a better indicator of the overall model accuracy, was 12.9 h in line with the studies we discussed above. Some models or forecasting centers did significantly better on average but the mean absolute error was broadly the same across all models and forecasters. Riley et al. [25] discuss a number of potential avenues for improvement (on which we will expand later) and bring up the issue of the lead-time for a specific forecast. Obviously, the earlier a forecast is made the more useful it may be to the end-user.

Figure 2.

Variation in Δt as a function of year for all 32 different model predictions. The line in the centre of the box gives the median of the data, while the tops and the bottoms of the box give the lower and upper quartiles. The ends of the vertical lines give the minimum and maximum values of the data (provided that there are no outliers), while any circles give the values of outliers (more then 1.5 times above/below the upper/lower quartiles). Reproduced with permission from Riley et al. [25]. Copyright © 2018, John Wiley and Sons.

Space weather is a worldwide concern, however, drawing significant research interest within the Chinese science community in the recent years. This led to the establishment of the Space Environment Prediction Center (SEPC) [38] and efforts to transition research models to operational forecasting (R2O). The MHD models adopt a different approach [29] than the WEC-type models used in the West. A report from the early R2O efforts in SEPC shows modest performance with a mean absolute error of 18 h for 25 CMEs [38].

(e). Machine learning

Finally, machine learning algorithms have been introduced to the ToA problem [27,28] spurred by the increase in computing power and the rapid development of artificial intelligence algorithms.

A neural network (NN) to obtain the arrival time from the CME speed in LASCO FOV and the location of the associated flare (taken as a proxy for the CME propagation direction) was applied in [27]. This is effectively an empirical model approach as in [6] or [15] but without the need to define a fitting function. The application to 153 events resulted in average error of ≈ 12 h, similar to the performance of those models.

An open source machine learning package, the support vector machine (SVM) algorithm, was adopted by Liu et al. [28], who used 182 geoeffective events to train the model. To build the engine, they investigated the statistical impact of 18 CME and solar wind observational parameters in predicting the ToA. Rather as expected, the CME average and final speed (in the LASCO FOV) were the most important, followed by the width and mass. Training the model on 145 events and validating it against 37, the authors obtained an MAE of 5.9 ± 4.3 h, which compares well against other models [25]. While these early attempts look promising, they are still empirical approaches and lack physical insights into the CME evolution problem.

(f). Assessment of time-of-arrival studies

We summarize in table 1 the findings from the studies we discussed so far. Because not all papers report their ToA or SoA consistently, we decided to focus on the MAE because we find it easier to interpret. We calculate the MAE (and its standard deviation) when it is not reported on the paper. We mark those entries on the Table and any errors are our responsibility. We calculate the unweighted mean of all MAE values as the representative value for the current state of accuracy of ToA studies. It is 9.8 ± 2 h, which is similar to the MAE from large-scale studies [25,26] involving MHD modelling of the propagation. The large range in sample sizes complicates the comparison among the various approaches. It seems obvious that small-scale selective studies could bias ToA towards smaller values. To test this, we plot, in figure 3, the MAEs from table 1 but use the standard error ($\sigma /\sqrt{N}$), where N is the number of events in the sample, as error bars. We mark studies using projected CME quantities (or those relying on single-viewpoint observations) with red font. It appears that small-sample studies (N < 10) may be biased. It also seems that the effects of projection, and/or reliance on single-viewpoint observations, become apparent in large-scale studies. The case for studies using 3D CME kinematics is a bit unclear. There is quite a scatter in the results for sample sizes between 10 and 70 events. We believe that more large-scale studies, along the lines of [26], are needed, possibly covering different phases of the cycle or event speeds, before demonstrating a clear benefit of 3D kinematic inputs.

Figure 3.

Mean absolute error in the time-of-arrival of CMEs versus event sample size from the studies compiled in table 1. The error bars represent the uncertainty in the mean value ($\sigma /\sqrt{N}$, where N is the sample size). The star symbols and thicker error bars denote studies with projected CME parameters (or a mixture of projected/unprojected inputs). The dashed line is the overall unweighted average (9.8 h).

(g). Hit/miss

Before investing resources into predicting an accurate ToA, a forecaster needs to be reasonably sure that the event seen in the coronagraph images will impact the Earth. The ‘hit/miss’ prediction, which was not often discussed in the past, is attracting interest. Several papers, in the last few years, have examined hit/miss statistics and methods to improve the hit rate. [14,18,26,39,40].

There exist several metrics to assess the ToA accuracy performance of a given method or model [e.g. see table 2 in [26]]. Those are derived from contingency tables that report the number of hits (H), false alarms (FA), misses (M) and correct rejections (CR). For the sake of brevity, we select four metrics to summarize the performance of the handful of studies where hit/miss statistics were reported (table 2). The metrics are: success ratio: H/(H+FA), false alarm ratio: FA/(H+FA), accuracy: (H+CR)/total and bias score: (H+FA)/(H+M). A bias score greater than 1 indicates a tendency to overforecast. In other words, a CME arrives less often than is predicted. All four studies show similar overforecasting biases, which may arise from the tendency to make a ToA prediction for any event that appears Earth-directed, in case it is geoeffective (see also [26]). We also note a trend of lower scores as the number of simulated events rises. This may point to a selection bias for small samples and/or the increasing impact of the many uncertainties in the ToA chain (which we discuss later).

Table 2.

Comparison of selected skill scores for ToA predictions.

success ratio	false alarm ratio	accuracy	bias score	sample size	model & reference
0.77	0.23	0.83	1.29	30	WEC; [18]
0.40	0.60	0.85	1.33	1663	WEC; [26]
0.44	0.56	0.90	1.44	697	SSEF; [14]
0.80	0.20	0.84	1.25	25	DBM; [17]

Obviously, any information that can improve the hit/miss statistics is welcome. To test whether the CME orientation in the low corona is a reliable predictor for Earth impact, [39,40] applied their CME deflection and rotations model on a set of 8 Earth-directed CMEs. After simulating a larger number of orientations, they derive a regression curve that separates hits from misses based on the CME tilt and normalized angular distance [39]. The regression curve continues to perform well when the study is expanded to 45 events showing only six non-conforming events [40]. The authors suggest that the discrepancies arise from the thickness of the CME, defined as the ratio of the cross-sectional width to the full width.

(h). What limits the accuracy of coronal mass ejection arrival prediction?

Much research has been devoted to the CME arrival at an in situ detector (particularly at Earth). The prediction of the CME impact and its timing (tables 1–2) have benefited considerably from the availability of off Sun–Earth Line (SEL) viewpoints from STEREO. However, the errors remain at almost half a day (actually 9.8 h, table 1) and they seem to depend on the sample size, event selection, level of solar activity, etc. There are various reasons, observational and modelling, that prevent a more accurate determination of the ToA. Most of them are discussed, in various degrees of detail, in the references. Measurements or modelling deficiencies are partly responsible. Our (still) limited understanding of the physics of the inner heliosphere and the immense size of the physical system we have to deal with are responsible for the rest. In the following, we attempt to categorize the most important of issues in three categories: physics, observations and models.

• —
  Physical limitations

  • (i)
    CME evolution 30 R_s-1 AU. This, admittedly broad, subject lies at the heart of almost every discussion on ToA accuracy. CMEs have been extensively imaged and measured (remotely) within about 30 Rs (the LASCO-C3 FOV). At the same time, CMEs have been measured in situ at 1 AU, and sporadically within 1 AU, for much longer. The connection between the two regimes only began to be explored in earnest in the last 10 years, thanks to the STEREO mission [e.g. [41–43]]. Regarding the CME kinematics, the literature we have reviewed revolves around the following issues.

    • (a)
      CME interactions with the ambient solar wind, including other CMEs (see recent review by Manchester et al. [44]). The structure of the background affects the drag experienced by a CME resulting in very different kinematic behaviour from event to event [45,46]. CMEs can rotate, deflect (in latitude as well as in longitude), break or accelerate ([44] and references therein). We need to understand the force balance of a CME as it propagates in the IP medium, yet such studies are rare.
    • (b)
      The CME energy budget. This is another area of research that is sorely lagging behind. The few studies examine the CME energy balance below 30 R_s [47–49]. There are no similar studies at higher distances from the Sun.
    • (c)
      Shape of the CME front/shock at 1 AU. As we saw earlier, attempts to take into account the curvature at the CME front (as deduced by 3D reconstructions) are not always successful in improving ToA and/or SoA [10,20]. Theoretical models, based on compilations of extensive sets of in situ observations, relating CME/shock shapes at 1 AU with coronal properties such as their width exist [50], and should be compared with actual observations. A recent evaluation of different shapes in the MHD modelling of a single (and rather simple) event demonstrated the importance of that parameter for SWx forecasting [51]. Yet, the multi-view imaging resolution is too low to constrain the shape beyond about 30 R_s. More sensitive imagers are needed to make progress on this.
  • (ii)
    The CME size at 1 AU is much larger than the Earth. This seems to be the ultimate physical challenge. We do not have a direct measurement of the CME size at impact but the in situ reconstructions and MHD modelling reveal structures with sizes at considerable fractions of 1 AU (figure 4). Obviously, accurate predictions of ToA (and momentum or magnetic field) will require high precision modelling of the 3D shape of the transient that is beyond our current capabilities.
• —
  Observational limitations

  • (i)
    IP interactions are not well characterized by current sensors. The heliospheric imaging from SECCHI and SMEI [53] gave us a tantalizing peek at the behaviour of solar structures during transit. The designs of these trailblazing telescopes were based on estimates of the expected signals rather than actual observations and hence were conservative. We now know the possibilities and challenges. We can image the extremely low brightness of ICMEs (at 15 orders of magnitude fainter than the solar disk!) but we cannot reliably image the internal ICME structure, particularly the magnetic flux rope (MFR), nor can we follow the evolution of the CME front or of the shock sheath, both of which are key parameters in SWx forecasting. Higher spatial resolution, better 3D discrimination along the line-of-sight, from off-‘Sun-Earth-Line’ (SEL) viewpoints, are all required from future imagers.
  • (ii)
    Uncertainty in coronagraph measurements. All ToA techniques rely on measurements of the CME—direction, speed, shape—at coronagraph heights (up to approx. 30 R_s). Despite the availability of high spatial resolution and high sensitivity imaging of thousands of events, the measurement uncertainties (particularly for SEL geometries) remain high (table 1). It is our opinion that this is an area where progress could be made using existing datasets and analysis methods, simply by a more careful interpretation of the images. For example, we now know that the outer front of a halo CME is the shock sheath, e.g. [54,55]. Therefore, with the proper care, the shock or the CME front (depending on the study objective) could be measured consistently from image to image. The more realistic ‘croissant’ CME shape should probably replace the popular ‘ice-cream cone’ assumption, since it is evident that CMEs are not ‘bubbles’. Another suggestion is to standardize the derivation of the kinematic profiles by requiring a minimum number of height-time measurements (say, at least 4), always fitting them with the same function (e.g. a 2nd-degree polynomial) and providing the speed and direction, always at the same heights (e.g. 20 R_s). This should ease inter-comparison of propagation models.
• —
  Modelling limitations

  • (i)
    Weak coronal and background solar wind modelling constitutes the ‘Achilles’ heel' of current MHD modelling. The lack of synchronous photospheric field measurements across the full solar surface limits the reliability of the solar wind simulations, especially during periods of high solar activity. Jian et al. [56] tested various solar wind models under the Community Coordinated Modelling Center (CCMC) and found that each model performs better in a different physical parameter (e.g. density, magnetic field, etc.). The limited knowledge of the physical parameters in the sub-Alfvènic corona (below about 20 R_s) hinders robust modelling of the initial stages of CME propagation and shock evolution.
  • (ii)
    Large-scale MHD simulations, required to capture the propagation to 1 AU, frequently lack proper treatment of microphysics (turbulence, waves). It is unclear how important these physics are for SWx forecasting.
  • (iii)
    Modelling techniques long used for weather forecasting, (e.g. data assimilation, ensemble modelling) are still in their infancy in Heliophysics.

Figure 4.

Plasma β (colour scale) and magnetic flux function (black contours) are displayed for three times during an MHD simulation. The edge of the closed field lines for the magnetic flux rope are displayed as a thick line (green). The black crosses denote manually selected positions through the flux rope, and the dashed line is an arc of a circle that is optimally chosen from the crosses. The middle panel identifies the uniformly distributed positions along the arc that is later used in our analysis (red squares). Reproduced with permission from [52]. Copyright © AAS. (Online version in colour.)

(i). Path forward

The extensive work on ToA seems to paint a clear picture of the issues that restrict the forecasting accuracy of CME ToA and SoA. On one hand, we need to understand the CME propagation beyond about 30 R_s, i.e. force balance, CME response, shock evolution, structure of the ambient wind, etc. On the other hand, we believe that the ToA accuracy (say approx. 9.8 h) is about the best that can be achieved with the currently available data. The 3D aspects of the STEREO data have been exploited extensively. Imaging and in situ data (in multiple inner heliospheric locations) have been combined and analysed. Although better image processing algorithms or in situ reconstruction methods may improve localizations and kinematics, they are unlikely to result in a breakthrough.

So, how can we move forward? We can do so only by improving the modelling of CMEs through the inner heliosphere. Better modelling needs more accurate boundary conditions, stronger observational constraints, and consistent methodology to assess success. Our suggestions, in a nutshell, are:

• —Better observational constraints
  • –Improve instantaneous coverage of the photospheric magnetic field (e.g. via a magnetograph on an L5 mission).
  • –Deploy HI with higher signal-to-noise imaging.
  • –Maintain off-SEL imaging.
  • –Deploy in situ monitors ahead of L1, optimally at 0.3 AU from Earth.
• —Consistent methodology
  • –Provide skill scores in studies.
  • –Develop a common set of metrics for easier comparison of techniques.
  • –Repeat CCMC CME Scoreboard ² studies on regular intervals.
• —Better models
  • –Develop data assimilation techniques, particularly of CME kinematics in the inner heliosphere.
  • –Incorporate the magnetic structure of the CME in the MHD models.

Obviously, this ‘wish list’ contains both short- and long-term items. Some are easy to implement while others require significant (and long-term) investment from funding agencies and efficient international cooperation. In other words, we need a detailed R2O strategy and implementation plan (as we discussed in [57] and return to in §6) that includes the appropriate mix of instrumentation, data analysis and theory/modelling components.

3. Forecasting coronal mass ejection mass at 1 AU

(a). State-of-the-art

Compared to the extended literature behind the prediction of the entrained magnetic field (discussed in §5), the CME mass (or density) at 1 AU has drawn very little attention. It can have considerable space weather impacts because the CME sheath (and following structures) compress the magnetosphere and drive currents through the system [58]. Since the CME density is not uniform in space and time, the resulting modulation of the magnetospheric compression exacerbates these phenomena (see [52], and references therein). As these effects usually occur in conjunction with the magnetic effects of the CME, the ability to separate those effects will go a long way towards understanding the genesis of terrestrial SWx.

The first investigation into the possibility of forecasting the momentum flux of a geoeffective CME was undertaken by Savani et al. [52]. They used multiviewpoint observations of a CME in COR2 to reconstruct the event and extract the density profile of its front. They then propagated the CME ballistically to 1 AU and compared the resulting density time-series to the observed ones with very good results.

On the modelling side, [46] used the WEC to estimate the 1 AU ram pressure of the 7 March 2012 CME to drive a model of the magnetosphere. The results were in qualitative agreement with the in situ measurements. In a larger study examining ensemble WEC modelling, [18] found that the predicted densities at 1 AU were higher, by a factor of 2–3, than the observed densities. The reason is that the operational ENLIL performs a hydrodynamic simulation of the CME and requires overdensity to drive it. Hence, the resulting density overestimations are not surprising and point to the need for magnetized CME modelling, as we mentioned in the previous section. Finally, [59] looked at the effect of the CME shape on the standoff distance of the magnetopause using a different hydrodynamic CME model and concluded that the standoff distance plays a role. They do not discuss the modelled densities though.

(b). Issues and path forward

Given the scarce literature, the analysis and use of the CME density structure in SWx forecasting is still in its infancy and there is insufficient information for a detailed discussion of the issues. The obvious path forward seems to be the analysis of large samples of the available Earth-directed CMEs using the [52] or similar methodologies to: (i) improve the extrapolations of the density in the corona and better understand its potential (i.e. for magnetospheric driving, for constraining CME directivity and/or shape, etc.) by comparing with the observed densities, and (ii) to help constrain the development of magnetized CME simulations in ENLIL or other MHD models.

4. Forecasting coronal mass ejection size at 1 AU

(a). State-of-the-art

(i). Duration of impact predictions

The duration of CME impacts at 1 AU, or conversely their size, is an important parameter for their geoeffectiveness (e.g. [60]). However, extensive studies of the duration of impact have been barely undertaken to date. A comprehensive study of predicted versus observed CME impact durations has been performed in [13]. 3D reconstructions of the structure and kinematics of 31 Earth-directed CMEs with flux-rope morphologies observed in the corona and the inner heliosphere by the STEREO coronagraphs were used to determine the duration of their impacts at L1 which were compared with the associated magnetic cloud (MC) durations, as inferred by in situ observations at L1. The [13] CME reconstruction method fits amongst others a density shell to the CME front- and back-ends, therefore supplying an assessment of its cross-sectional size, assuming self-similar expansion. The results are displayed in figure 5. It is clear that the predicted flux-rope durations are significantly larger than the actual magnetic cloud durations, with average values of 54.8 and 18.5 h, respectively.

Figure 5.

Predicted ICME durations at 1 AU from geometrical modelling of CME STEREO observations in the corona and the inner heliosphere and associated MC durations from in situ observations. Reproduced with permission from Wood et al. [13]. Copyright © AAS. (Online version in colour.)

(b). Issues and path forward

The significant discrepancies between the predicted and observed ICME durations at 1 AU could be, at least partially, related to how CMEs, particularly in the interplanetary space (ICMEs), are identified in remote sensing and in situ data, given the intricacies of each mode of observation. First, coronagraphic and heliospheric imaging observations identify enhanced density structures with flux-rope morphologies as CMEs/ICMEs (e.g. [55]) whereas the in situ observations associate density-depleted structures of enhanced magnetic fields exhibiting rotation with magnetic clouds (MCs) (e.g. [61]). Second, while remote-sensing observations supply a global view of CMEs/ICMEs, the small-scale resolution is smeared due to line-of-sight superposition of various related or otherwise structures. This issue is partially mitigated by the multi-viewpoint STEREO imaging in the corona. The situation is more challenging with heliospheric imaging observations. Indeed, the STEREO HI identify ICMEs by detecting the density at the compression region, i.e. the sheath, developed around the ejecta, and not the (magnetic) ejecta per se. In addition, the long exposures required to capture the faint heliospheric signal of transients contributes to the blurring of CME small-scale structures. On the other hand, in situ observations are single-line cuts through immense structures (figure 4). Third, the strongly twisted magnetic field lines in ICMEs, which are identified as MCs, could represent a fraction of a more complicated and extended ICME structure as suggested by the analysis of in situ observations of many ICMEs (e.g. [62]). Such a pattern could be due to multiple ICMEs with their sheaths arriving as a complex structure at 1 AU. In addition, the separation of ICME magnetic flux into strongly and weakly twisted components could also result from the evolution of plasma β in a CME during its heliospheric propagation and how this specifically affects the distribution of magnetic flux within it. This effect was studied in detail with MHD simulations of CME propagation in the heliosphere [52]. In figure 4 from this study, it can be seen that the CME can be decomposed into two distinct domains: a central, nearly circular inner core threaded by strongly twisted magnetic fields, surrounded by an envelope with more deformed shape, due to its stronger coupling with the ambient solar wind. The MC that would be identified by an in situ observation would be the inner core of this composite structure. As a matter of fact, strongly twisted magnetic field lines could inhibit the expansion and size of MFR as modelling of in situ observations of MCs suggest (e.g. [63]).

Clearly, more studies of the duration of CME impacts are needed. This should be a rather straightforward task given the availability of 3D geometrical reconstructions and associated ENLIL and other MHD simulations for a large statistical sample of CMEs observed during the STEREO era. In addition, and given the discussion in the previous paragraph, imminent advances in heliospheric imaging due to the Parker Solar Probe [64] (PSP) and Solar Orbiter [65] missions should help constrain CME size and structure in the inner heliosphere. Finally, it could be potentially helpful to shift the focus of CME/ICME imaging observations from the bright fronts to the inner and darker cavities so as to more sensitively connect with the in situ observations.

5. Forecasting coronal mass ejection magnetic fields at 1 AU

In terms of geoeffectiveness, the CME (and associated shock) magnetic field geometry and strength are the most important physical quantities, e.g. [66]. In particular, the strength and duration of the southward (relative to the Earth's dipole) component, commonly referred to as B_z, determines the degree of magnetic reconnection and hence the transfer of energy from the CME to the magnetosphere. Another component, B_y, plays an important role in magnetospheric dynamics, particularly during times of northward B_z (see [67] and references therein) but it does not drive major geomagnetic storms by itself and thus we do not discuss it further here. B_z is, in a sense, the ‘fire hose’ of geomagnetic storms and, naturally, its forecasting lies at the top of the space weather operations wish list. Unfortunately, B_z is also the most difficult parameter to forecast because the magnetic field of the CME cannot be measured remotely close to the Sun (contrary, for example, to the CME speed), it can be highly structured, and the propagation of the CME to 1 AU is not well understood, as we have discussed already.

In principle, a simulation which self-consistently treats magnetic flux emergence, CME formation, coronal and heliospheric propagation and evolution could forecast B_z, and many of the other parameters of SWx interest. However, we do not yet have the capability, or system understanding, to perform such comprehensive simulations [68,69]. For now, the efforts around B_z prediction invoke empirical or semi-empirical models that attack the full problem or parts of it. We organize the review of these efforts by first decomposing the problem.

(a). Decomposing the B_z problem

As we understand it, the prediction of B_z consists of two broad steps: (i) estimate (or measure) the magnetic field strength and geometry of the transient near the Sun, and (ii) propagate the (magnetized) structure to 1 AU. The two origins of geoeffective B_z, and their differences, are conceptually summarized in table 3. In a nutshell, the CME field originates from stressed fields over a polarity inversion line (in the core of an active region for the most geoeffective events) while the field in the shock sheath is the accumulation of the magnetic field surrounding the CME as it exits the corona into the heliosphere. The sheath fields are, on average, weaker but comparable to CME core fields (for fast events [70]), and their basic features (e.g. ToA, compression) could generally be modelled via standard MHD modelling—this is how ENLIL, for example, is used to predict B_z. Because these are generally not as geoeffective as the CME B_z (e.g. [70]) and due to space constraints, we focus only on the CME-related B_z predictions in the remainder of the section.

Table 3.

The origins of geoeffective B_z.

	CME	shock sheath
origin	bulk of erupting flux due to reconnection during CME	drapping of fields ahead of CME
geometry	magnetic flux rope (MFR)	∼2D sheets
topology	twisted, helical	random
evolution	reconnection (internal, with ambient)	compression, reconnection
forecasting method	empirical	MHD modelling

(b). The magnitude of the magnetic field

The magnetic field in the corona can be measured remotely either with radio techniques [71,72] or off-limb spectroscopy and spectropolarimetry [73]. Within a CME, it has been estimated only for a handful of cases, all using radio diagnostics, such as gyro-synchrotron emission from relativistic electrons trapped in the ejecta (e.g. [74–77]) or Faraday rotation (e.g. [72,78,79]). Unfortunately, these diagnostics cannot be used on a routine basis because of the dearth of solar-dedicated radio arrays with the required sensitivity and observing continuity.

Magnetic field extrapolations is another way to estimate the field in the CME by estimating the magnetic flux removed by the event (e.g. [80]) using magnetograms before and after the eruption. This is, however, not straightforward. It requires rather accurate knowledge of the volume, location and magnetic connectivity of the erupting structure which in turn requires off-SEL viewpoints and multiple wavelengths to enable tracing the structure from near the solar surface to the final CME formation at 2–4 R_s.

(c). Empirical models

An array of physics-based empirical models to infer/forecast the CME magnetic field has emerged over the last decade. They are based on basic physical arguments/concepts (e.g. magnetic flux and helicity conservation), analytical models, and empirical relationships between ICME parameters and radial distance derived from HELIOS observations (e.g. [81–83]). In addition, they require a set of empirical constraints on the Sun, corona and 1 AU (e.g. source-region observations, multi-viewpoint coronagraphic observations, in situ observations), which are typically readily available. The computational simplicity of these models makes them ideal for exploring the physical parameter space efficiently, and hopefully lead to meaningful forecasts. A summary of the basic elements of these models is given in table 4; a more detailed discussion is in order.

Table 4.

Basic elements of physics-based empirical models of CME magnetic field inference in the corona and in the inner heliosphere. FF: force-free; NFF: non-force free; a: CCMC simulations of CME sheath region are used to obtain the CME magnetic field magnitude at 1 AU; b: in situ observations at 1 AU are used to obtain the CME magnetic field magnitude at 1 AU; c: in situ observations at Mercury were used.

model	principle	shape	B	coronal —B—	IP evolution	B vector
HELIO-XM	hoop force	torus	NFF	Y	force balance	Y
BFE	CME energetics	cylinder	FF	Y	flux conservation	N
BZ4CAST	region of influence	cylinder	FF	Na	flux conservation	Y
H-CME	magnetic helicity conservation	cylinder/torus	FF/NFF	Y	power-law with varying αB	N
FIDO	force balance	torus	NFF	Nb	flux conservation	Y
Fried	flux conservation	torus	FF	Nb	flux conservation	Y
FRED	reconnected flux	torus	FF	Y	flux conservation	N
CORE3D	populate 3D torus with 2D sections	torus	FF	Nc	flux conservation	Y

However, the near-Sun CME magnetic fields of these models are based on inferences, and not ‘direct’ measurements as discussed in the previous section. Also, ToA forecasting (§2) is either decoupled from the CME magnetic field forecasting or simply disregarded by conveniently placing the simulated magnetic field time-series at the start of the corresponding ICME interval. In addition, the models are based on smoothed prescriptions of the magnetic field within CMEs (e.g. Bessel functions for the cylindrical linear force-free Lundquist model [84]). Therefore, they are amenable to comparisons with (and predictions of) only the large-scale magnetic structure of CMEs, and not their ‘fine’ spatial or temporal structure (say, of a couple of hours or less). We proceed to discuss these models below.

(i). Empirical models

An indirect way to estimate the magnetic field of CMEs is by associating the observed geometry to the idealized MFR structure that all CMES are thought to contain. In its basic application, the comparison provides an estimate of the magnetic flux within the transient, which can be extrapolated to 1 AU under the assumption of flux conservation [48,85] and turned into the average field strength (but not the direction) within the CME.

Project-Zed [86] takes a completely different approach to the problem. It applies a simple pattern recognition algorithm to archival solar wind 1 AU observations. For a given reference time t, and temporal window dt (hours), Project-Zed considers all backward sliding windows of length dt, and compares them with the dt-hour window prior to t. The 50 best-matching sliding windows are used in ensemble forecasts for the dt hours past t. Currently, Project-Zed does a decent job at forecasting quiescent or recurrent conditions but fails for CME or sheaths B_z. This implies that the structure of these fields is driven by each event and carries no ‘memory’ across solar cycles.

A somewhat similar approach but targeted more specifically to the coherent B_z disturbances driven by CMEs (and CIRs) is the Bayesian prediction method of [87] which was expanded in [88]. Using real-time in situ magnetic field measurements and an MFR model for the magnetic structure of a CME, the method tries to reconstruct forward in time the arriving structure, thus estimating the B_z profile and duration. It then uses a Bayesian framework trained in prior cycle in situ magnetic field measurements to predict the severity of the ensuing storm. The model, validated against approximately 13 years of L₁ magnetic data from the Wind spacecraft, showed an 81% success rate in predicting moderate to large storms (as expressed by the maximum Dst index of the storm) [89]. The authors assess that the method could provide warning times of 10–15 h with continuing improvement as additional data (hence, B_z patterns) are ingested into the Bayesian framework.

The Helicity-CME (H-CME; [46]) model is based on the application of the magnetic helicity conservation principle [90] in flux-rope CMEs. H-CME uses calculations of the eruption-related or source region magnetic helicity from photospheric magnetic field and/or flow observations and occasionally magnetic field extrapolations [91]. Next, application of the GCS model to STEREO and SOHO coronagraphic data supplies estimates of the corresponding CME geometrical parameters (e.g. length, radius). Invoking the magnetic helicity conservation principle, and plugging the derived magnetic helicity and CME geometrical parameters to analytical relationships developed for various models (e.g. [92]), supplies the coronal CME magnetic field magnitude. This near-Sun CME magnetic field magnitude is extrapolated to 1 AU assuming a power-law drop with radial distance with an index α_B (e.g. [93]). The model has been applied to a single event study [46]. Parametric studies of H-CME taking into account the statistical distributions of its input parameters applied to various analytical models showed that it could reproduce the statistical distributions of ICME observed magnetic field magnitudes at 1 AU, for a narrow range of α_B values [94,95]. The estimated near-Sun CME magnetic fields are difficult to validate but they were higher than the background coronal magnetic fields derived from radio observations at the same heights. A useful byproduct of this model is the calculation of the helicity sign which could be associated with the erupting flux, which is a major element in determining the full magnetic field vector in CMEs.

Finally, BZ4CAST [96,97] is a modular B_z prediction framework, developed specifically for operational deployment (figure 6). It treats the CME as an MFR-carrying structure. The framework considers the full life cycle of the event, from its source location on the solar surface (which provides the initial helicity and orientation of the magnetic structure), to its coronal evolution (which provides the final orientation via a GCS fit at 15 R_s) to its encounter with Earth. The MFR propagates radially in the inner heliosphere without distortions or other alterations (in the current framework). The solar and coronal observations provide an estimate of the volume of influence at Earth impact, including the impact radius (figure 6, central panel). The MFR structure is derived from a constant-alpha force-free model, its projection along the Earth trajectory is derived from the coronagraph plus propagation models, and the magnetic field strength can be provided from a variety of methods (e.g. [46,75,98,99]). The ability of the BZ4CAST framework to incorporate different models and estimates makes it very appealing for further development.

Figure 6.

Screenshot of the Bz4Cast model technique being implemented for the 26 August 2015 CME event. The Earth trajectory is created by using solar imagery to deduce the CME location. The Bphi component (fourth panel from top in the times series) correctly increases with time. (Online version in colour.)

(ii). Semi-empirical models

The HELIO-XM [98,100,101] model is based on the Chen [102] flux-rope model. It includes the Lorentz-self force (hoop force) which corresponds to an outward force due to the radial gradient of the poloidal magnetic field in curved flux ropes. In addition, the model incorporates the aerodynamic drag-force to describe the momentum exchanges between the flux rope and the ambient solar wind. The major input to this model is the poloidal magnetic flux injection profile constrained by fitting of the height-time plot of the CME, in the low corona or even in the inner heliosphere. Other model parameters include the flux rope location, footpoint separation, mass and the ambient solar wind and heliospheric magnetic field properties. Solving the force-balance equations of the expanding flux rope yields the full magnetic field vector at any distance in the corona and the interplanetary medium. The flux rope is assumed to propagate in the radial direction without experiencing any rotations. The model has been applied to one event in [98] with a good ballpark agreement between the modelled and observed CME magnetic field vector at 1 AU. HELIO-XM is the only model of its kind to include both CME initiation and propagation.

The ForeCAT In Data Observer (FiDO; [39]) model aims to forecast the magnetic field profiles of CMEs. The backbone of FiDO is the Forecasting a Coronal Mass Ejections's Altered Trajectory (ForeCAT; [103]) model. ForeCAT is a semi-analytical model which calculates longitudinal and latitudinal deflections and rotations of flux-rope CMEs in the corona. The deflections result from magnetic forces due to the background solar wind magnetic field draping around the CME body, while the rotations result from differential deflection forces acting on the CME's toroidal axis. FiDO propagates into the inner heliosphere the ForeCAT-derived CME longitude, latitude and orientation. The magnetic content is provided by populating the toroidal body of the CME with a Lundquist magnetic field distribution. Assuming self-similar expansion and applying flux conservation provides the temporal variation of the CME axial magnetic field. However, in its current implementation, FiDO uses 1 AU in situ ICMES observations to constrain the CME magnetic field magnitude in the heliosphere. To date, FiDO has been applied to 49 events with a ballpark agreement between the modelled and observed CME magnetic field vector at 1 AU [39,40], with an hourly average error of approximately 35% for the three components of the magnetic field (normalized to the mean ICME magnetic field magnitude).

FRiED is a 3D flux rope model [104]. It is an analytical fitting model applied to both coronagraphic observations and in situ measurements of the associated ICMEs. Its remote-sensing module fits the large-scale structure of CMEs as observed by multi-viewpoint STEREO and SOHO observations, with a flux-rope model with its 3D shape resulting from force-balance considerations. It goes beyond the frequently used GCS model because it incorporates geometrical deformations such as pancaking, front flattening and rotational skewing. The 3D shape of the flux-rope CME is then populated with field lines locally (i.e. at every cross-section along the flux rope) described by the Lundquist model; the variation with distance along the flux rope of the maximum axial magnetic field results from magnetic-flux conservation. The model requires a relatively large number (11) of positional, geometrical and magnetic parameters, and it is fitted separately against coronagraphic and in situ observations at 1 AU. Given the lack of pertinent observations in the corona, the magnetic parameters are included only in the in situ fittings. Radial IP propagation of the CME at constant speed is assumed. Similar to FIDO, FRiED uses the fits of the in situ measurements of the actual CMEs at 1 AU, to constrain the CME magnetic field magnitude in the corona. To date, FRiED has been applied to two events with a ballpark agreement between the modelled and observed CME magnetic field vector at 1 AU.

The Flux Rope from Erupting Data (FRED; [105]) model combines photospheric and coronal observations to estimate the near-Sun magnetic field of CMEs related to flares. FRED calculates, using photospheric magnetograms, the magnetic flux straddled by the post-eruption arcades observed in coronal images. This flux is a proxy of the reconnected flux during the associated eruptive flare and sets a lower limit of the poloidal magnetic flux of the erupting flux rope in the case that a flux rope existed in the corona before the eruption. Then, GCS modelling of the associated CME in the corona, employing STEREO and SOHO coronagraphic data, yields estimates of its geometrical parameters such as its length and radius. The near-Sun CME magnetic field is finally estimated by adopting the poloidal magnetic flux and geometrical parameters described above in a Lundquist model. The near-Sun CME magnetic field derived by FRED is then extrapolated to 1 AU assuming self-similar expansion and magnetic-flux conservation (α_B = − 2). To date, FRED has been applied to a single event. A variant of FRED, based on single-viewpoint SOHO observations, was applied to 49 more events [99]. The derived near-Sun CME magnetic fields were found to be higher than the background coronal magnetic fields derived from radio observations at the same heights, as anticipated.

The three-dimensional Coronal ROpe Ejection model (3DCORE; [106]) uses solar, coronagraph and interplanetary observations to derive the 3D magnetic field distribution of flux-rope CMEs in the corona and the interplanetary medium. 3DCORE uses an elliptical torus to describe the shape of the ICME. The torus is tapered (i.e. its cross-section increases with distance) and is populated with a stack of cross-sections with their magnetic field distribution described by the uniform-twist Gold and Hoyle model. The maximum axial magnetic field in each cross-section results from the application of a radial power-law prescription with α_B = − 1.64 from [83] to the CME apex instantaneous distance. In its current implementation, 3DCORE uses in situ observations of the actual CME either around Mercury (MESSENGER) or at 1 AU to determine the near-Sun CME magnetic field. The torus is initiated in the corona using geometrical and positional parameters and CME speed derived from a GCS fit and from fitting the elongation-time plots in HI. The CME propagates radially in the IP medium under the DBM model. Its minor radius follows a radial power-law deduced resulting from ICME fit in HELIOS. To date, 3DCORE has been applied to one event with a ballpark agreement between the modelled and observed CME magnetic field vector at 1 AU.

(d). Why is difficult to predict B_z?

As present, we cannot reliably predict B_z, or B_y for that matter, beyond the 40–60 min measurement horizon afforded by direct in situ measurements from L₁. The existing physics-based empirical models can only reproduce, rather than forecast, the large-scale CME magnetic structure at 1 AU. Our summary of the main reasons for this shortcoming follows:

• —The magnetic properties of the CME at-birth are unknown. CME formation is very much a subject of intense research. The magnetic properties (magnitude, topology, helicity) of the ejected structure cannot be measured directly via remote methods in a comprehensive fashion. Radio or EUV off-limb imaging and spectroscopy provide occasional estimates for parts of the problem (e.g. magnetic field magnitude).
• —The ambient coronal properties within the Alfvén surface (nominally less than 20 R_s) are uncertain. This is the region where the solar wind heats and accelerates and where the lack of coronal density, temperature, and magnetic field measurements greatly reduces the reliability of MHD modelling in the region. This is also the region where the initial evolution of CMEs and their shocks occurs and where many of their physical properties are established [55]. Therefore, we have a weak handle on how the ambient field responds to the generation of the shock and the propagating CME, how the CME magnetic energy transforms into heat and kinetic energy [47,48], and how (and when) the magnetic connection to the source region is severed.
• —The CME evolution to 1 AU is uncertain. This is the same problem that affects the ToA forecasting, discussed in 2(g). The issues that affect B_z are the evolution of the magnetic structure (rotation, compression, deflection), including erosion from reconnection to ambient fields (see §7.2 in [44]), and the evolution of the sheath. These processes affect both the magnitude and geometry of the CME magnetic field at 1 AU [62]. Again, we have no means to probe the actual magnetic structure of the interplanetary CME remotely. We can only make inferences from observations of the density structure, which are quite uncertain as we discussed in 2(g).

(e). Path forward

To a large extent, most of the recommendations for improving the ToA forecasting (2(h)) are also valid for B_z. We repeat those below but we add the primary B_z-specific justification in italics.

• —Better observational constraints
  • –Improve instantaneous coverage of the photospheric magnetic field (e.g. via a magnetograph on an L5 mission [57,107]). To improve characterization of ambient magnetic field for sheath and CME front evolution.
  • –Deploy HI with higher signal-to-noise imaging. To resolve and follow the evolution of the MFR cavity in the ICME.
  • –Maintain off SEL imaging. To resolve and follow the evolution of the MFR cavity in the ICME.
  • –Deploy in situ monitors ahead of L₁, optimally at 0.3 AU from Earth. To measure directly the CME magnetic properties with approximately 24 h warning time.
  • –Improve measurement/estimation of coronal magnetic fields in CMEs. It requires a multi-prong approach:
    • (i)Develop hybrid models capitalizing on the best of the empirical models of Bz prediction of §§5c.
    • (ii)Improve fidelity of coronal magnetic field extrapolations via detailed comparisons to observed structure (e.g. [108,109]), stereoscopy [110], and measurements of the magnetic field vector in multiple heights [111].
    • (iii)New or upcoming radio imaging spectroscopy instrumentation in the radio domain (LoFAR, MUSER, MWRI, SKA [112]) could achieve higher sensitivity observations, and significantly increase the sample of CMEs with mapped magnetic fields in the corona; the radio-derived near-Sun magnetic field diagnostics could be used to validate/ramify the physics-based empirical methods.
    • (iv)Bound the erupting volume via stereoscopic EUV imaging in several wavelengths (see 7.1 in COSPAR-ILWS Roadmap [113]).
    • (v)Off-limb spectroscopy and coronography (preferably from off-SEL) to assess CME initial structural and energetic evolution [114].
• —Consistent assessment approach
  • –Provide skill scores in studies.
  • –Develop a common set of metrics for easier comparison of techniques.
  • –Repeat CCMC Scoreboard studies on regular intervals.
• —Better models
  • –Develop data assimilation techniques, particularly of CME kinematics in the inner heliosphere.
  • –Incorporate the magnetic structure of the CME in the MHD models. Several such efforts are currently under way (e.g. EUPHORIA [115], COIN-TVD [116], SUSANOO [117]).
  • –Update existing models (empirical or MHD) with the upcoming measurements from Parker Solar Probe (PSP; [64]) and Solar Orbiter [65] missions. Their measurements could potentially alter the current assumptions on the radial evolution of the magnetic field, including within CMEs (if a sufficient number of events is observed).

6. Conclusion

Our goal here is to assess the current state of CME geoeffectiveness forecasting by undertaking a survey of the extensive literature on the subject. This is an active area of research and we may have not captured all efforts and publications. We hope, however, that this work makes a good starting point for deeper exploration for the interested reader.

We have surveyed the five physical properties of CMEs most relevant when considering the geoeffectiveness of an event. We summarize the current state of affairs in table 5.

Table 5.

Current state of forecasting for the main CME geoeffective parameters.

SWx concern	CME parameter	prediction status
occurrence onset/regional effects	ToA	9.8 ± 2 h
strength of interaction	Bz strength	40–60 min
	Bz topology	MFR
	speed	±200 km s−1
duration of interaction	size	∼3x overestimate
	direction	80%

We have also attempted to isolate the issues that prevent progress on forecasting accuracy and suggested a path forward for each of the five geoeffective parameters (where appropriate). Our primary conclusion from this survey is that a path forward exists; it passes through a closer integration between the research and operational communities (i.e. a strengthening of the R2O component); and requires the development of a detailed attack strategy and subsequent actions that involve both research and SWx agencies.

To keep the discussion going, we conclude the paper with the rough outline of such a plan.

(a). A research-to-operations action plan

A reading of the issues we identified in §§2–5 quickly reveals that progress in forecasting accuracy involves multiple disciplines and substantial investments in financial and technical resources. We have neither the mandate nor the technical expertise to provide such a plan. We can, however, use the lessons learned from compiling this review and our past experience to identify some of the key concepts to build a plan around. For the sake of brevity, we outline these components below and in figure 7.

Figure 7.

Improving the forecast accuracy of CME geoeffective properties requires a multi-pronged attack plan. The draft outline here demonstrates the required scope of such a plan that should extend from theory to research to instrumentation to incubation of new technologies across the full breath of Heliophysics. (Online version in colour.)

• Sensor deployment. It seems obvious that further progress in forecasting accuracy is impossible without capturing some critical measurements. We need ways to measure/estimate the CME's magnetic field at birth and initial evolution and need a robust way to follow the 3D CME propagation in the inner heliosphere, among others (see 2(g), 5(d)). These gaps can be filled by a carefully managed sensor deployment (e.g. upstream-L₁ monitors, binocular vision mission (F.1 in [113]), mission to L₅, etc. [57]) and strategic flagship missions to unexplored regimes, such as over the solar poles [118].

• Technology incubation. Some of the measurements critical for Space Weather forecasting are currently beyond our reach. We cannot measure simultaneously the magnetic field across the full solar surface, nor can we map the CME-entrained magnetic field. We cannot resolve the magnetic structure of an ICME from the surrounding density envelope, nor can we reliably follow its 3D trajectory. We need large aperture telescopes, higher EUV spectroscopic or visible light sensitivities, and access to energetically-costly orbits (e.g. solar polar, sub-L₁). Many of these problems could be overcome with new propulsion and communication technologies, and miniaturized payloads and spacecraft, if sufficient investment was made available.

• Modelling and theory improvements. Modelling and simulations can readily benefit from a slew of well-known approaches from terrestrial forecasting, such as data assimilation and ensemble modelling techniques. Several ongoing efforts are apace within the Heliophysics modelling community. Deploying magnetized CMEs in operational models is another major step. Of course, the modelling accuracy will benefit greatly from better boundary conditions (i.e. greater longitudinal coverage of the photospheric magnetic field) and from near-future measurements from PSP and Solar Orbiter.

• Sustained investment in focused research. NASA's Living-With-A-Star (LWS) program pioneered the concept of focused research to address SWx-related issues. We believe that the program remains a great success and should be expanded with more specific R2O (and O2R) subjects and larger number of teams and awards (by focusing, say, in each of the five areas on table 5).

• International cooperation. Several countries have now established SWx forecasting centers and are funding research and operations. Effective coordination can only make things better. For example, leveraging modelling advances or adopting the focused research concept (say by the European Union's H2020 programs) will accelerate the pace of discovery and guide more efficiently the deployment of hardware and technical resources in SWx forecasting.

We conclude with a call to action for all those interested in SWx, whether for its research value or its operational consequences, to come together and develop a proper action plan to finally solve the SWx forecasting problem and to understand the relationship of our planet to the Sun, that can have much wider implications across Astrophysics and the search of habitable exoplanets (e.g. [95,119,120]).

Data accessibility

This article does not contain any additional data.

Author's contributions

A.V. led the study and drafted the manuscript. S.P. led and compiled §5 and assisted in the writing. N.P.S. contributed the discussion on the metrics and assisted in the writing. All authors read and approved the manuscript.

Competing interests

We declare we have no competing interests.

Funding

A.V. was supported by NASA grant nos. LWS NNX15AT42G and HGI NNX16AH70G and by NRL grant no. N00173-16-1-G029.