New DMSP Database of Precipitating Auroral Electrons and Ions

Abstract

Since the mid 1970’s, the Defense Meteorological Satellite Program (DMSP) spacecraft have operated instruments for monitoring the space environment from low earth orbit. As the program evolved, so to have the measurement capabilities such that modern DMSP spacecraft include a comprehensive suite of instruments providing estimates of precipitating electron and ion fluxes, cold/bulk plasma composition and moments, the geomagnetic field, and optical emissions in the far and extreme ultraviolet. We describe the creation of a new public database of precipitating electrons and ions from the Special Sensor J (SSJ) instrument, complete with original counts, calibrated differential fluxes adjusted for penetrating radiation, estimates of the total kinetic energy flux and characteristic energy, uncertainty estimates, and accurate ephemerides. These are provided in a common and self-describing format that covers 30+ years of DMSP spacecraft from F06 (launched in 1982) through F18 (launched in 2009). This new database is accessible at the National Centers for Environmental Information (NCEI) and the Coordinated Data Analysis Web (CDAWeb). We describe how the new database is being applied to high latitude studies of: the co-location of kinetic and electromagnetic energy inputs, ionospheric conductivity variability, field aligned currents and auroral boundary identification. We anticipate that this new database will support a broad range of space science endeavors from single observatory studies to coordinated system science investigations.

1. Introduction

Since 1965 the US Air Force has been operating the DMSP spacecraft in low earth orbit. Space environment instruments were first manifested on DMSP 5C/F1 (launched in 1974). Observations were declassified in December of 1972 and publicly disseminated through the National Oceanic and Atmospheric Administration [Nichols 1975, 1976; Brandli 1976]. Modern payloads include sensors to measure precipitating auroral particles (SSJ), cold/bulk plasma properties of the ionosphere (SSIES), the geomagnetic field (SSM), and optical emissions in the far and extreme ultraviolet (SSULI, SSUSI). These spacecraft are polar orbiting, sun synchronous (i.e. fixed local time), axis stabilized (since DMSP 5A/F1) and nominally have an orbital period of 101 minutes, an inclination of 98.9 degrees and an altitude of about 840 km [Hardy 2008; Ober 2014].

Auroral particle measurements on DMSP date back to 1974 and were initially motivated by a need to enhance the performance of ballistic missile early warning systems whose performance can be degraded by the presence of radar auroral clutter [OFCM 2013], an effect that is co-located with the auroral zone. Thus, the need to improve the identification of the equatorward auroral boundary in support of these systems was the original justification for outfitting DMSP spacecraft with SSJ instruments for measuring auroral particle precipitation. The purpose of these instruments has evolved to support a diverse suite of operational and research topics including energy inputs, spacecraft charging, ionospheric conductivity, auroral forms, boundary identification and field aligned currents (see section 4).

Long term observations offer tremendous value for establishing climatic baselines, testing basic theoretical understanding, searching for new phenomena and refining occurrence rate estimates for rare events [e.g. Love et al., 2015; Meredith et al., 2016]. Databases of the DMSP SSJ measurements have been made available from various groups, each covering different subsets of the available products (see section 3). However, no comprehensive data set inclusive of the entire processing chain from instrument counts to calibrated fluxes and derived parameters, their error estimates and key ancillary information has been assembled together before and made available in a universally accessible and modern scientific data product. Several research groups are already utilizing the improved data set described herein (section 4).

The present manuscript is dedicated to describing a new SSJ dataset available from the National Oceanic and Atmospheric Administration (NOAA) NCEI (formerly NGDC) and the National Aeronautics and Space Administration (NASA) Space Physics Data Facility (SPDF) CDAWeb. Upgrades include differential fluxes using calibration factors that account for detector degradation, the removal of contamination due to penetrating radiation, estimates of the total energy flux and characteristic energy, uncertainty estimates for all environmental parameters, corrected ephemerides, and storage in a common and self-describing format. The products discussed herein are the outcome of a collaboration between NCEI, the Air Force Research Laboratory (AFRL), and the University of Colorado, Boulder. The companion paper Kilcommons et al. [this issue, 2016] similarly describes a new Special Sensor Magnetometer (SSM) dataset. In this manuscript section 2 describes the SSJ instrument, section 3 details the creation of the modern dataset, section 4 reviews applications, and section 5 provides a summary of the project.

2. Instrumentation

In this section, we summarize the heritage of the DMSP SSJ instrument, note several key differences and provide a description of the viewing geometry with respect to a nominal particle loss cone for SSJ4 and SSJ5.

Sensors to measure precipitating auroral particles have been included on the DMSP spacecraft since 1974 (DMSP 5C/F1). The transmission characteristics of cylindrical and spherical Electrostatic Analyzers (ESAs) were described in the early pioneering work by Paolini and Theodoridis [1967] and Theodoridis and Paolini [1968]. Early applications of ESAs to space research were discussed by Basto et al. [1976] and, more recently, by several authors in the monograph by Pfaff et al. [1998]. The first SSJ sensors were designed to measure precipitating electrons with energies from 200 eV to 20 keV using 6 energy channels. Version SSJ3 (F2 through F5) measured electrons from 50 eV to 20 keV using 16 channels [Hardy et al, 1979]. SSJ3 measurements from F2 and F4 were used to develop the “Hardy” auroral electron precipitation model [Hardy et al., 1985] and the “equatorial auroral boundary” selection criteria of Gussenhoven et al. [1983]. Version SSJ4 (F6 through F15) broadened the energy range to be 30 eV to 30 keV using 20 channels (two overlapping channels at ~1 keV), and observed both precipitating electrons and ions [Hardy et al, 1984; Schumaker et al, 1988]. SSJ4 measurements extended auroral models to include ion precipitation [Hardy et al, 1989; Gussenhoven et al. 1987]. SSJ3 and SSJ4 sensors used a cylindrical ESA geometry with two ESA detectors per energy range and species. These sensors had relatively narrow physical fields of view (FOV) (<12°) that were aligned in a radially outward, zenith direction and pointed within the atmospheric loss cone at auroral latitudes (Figure 1). For laboratory measured energy dependent geometric factors and angular responses for select DMSP SSJ4-equipped spacecraft see Hardy et al. [1984]. The current generation SSJ5 sensor (F16 and beyond) was designed as a single triquadraspheric ESA with a spherical geometry that provides six nonoverlapping in-plane look directions (each 4° (cross-plane) × 15° (in-plane)) for a total physical fan-shaped FOV of 4° × 90° (Figure 1) [Johnstone et al., 1987; Coates et al., 1984; Hardy et al., 1993; Hardy et al., 2008]. SSJ5 is mounted so that this 90° total FOV covers the ram direction to zenith. The energy range and energy channels for the SSJ5 were specifically designed to match the SSJ4. The added angular information from SSJ5 would have provided tremendous observing advantage over SSJ4, however, due to telemetry constraints, the standard operating configuration (mode-A) of the SSJ5 sums the counts across all 6 look directions per integration period and this has consequences for the interpretation of its measurements (e.g. as noted in Hardy et al. [2008]).

Figure 1.

SSJ4 (green) and SSJ5 (dark grey) viewing geometry with a nominal geomagnetic field and particle atmospheric loss cone (light-grey) for ascending (a) and descending (b) orbit phases.

Figure 1 depicts the rough detector look directions for SSJ4 (F6 – F15) (green ~12° FOV) and SSJ5 (F16 – F18) (dark-grey 90° fan FOV) with respect to a nominal background geomagnetic field (“B”) and approximate atmospheric particle loss cone (light-grey triangles) for the ascending phase (left) and descending phase (right) of an orbit. Clearly, given the mode-A configuration, SSJ5 is effectively not observing the same particle populations during ascending and descending orbit phases (compare Figure 1 (a)(i–iii) with (b) (i–iii)). Care must be exercised when interpreting SSJ5 or when inter-comparing SSJ4 and SSJ5 observations.

3. Dataset

The new data set includes original counts, estimated background counts (from penetrating radiation), differential energy fluxes, integrated energy fluxes, characteristic energy, uncertainty estimates for electrons and ions and the spacecraft ephemeris in multiple frames. Section 3.1. describes the ephemeris calculations, section 3.2. describes the calculations used to arrive at particle fluxes and section 3.3. the known issues to date.

3.1. Ephemeris

The spacecraft ephemeris is provided in three coordinate frames: Earth Centered Inertial (ECI) True of Date (TOD) Epoch, geographic (GEO), and Altitude Adjusted Corrected Geo Magnetic (AACGM) on the same cadence and precise timestamps of the 1-second SSJ measurements. Here we describe the processing details arriving at these ephemerides. The expected accuracy in the ECI and GEO frames is on the order of a few kilometers [Hartman, 1993; Vallado et al., 2006]. The companion paper Kilcommons et al. [this issue, 2016] introduces the additional coordinate frames: the Magnetic Apex system [Richmond 1995] and a minimum variance based current sheet aligned frame [Sonnerup and Scheible, 1998].

The DMSP spacecraft estimate their location on board, and telemetry ground processing bundles these 1-minute estimates within instrumental data files which are ultimately conveyed to NCEI via Boston College as daily files. These estimated ephemerides are sufficient for operational purposes but often disagree with ephemerides computed in retrospect such as by the North American Aerospace Defense Command (NORAD). The disagreement should be worse for flight models without on board Global Positioning System (GPS) systems (e.g. F14 and earlier). Interpretation of the vector magnetic measurements is the most sensitive of the environmental measurements to spacecraft location inaccuracies [e.g. Alken et al., 2014; Knipp et al., 2014].

We developed two tools to compute more accurate ephemerides, one based on propagating Two Line Elements (TLE) using the Simplified General Perturbations (SGP) theory (Vallado et al. 2006) and the other based on interpolating ECI_TOD estimates from the NASA Space Physics Data Facility (SPDF). While, for specific applications, data users may request ephemeris estimates from NCEI using the SGP approach, our standard processing uses the latter approach which we will now describe in greater detail. Two common systems used in practice to specify a coordinate in the ECI frame TOD and the “mean equator and equinox of 2000” (J2000) [Tapley et al., 2004, pp. 31, 74]. The TOD system identifies the true equator and equinox for the date of the specified coordinate, while the J2000 system is fixed to the date January 1, 2000, 12:00UT. Thus the TOD equinox is time-varying in space while the J2000 equinox is fixed. We use the TOD system herein labeled as ECI_TOD. We gather 1-minute ECI_TOD estimates from the SPDF’s Spacecraft Locator tool (http://sscweb.gsfc.nasa.gov), and use an 8th order interpolation [Burden et al., 1993] to arrive at the spacecraft location in ECI_TOD on the timestamps of environmental measurements. A one day comparison between 1-second ECI_TOD (using SGP) and 1-minute ECI_TOD interpolated onto the second (86,400 points) yielded a root mean squared error less than 6 meters per axis, showing this interpolation scheme to be numerically sufficient.

The Interactive Data Language (IDL) Astronomy User’s Library (IDLAstro) “eci2geo” routine is then used to rotate ECI_TOD locations to the Geographic (GEO) frame (http://idlastro.gsfc.nasa.gov/). Testing revealed that this implementation is most consistent with expecting ECI in the TOD system (private communication P. Saint Hilaire, 2014). We use the Super Dual Auroral Radar Network (SuperDARN) IDL Altitude Adjusted Corrected Geomagnetic Model (AACGM) library (“analysis aacgm 3.1” distributed by the Johns Hopkins University/Applied Physics Laboratory, magnetic coefficients through the 2010 epoch) to transform GEO to AACGM latitude, longitude and Magnetic Local Time (MLT). It’s important to ensure various calculators are using the same geocentric radius or to adjust accordingly. IDLAstro “eci2geo” uses a radius of 6378.137 (earth approximate equatorial radius) while AACGM uses 6371.2 (Earth mean radius). So, we make the minor adjustment before rotating GEO to AACGM. Since this version of AACGM does not use time varying magnetic field coefficients, we approximate the time dependence by linearly interpolating the AACGM latitude, longitude and MLT estimates computed at the two nearest 5-year epochs onto the instrument timestamp. This is a minor adjustment. At the time of the construction of the new SSJ dataset, the latest AACGM magnetic coefficients available were from the 2010 epoch, so dates after 2010 are not interpolated. A future version of the dataset will incorporate new AACGM coefficients and updated versions of the software library as available. In summary, ephemeris parameters provided in the new repository include ECI_TOD in cartesian (km); geographic latitude, longitude, and geocentric radius (km); and AACGM latitude, longitude and local time.

3.2. Calibrated Fluxes and Uncertainty Estimates

The theoretical basis for computing particle fluxes from counts is described in Hardy et al. [2008] and herein. In this section, we detail the adjustment of the original observed counts (O) by an estimated background (C), and the application of time varying calibration factors to differential energy fluxes (je), integrated energy flux (JE), characteristic energy (Eavg) and uncertainty estimates (σ) of these quantities for electrons and ions.

The observed count in the i-th channel is compressed onboard then telemetered to the ground and this value is denoted as O_i. The observed counts are also contaminated by penetrating protons and electrons. Since the SSJ instrument lacks a channel dedicated to measuring the penetrating particle flux, we use a forward-backward variant of the AFRL algorithm to estimate the background. For each observation, we choose the background (B_i) to be the largest of the forward and reverse estimates. The corrected count is given by C_i = |O_i − B_i|. Note that the adjusted count C_i is forced to be ≥ 0 to be consistent with historical uses and other techniques (e.g. JHU/APL’s version of the DMSP adjusted fluxes). A version which allows the adjusted count to float about 0 is available on request. Since original counts (O_i) are provided in the public data set, the end user is free to develop their own background adjustment as they see fit. While O_i is integer valued, B_i and C_i are real valued. Redmon et al. [2010] (their Figure 1) shows original counts and the result from estimating the penetrating particle background to remove proton contamination in the South Atlantic Anomaly and central plasma sheet caused by electron contamination near the sub-auroral horns of the radiation belts.

The following uncertainty measures are estimates and covariances have been assumed to be negligible. Where comparable, the equations for uncertainty developed below are agreeable to Bevington and Robinson [1992]. It is assumed that the counts O_i and B_i are both Poisson distributed and independent. These assumptions are not completely true because: 1) O_i was compressed before telemetering, 2) B_i is estimated from O_i due to a lack of a dedicated background channel, and 3) the adjusted count C_i is not allowed to be less than 0.

Note that the sum of two independent Poisson random variables is also Poisson [Lehmann, 1986]. Considering Poisson counting statistics and telemetry compression the relative 1-sigma uncertainty associated with the measurement of the true count C_i is:

• $$\frac{{\sigma }_{C_i}}{C_i}=\frac{\sqrt{{\sigma }_{O_i}^2+{\sigma }_{B_i}^2+{\sigma }_{\mathit{\text{Compression}}}^2}}{c_i},$$ Equation 1

• σ_Oi, σ_Bi: Poisson uncertainties, $\sqrt{O_i}$, and $\sqrt{B_i}$

• σ_{Compression(Oi)}: Telemetry compression, determined numerically.

The uncertainty in a count is dominated by Poisson uncertainty (0–100%) and telemetry compression plays only a minor role (0–2%) when the count is less than 3000–4000. Compression dominates the counting error at higher count levels. The interplay is demonstrated in Figure 2. Clearly, the relative uncertainty is undefined for a 0 count event, approaching infinity as the count approaches zero, and the standard Poisson distribution is defined for integral mean counts greater than 1. Thus, we set the absolute and relative uncertainties equal to “undefined” (or IEEE NaN) for counts less than 1. Subsequent mathematical operations ignore NaN values.

Figure 2.

Relative uncertainty due to telemetry compression (top) and Poisson counting (bottom) as a function of the total count in an integration step.

Differential electron and ion energy fluxes are calculated from instrument counts. From Hardy et al. [2008] for electrons, we have:

• $$j_E(E_i,\Omega )=E_i{j}_N(E_i,\Omega )=\frac{C_i·E_i}{{\eta }_{\mathit{\text{orbit}}}(E_i)·{\eta }_{\mathit{\text{ground}}}(E_i)·{\mathit{\text{GF}}}_{\mathit{\text{ground}},i}·\Delta t·\Delta E_i}=\frac{C_i·E_i}{G{F}_i}$$ Equation 2
• $$\mathit{\text{units}}:\frac{\mathit{\text{eV}}}{{\mathit{\text{cm}}}^2·s·\mathit{\text{ster}}·\Delta \mathit{\text{eV}}}$$

where,

• i: channel index, ordered from high to low (30 keV down to 30 eV)

• E_i (eV): channel central energy for channel i

• Ω: angle

• C_i: counts detected for channel i (contaminated by penetrating particles)

• GF_i (cm²sr): effective geometric factor of the sensor (see Equation 3)

• η(E_i): efficiency of channel i

• Δt: dwell time (0.098 seconds through F15 (J4) and 0.05 starting with F16 (J5))

• ΔE_i: effective width (eV) of channel i, Full Width Half Max (FWHM)

Note that Hardy et al. [2008] uses energy units of keV while we use eV herein. The channel central energies (E_i) are only nominal. For laboratory-measured energy-dependent geometric factors and angular responses for select DMSP SSJ4 equipped spacecraft see Hardy et al. [1984].

As noted above, the dwell or integration time per energy is nominally 0.098 seconds for SSJ4 (up through F15) and 0.05 seconds for SSJ5 (F16 and greater). The geometric factors (GF_ground,i) and channel efficiencies η_ground(E_i) were determined before launch in the AFRL calibration chamber. As the instrument ages, its degraded efficiency is estimated by AFRL using methods such as multi-spacecraft intercomparisons and the South Atlantic Anomaly as a standard candle. These five quantities could be separately documented: η_orbit(E_i), η_ground(E_i), GF_ground,i, Δt and ΔE_i. However, the current version of the product files includes the lumped quantity for each of electrons and ions as an effective geometric factor:

• $${\mathit{\text{GF}}}_i={\eta }_{\mathit{\text{orbit}}}(E_i)·{\eta }_{\mathit{\text{ground}}}(E_i)·{\mathit{\text{GF}}}_{\mathit{\text{ground}},i}·\Delta t·\Delta E_i,i\in [1,19]$$ Equation 3
• $$\mathit{\text{units}}:{\mathit{\text{ cm}}}^2·\mathit{\text{ster}}·\mathit{\text{sec}}·\Delta \mathit{\text{eV}}$$

Assuming the uncertainty in the differential energy flux j_E(E_i, Ω) is due predominantly to independent uncertainties in the count (C_i) and the effective geometric factor (GF_i), the uncertainty can be estimated as follows. This assumes that there is no error in the channel energies; in general, they are uncertain to a few percent. Given:

• σ_Ci: Uncertainty in count, includes Poisson, and telemetry compression (see Eq. 1).

• σ_GFi: Uncertainty in effective Geometric Factor. Provided by AFRL.

Then, the relative uncertainty in the differential number and energy fluxes for channel i is:

$$\frac{{\sigma }_{j_E(E_i,\Omega )}}{j_E(E_i,\Omega )}=\sqrt{{\left(\frac{{\sigma }_{C_i}}{C_i}\right)}^2+{\left(\frac{{\sigma }_{{\mathit{\text{GF}}}_i}}{{\mathit{\text{GF}}}_i}\right)}^2}=\frac{{\sigma }_{j_N(E_i,\Omega )}}{j_N(E_i,\Omega )}$$ Equation 4

Estimated in this manner, the relative uncertainties in differential energy and number flux are identical. Practically, under significant particle flux, the calibration uncertainty (last quantity in the above equation) dominates the effective uncertainty and this quantity has been estimated by AFRL to be approximately 20% for electrons and 50% for ions.

The SSJ4 (up to F15) employs two detectors for each of electrons and ions, a high energy (949 eV – 30 keV) and a low energy (30 eV – 949 eV) detector. Since one of the overlapping 949 eV channels is sampled at the beginning of the sample period and the other is sampled at the end, their comparison could be used as a measure of the spatial and temporal variability of the aurora over the scan period. The current version of the NCEI processing ignores the highest energy of the low energy detector (949 eV). If we choose at a later date to average the overlapping 949 eV channels (10 and 11) as per:

$$j_E(E_{10,11},\Omega )=\frac{1}{2}·(j_E(E_{10},\Omega )+j_E(E_{11},\Omega ))$$ Equation 5

then the resultant uncertainty would become:

$$\frac{{\sigma }_{j_E(E_{10,11},\Omega )}}{j_E(E_{10,11},\Omega )}=\frac{1}{2}\frac{\sqrt{{\sigma }_{j_E{(E_{10},\Omega )}^2}+{\sigma }_{j_E{(E_{11},\Omega )}^2}}}{j_E(E_{10,11},\Omega )}$$ Equation 6

The total number (energy) flux is calculated in the following manner (adapted from Hardy et al. [2008]) by “integrating” differential number (energy) fluxes (Equation 2) over energy:

• $$J_{N,\mathit{\text{Total}}}(\Omega )=j_N(E_1,\Omega )(E_2-E_1)+[{\sum }_{i=2}^{18}{j}_N(E_i,\Omega )\frac{(E_{i+1}-E_{i-1})}{2}]+j_N(E_{19},\Omega )(E_{19}-E_{18})$$
• $$\mathit{\text{units}}:\frac{1}{{\mathit{\text{cm}}}^2·s·\mathit{\text{ster}}}$$ Equation 7

Replacing ‘N’ subscripts for ‘E’ subscripts yields the equation for total energy flux and the units become eV/cm² · s · ster.

The relative uncertainty in the computation of the total number flux J_N,Total(Ω) and total energy flux J_E,Total(Ω) can be estimated as follows assuming channels are uncorrelated (i.e. covariance terms are neglected):

$$\frac{{\sigma }_{J_{N,\mathit{\text{Total}}}(\Omega )}}{J_{N,\mathit{\text{Total}}}(\Omega )}\approx \sqrt{{\sum }_{i=1}^{19}{(\Delta E_i·{\sigma }_{j_N(E_i,\Omega )})}^2}/J_{N,\mathit{\text{Total}}}(\Omega )$$ Equation 8

where, ΔE_i are the energy differences (eV) in Equation 7. Replacing the ‘N’ subscripts for ‘E’ subscripts yields the same relation for total energy flux.

The characteristic energy is calculated as the ratio of the total energy flux and the total number flux [Hardy et al., 2008]:

$$E_{\mathit{\text{Avg}}}=\frac{J{E}_{\mathit{\text{Total}}}(\Omega )}{J_{\mathit{\text{Total}}}(\Omega )},\mathit{\text{units}}:\mathit{\text{eV}}$$ Equation 9

The relative uncertainty in the computation of the average energy can be estimated as:

$$\frac{{\sigma }_{E_{\mathit{\text{Avg}}}}}{E_{\mathit{\text{Avg}}}}\approx \sqrt{{\left(\frac{{\sigma }_{{\mathit{\text{JE}}}_{\mathit{\text{Total}}}(\Omega )}}{{\mathit{\text{JE}}}_{\mathit{\text{Total}}}(\Omega )}\right)}^2+{\left(\frac{{\sigma }_{J_{\mathit{\text{Total}}}(\Omega )}}{J_{\mathit{\text{Total}}}(\Omega )}\right)}^2-2\left(\frac{\sigma ({\mathit{\text{JE}}}_{\mathit{\text{Total}}},J_{\mathit{\text{Total}}})}{{\mathit{\text{JE}}}_{\mathit{\text{Total}}}(\Omega )·J_{\mathit{\text{Total}}}(\Omega )}\right)}$$ Equation 10

$$\frac{{\sigma }_{E_{\mathit{\text{Avg}}}}}{E_{\mathit{\text{Avg}}}}\le \sqrt{{\left(\frac{{\sigma }_{{\mathit{\text{JE}}}_{\mathit{\text{Total}}}(\Omega )}}{{\mathit{\text{JE}}}_{\mathit{\text{Total}}}(\Omega )}\right)}^2+{\left(\frac{{\sigma }_{J_{\mathit{\text{Total}}}(\Omega )}}{J_{\mathit{\text{Total}}}(\Omega )}\right)}^2}(\text{approx}.\text{upper bound})$$

While the uncertainties in the energy and number fluxes are not uncorrelated, for simplicity, we use the first two terms as an upper bound for the uncertainty in E_Avg. Numerically, the relative uncertainty in E_Avg is roughly 40% greater than that of the relative uncertainty in the integrated energy flux due to the relative variances in the integrated energy and number fluxes being roughly equal.

Figure 3 shows an example auroral crossing for F16 2010-01-10. Panels 2, 3, 5 and 6 from the top show the estimated uncertainties in JE, and E_Avg for electrons and ions, demonstrating that the uncertainties are smallest under significant auroral signal and increase dramatically outside the auroral zone (owing to low count Poisson uncertainty).

Figure 3.

F16 2010-01-10 first auroral crossing of the day. Top to bottom : a) background adjusted electron differential energy flux (jE) (eV/cm2-ster-ΔeV-s), b) integrated electron energy flux (JE) (eV/cm2-ster-s), c) average electron energy (E_avg) (eV); d)–f) same quantities for ions; g) AACGM Latitude and MLT (right y-axis). Uncertainty bars are shown for the integral quantities (panels b), c), e), f)) but not for the differential quantities.

3.3. Data Access

Several versions of DMSP SSJ data exist in the public domain and in this section we attempt to summarize the most utilized. Repositories include those managed by 1) NCEI, 2) CDAWeb, 3) JHU/APL, and 4) CEDAR Madrigal. The data at NCEI and CDAWeb are in the NASA Common Data Format (CDF) [Mathews and Townheed, 1995]. They are replicas of the data described in the present work and the authoritative version is held at NCEI. 1) The NCEI DMSP space environment entry point is http://www.ngdc.noaa.gov/stp/satellite/dmsp/ and includes software and documentation and details not captured here. 2) The CDAWeb entry point is http://cdaweb.gsfc.nasa.gov/istp_public/ and SSJ holdings are kept synchronized with the public NCEI holdings. We are taking an iterative approach to releasing new DMSP space environment databases to capture, correct and document defects before broader release. At present, the NCEI and CDAWeb URLs above provide access to F16–F18 for 2010–2014. Years 1982–2009 are available on request through the contact listed at the NCEI URL. These earlier data are being actively utilized (e.g. McGranaghan et al., 2015) and will be added to CDAWeb over the next year. 3) The JHU/APL currently maintains two access mechanisms at http://sd-www.jhuapl.edu/Aurora/: a file based entry point (binary encoded uncalibrated counts and decoding software (SSJ4 only)); and an interactive user interface capable of exporting calibrated data for a short time range (ASCII formatted). 4) The Madrigal entry point is http://cedar.openmadrigal.org and currently includes SSJ observations from 2010 forward with plans to include observations from 1982 to present.

3.4. Known Issues

In this section, we describe key known issues. The on-orbit degradation factors for F6 and F7 cannot be estimated prior to 1987. The lowest energy channels are generally inaccurate due to ground calibration challenges and on-orbit spacecraft charging (not captured in our error analysis). Also, the low-energy ion detectors on F13 and F15 are suspected to be less sensitive than planned and slightly higher uncertainties are expected (included in our error analysis). Furthermore, in January, 2000, F15’s low energy ion detector became insensitive. Additional elaboration will be made available in documentation at the NCEI DMSP space environment URL.

4. Applications

In this section we discuss the broad range of research topics enabled by DMSP SSJ observations including energy inputs, spacecraft charging, ionospheric conductivity, auroral forms, boundary identification and field aligned currents. Auroral precipitation constitutes an important source of the energy input to the high-latitude ionosphere and the character of the precipitating spectra is indicative of the physical process. Important summarizing reviews of the response to auroral energy inputs are provided by Thayer and Semeter [2004], Knipp et al. [2004], Moore and Khazanov [2010] and references therein.

SSJ observations have been studied for a variety of research purposes, such as to create empirical models of precipitation [Hardy et al., 1985 and 2008; Sotirelis and Newell, 2000; Newell et al., 2002, 2014; McIntosh and Anderson, 2014] and study their patterns (e.g. Newell et al. [2009]). They have been used to investigate the storm-time thermospheric response [Knipp et al., 2013], the coincidence of electromagnetic and kinetic energy inputs [Deng et al., 2015] and the effectiveness of precipitation spectra hardness on O⁺ ion upwelling [Redmon et al., 2014]; and to create new conductivity estimates (e.g. Ieda et al., 2014; McGranaghan et al., 2015; 2016a; 2016b). Anderson [2012] evaluated over 1600 surface charging events for a full solar cycle. Researchers have developed and used auroral boundaries to organize energy inputs and the ionospheric response [Gussenhoven et al., 1983; Sotirelis and Newell, 2000; Redmon et al., 2010, 2012a, 2012b, 2014; Ieda et al., 2015] and to organize field aligned currents (FACs) [Ohtani, et al., 2010; Wing et al., 2010; Kilcommons et al., this issue, 2016]. Several studies have used multiple DMSP space environment instruments to further characterize the electrodynamics of the auroral zones (e.g. Anderson [2012]; Knipp et al. [2013]; Redmon et al. [2014]; Deng et al. [2015]; Sandholt et al. [2015]). In this section, we will review a subset of these efforts, highlight findings supported by the new dataset and provide suggestions how the new SSJ dataset may spur future revelations.

While precipitation is not as directly effective at heating the whole high-latitude upper atmosphere as Joule and solar extreme ultraviolet heating (Knipp et al., [2004]), the localized altitude and local time effects can dramatically influence auroral dynamics (e.g. through enhanced ion production and subsequent increased Joule heating [Deng et al. 2013]). The development of empirical models of precipitation has a long history. Hardy et al. [1985, 2008] created the first global auroral estimates of number flux, energy flux and characteristic energy at ~850 km altitude as a function of magnetic latitude, local time and geomagnetic activity, using 600 million energetic electron spectra from SSJ4 on 9 DMSP spacecraft. They concluded that the precipitating spectra is multi-modal (i.e. no single mode distribution function provides a sufficient description). Sotirelis and Newell [2000] and Newell et al. [2002, 2014] also developed empirical electron precipitation models and implemented the detection of many important in situ auroral boundaries. Using these tools, Newell et al. [2009] partitioned the electron spectra into contributions related to diffuse, discrete due to quasi static electric fields (i.e. “inverted-v”) and discrete due to broadband Alfvenic dispersion. They calculated their relative influence and reported that the diffuse (electron and ion) aurora is the dominant population during both low and high solar wind driving. McIntosh and Anderson [2015] characterized 30 million DMSP spectra and found that 47% were best fit with Maxwellian spectra, ~32% were best fit with kappa distributions, 12% were better described as monoenergetic spectra and ~9% were better fit as broadband spectra. From these efforts, it is clear that 2-parameter representations of the precipitating spectra (i.e. total energy flux and characteristic energy) is inadequate for many purposes, and future applications can take advantage of the new comprehensive dataset described herein to further study the spectra under different space environmental states.

Precipitating charged particles enhance the ionospheric conductivity through impact ionization. Knipp et al. [2013] investigated the storm-time response of the thermosphere to a class of storms that resulted in a misforecast of the neutral density and expected satellite drag by satellite operators. Through intense solar wind pressure pulses (aka “sheath enhanced”), the particle precipitation into the auroral ionosphere far exceeded non “sheath enhanced” storms, resulting in additional nitric oxide infrared cooling and thermospheric damping. Deng et al. [2015] used 29 cusp crossings by F13 to evaluate the spatial correlation between electromagnetic and kinetic energy inputs at ~850 km, revealing a complex correlation and a spatial displacement by as much as 1° magnetic latitude. Recently, McGranaghan et al. [2015] have used 60 million DMSP particle spectra from six spacecraft years (F6, F7, F8 (1987) and F16, F17, F18 (2010)), which collectively provided optimal magnetic latitude and MLT coverage, afforded by the new dataset described herein. McGranaghan et al. [2015; 2016b] created the first ever global auroral model of ionospheric Hall and Pedersen conductance variability as empirical orthogonal functions (EOFs) in 2-D and 3-D, respectively. McGranaghan et al. [2016a] showed that the 2-D EOFs could be used in a conductivity optimal interpolation method that significantly improved the agreement between ground- and space-based observations of the ionosphere. The new dataset specifically supported these efforts.

Coordinated ground optical and DMSP space measurements during the 1990s were used to identify ionospheric signatures of magnetic reconnection in the form of Poleward Moving Auroral Forms (PMAFs) [Fasel, 1995]. PMAFs manifest as a sequence of primarily red-line (630 nm) arcs observed near magnetic noon during negative IMF Bz. They initially appeared near the equatorward boundary of the auroral zone and moved poleward, diminishing in intensity upon entry into the polar cap. Sandholt et al. [1986] discuss the phenomenon as an ionospheric manifestation of flux transfer events (FTEs) occurring at the dayside magnetopause [Russell and Elphic, 1978]. Simultaneous observations of PMAFs and SSJ particle measurements provided additional observational evidence for their association with magnetopause FTEs [Denig et al., 1993]. Specifically, the morphological association of DMSP electron and ion precipitation at 840 km altitude to magnetospheric regions [Newell and Meng 1992] indicated that PMAFs mapped to the dayside cusp/cleft regions on merged magnetic field lines which were exposed to the solar442 wind. As these merged field lines, tied to the IMF, move in an anti-sunward direction from the dayside into the polar regions the slower moving ions experience diminishing access to the ionosphere resulting in the classical ion energy dispersion signature within the associated PMAFs [Lockwood et al., 1993]. The long duration of the newly calibrated electron and ion fluxes and their uncertainty estimates would prove a valuable asset for exploring this topic further.

Spatial variations in the DMSP SSJ precipitation spectra have been extensively used to develop high-latitude boundaries. Gussenhoven et al. [1983] used F2 and F6 to create a model of the equatorward edge of the diffuse aurora as a function of MLT and Kp. Sotirelis and Newell [2000] (and references therein) described the practical application of various ion and electron boundaries and developed a technique that merges these boundaries to determine the equatorward and poleward edges of the main auroral oval and demonstrated the value in organizing and partitioning the energy inputs and estimating the total power. Ieda et al. [2015] used SSJ boundaries as a constraint to aid in studying sub-auroral conductance dependencies on solar zenith angle. Redmon et al. [2010] used SSJ spectra to develop new equatorward and poleward boundaries using a technique to isolate precipitation associated with detached polar cap arcs, further constraining the location of the in situ main oval. Their boundaries were used to organize cold plasma seed population parameters in dynamic boundary related coordinates for studies of O⁺ upwelling (Redmon et al. [2012a, 2014]) to outflowing (Redmon et al. [2012b]). Now, Kilcommons et al. [this issue, 2016] have used the precipitation uncertainty measures developed herein to improve these boundary identifications and study region 1 and region 2 field aligned currents.

5. Summary

Herein we described the creation of a new public database of precipitating auroral electrons and ions from the SSJ instrument covering 30+ years of DMSP spacecraft from F06 (launched in 1982) through F18 (launched in 2009). Past public access to these data has been largely limited to browse graphics and manual exportation of limited periods (e.g. from institutions such as JHU/APL, NOAA/NCEI) or archives of uncalibrated and contaminated instrument counts with no derived environmental parameters (e.g. no differential electron and ion fluxes) and a nonstandard binary format for storage (NOAA/NCEI). The new dataset includes calibrated differential fluxes adjusted for penetrating radiation, estimates of the total kinetic energy flux and characteristic energy, uncertainty estimates, and accurate ephemerides and is presented in a common, self-describing format. This new database is accessible at NCEI and CDAWeb.

Key points.

1. DMSP SSJ precipitating electron and ion observations upgraded.

2. Fluxes are derived from newly decontaminated count rates and calibration factors.

3. Uncertainties are estimated and ephemeris is improved.