A Comparison of Major Petroleum Life Cycle Models

Abstract

Many organizations have attempted to develop an accurate well-to-pump life cycle model of petroleum products in order to inform decision makers of the consequences of its use. Our paper studies five of these models, demonstrating the differences in their predictions and attempting to evaluate their data quality. Carbon dioxide well-to-pump emissions for gasoline showed a variation of 35%, and other pollutants such as ammonia and particulate matter varied up to 100%. Differences in allocation do not appear to explain differences in predictions. Effects of these deviations on well-to-wheels passenger vehicle and truck transportation life cycle models may be minimal for effects such as global warming potential (6% spread), but for respiratory effects of criteria pollutants (41% spread) and other impact categories, they can be significant. A data quality assessment of the models’ documentation revealed real differences between models in temporal and geographic representativeness, completeness, as well as transparency. Stakeholders may need to consider carefully the tradeoffs inherent when selecting a model to conduct life cycle assessments for systems that make heavy use of petroleum products.

Introduction

A petroleum model is a necessary core component of any life cycle database. They are included in the reference life cycle databases of China (CLCD), Europe (ELCD), and the United States (USLCI) as well as commercial databases such as ecoinvent and GaBi. When conducting LCAs, whether it be from the point of view of the producer, the intermediate consumer, or end consumer, petroleum-related environmental impacts need to be accounted for with a high degree of accuracy, as they are likely to influence final results. Developers must create models of the life cycle of petroleum consisting of detailed inputs and outputs to and from each stage of production; to be a comprehensive model, these life cycle stages should typically reach from cradle to grave. In the US, a number of datasets and models have been created to estimate petroleum-related impacts for related activities from petroleum extraction to combustion. US federal agencies sponsor many of these models. Understanding these models, including their assumptions, differences among one another, and limitations is important for LCA practitioners conducting product life cycle studies with activities occurring in the United States.

Refineries are complex systems that require the use of full-refinery models to accurately characterize any single process (ICCT 2011). Hundreds of petroleum refineries across the world produce gasoline and other petroleum products; many operate under unique conditions with varying technologies. These refineries draw crude oil of wide-ranging chemical makeup from different supplies across the world and then use unique operations, technologies, and optimization to convert the crude into commercial petroleum products to support local market demands. Modern refineries produce a wide mix of useful products simultaneously from a base stock of crude oil and continuously change the ratios of these products based upon current crude feedstock characteristics, refinery technological capacity, and market demand. Their internal processes are both interdependent and constantly changing to optimize the economic value of their products under these variables. In addition, refineries are not subject to uniform emission reporting because of factors such as differences in governance across producers and variable reporting standards based on emission output. Although the uncertainties caused by vast scopes and unknowns in supply chains and environmental performance are well known in LCA (Brandt et al. 2015; Weber 2012), the above aspects of petroleum refineries make creating simple and consistent models of petroleum production unfeasible.

Many groups have attempted to tackle these problems, each using their own methods to estimate the life cycle impacts of fuels. Several have produced models of the gasoline life cycle that are now used widely in both academia and industry for predicting the life cycle impacts of various projects. Given the high sensitivity of petroleum in many LCAs, it is important to understand existing models and their differences and to identify best practices for future efforts. In this study, we have chosen and compared five models of interest from five sources: (1) a model developed to support emissions modeling research done by Sengupta and others at the Environmental Protection Agency (Sengupta), (2) the Swiss Centre for Life Cycle Inventories’ ecoinvent Database (ecoinvent), (3) the National Renewable Energy Lab’s published petroleum processes in the USLCI database (USLCI), (4) the Argonne national laboratory’s greenhouse gases, Regulated Emissions, and Energy Use in Transportation model (GREET), and (5) an inventory published by the National Energy Technology Laboratory (NETL). These models were not chosen because of their characteristics, but because (with the exception of Sengupta et al. 2015) these are some of the most popular sources of petroleum impacts across both ISO-compliant LCA studies and the wider field of more general impact studies and tools in North America.

To our knowledge, these models have not been directly compared to each other in the literature and similar comparisons are rare but extant. Suh et al. (2013) tested interoperability between ecoinvent v3 and USLCI but did not delve deeply into the individual models; instead, they compared the overall structure of the databases and their internal standards. Ingwersen (2015) likewise tested interoperability between GREET, NETL, USLCI, and other databases (including one internal EPA source) in a variety of subjects, finding large differences between the databases from their compatibility with LCA software down to the basic availability of supporting documentation. Takano et al. (2014) compares five life cycle databases, one of which is ecoinvent, in their assessment of the construction of three finnish buildings constructed of different materials but to similar plans; the study found that all five databases were in relative agreement about the relative merits of each building, with disparities stemming primarily from differences in modeled technology. Miller and Theis (2006) compared GREET to two other databases for the purpose of modeling soybean production. It found that models experienced disparities of up to a factor of two in CO₂ production for Soybean farming (which was otherwise in relative agreement between models) and that this disparity could grow much larger for industrial processes the primary culprits were unexplored differences in petroleum and industrial coal boiler emission factors. Plevin (2009) compared the BESS model with GREET for corn ethanol production and found that the Greenhouse Gas (GHG) prediction divergence in the models could be nearly eliminated by matching their modeled refinery efficiencies and properly accounting for upstream emissions; it suggests that GREET is the more accurate of the two models, but neither is perfect. Another set of studies led by the National Renewable Energy Laboratory has focused on data harmonization of various energy system models (Brandão et al. 2012) where results from various life cycle inventories to determine means and variation in model results.

Given the high importance of the petroleum industry in the United States, it is important to analyze and compare these models to inform researchers and decision makers of the quality of the data that comprise these models and their expected results. The goal of this study is to assess qualitative and quantitative differences between major petroleum life cycle models with an emphasis on the refining stage. We hope to inform stakeholders of the characteristics of these models and their implications both for usage and for calculated results, allowing stakeholders to make an informed decision about which model best aligns with their goals and methodological preferences. This will help LCA practitioners and those using LCA models to better understand and communicate the assumptions of their model of choice and the uncertainty present in modeling petroleum life cycles; although uncertainty management methods are becoming more robust (Dorini et al. 2011; Rivera and Sutherland 2015), the need for proper accounting is constant. In addition, we hope to derive best practices in petroleum LCA modeling with the goal of reducing disparity between future models by way of closing data and method gaps.

Methods

Our assessment began with a review of the supporting literature of each model. Other models required communication with developers to find proper model citations. Once literature review was complete, we compared the five petroleum models using quantitative and qualitative methods.

For the quantitative analysis, we estimated the life cycle inventory results of a kilogram of ethanol-free gasoline and diesel pumped into a vehicle at a refueling station. As such, the reference flow for each model was one kilogram of finished fuel at the refueling station. This is commonly considered a “well-to-pump” analysis. The five models are different in their scope following the refinery stage: Sengupta and ecoinvent include vehicle refueling, USLCI and GREET stop after transportation to the refueling station, and NETL models a less complete set of emissions post-refinery. We have not attempted to ‘fill in the gaps’ on these differences, instead running the models ‘as is,’ as we intend to present the model results as we would anticipate users to implement them. Although there are many methods of grouping life cycle stages, for the purposes of this paper, we group the pre-combustion petroleum life cycle into five stages for description and comparison: raw material acquisition (1), raw material transport (2), refining (3), refined product transport (4), and fuel dispensing (5).

Not all LCIs report enough information to calculate all environmental impacts of interest, but they do all include inventory for GHG and criteria pollutant emissions. We first compare the models based on GHG and criteria pollutant emission totals. To supplement this, and better express the relative importance of our results in the context of life cycle impact assessment, we aggregated these results into three relevant impact categories for the models that report emissions to support them: global warming potential, human health criteria, and smog formation. Using TRACI (Bare 2011) characterization factors, these are expressed in CO₂equivalents, PM_2.5 equivalents, and O₃ equivalents, respectively. To understand the importance of accurate well-to-pump models in a well-to-wheel context, we combine them with a common fuel use process from the USLCI database as an additional example. Due to the variety of extant allocation methods and their importance to the results of their models (Finkbeiner et al. 2006), we analyze allocation for the refinery stage, the stage in which allocation is most relevant to petroleum models because it yields coproducts. Refinery allocation summary factors for crude oil (kg crude/kg refinery product) and CO₂ emissions (kg CO₂/kg refinery product) were calculated and compared for each model. Only mean values for flows in the life cycle inventory were used for the comparison, as data on ranges were not readily available for all the models—this aspect of the data is further discussed below.

Due to the complexity of the refinery models, the effect of allocation on refinery emissions as well as upstream crude oil production was evaluated through calculation of various allocation factors for refinery crude oil input and emissions per product. A qualitative assessment, primarily of the refining stage, was used to supplement the quantitative assessment of the five models. Documentation was evaluated for several aspects of data quality described in the ISO 14044 standard (Finkbeiner et al. 2006). We began performing flow- and process-level data quality assessment according to the methodology and criteria described in Edelen and Ingwersen (2016). Tables 1 and 2 contain brief definitions of the process-level and flow-level indicator categories we have chosen for our assessment. This method includes a numerical DQI, the results of which are included in Online Resource Tables 5, 6, and 7, but primarily consists of an in-depth reading of the documentation and assessment of data sources for specific qualities. Winkler and Bilitewski (2007) performed a similar assessment, but it is only briefly mentioned and criteria for grading are not expanded on.

Table 1.

Description of process-level data quality categories

Category	Description
Reliability	Indicates the generation method of the model data. Ideal data are generated by verified measurements at the source. Data generated from standard algorithms and calculations are of medium quality; unverified estimates and unknown sources are of lowest quality
Completeness	Indicates the inclusion of all relevant inputs and outputs. An ideal model contains all inputs and outputs necessary to include all upstream activities and enable impact estimation in all categories used in current impact assessment methodologies, like TRACI
Reproducibility	Indicates the extent to which an independent practitioner could repeat the methodology. An ideal model would make all data and methods understandable in such a way that the model can be reproduced by a third party using the published documentation
Data accessibility	Indicates the availability of the underlying data and methods to third party observers. An ideal model makes all data and documentation readily available to the public and clearly linked
Interoperability	Indicates how easily a model can be integrated with existing life cycle data from other sources. An ideal model makes its data available in a way that they can be readily used in LCA software and are interoperable with common life cycle databases
Precision and uncertainty	Indicates the degree to which data include ranges or simple point estimates

Table 2.

Description of flow-level data quality categories

Category	Description
Temporal representativeness	Indicates the correlation between the time the data were Collected and the time the model represents. An ideal model uses data for the same period the model is intended to represent
Geographical representativeness	Indicates the resolution and representativeness of the data as it relates to the application. Ideal data are taken from the same geographic location as its intended use and not from a different, larger, or smaller sample area
Technological relevance	Indicates the representativeness of technologies from which the model data are gathered. An ideal model for our purposes represents the current mix of production in the United States
Sampling correlation	Indicates the size of the population from which data are collected as related to the size of the relevant population. Ideally, data are from samples from every relevant site, but most often data are collected from a smaller population of relevant sites

Running the models

The Sengupta, ecoinvent, USLCI, and NETL models were run in OpenLCA 1.4.1 (GreenDelta 2015) with an LCIA method that tallied GHGs and criteria pollutant emissions by species (an inventory analysis method) and an LCIA method that included three categories from TRACI 2.1: climate change, human respiratory effects from criteria pollutants, and smog formation potential (Bare 2011). The GREET model was run in its own native software using the GREET 2014 version (Argonne 2014) because it was not possible to fully recreate the model in OpenLCA. Each of the five models were scaled so that their final output was one kg of ethanol-free fuel, and the processes used to achieve this are found in Tables 8 and 9 of Online Resource. The contributions of individual life cycle stages were captured by taking the difference of well-to-pump emission totals with stage totals. Only mean values for flows in the life cycle inventory were used for the comparison, as data on ranges were not readily available for all the models—this aspect of the data is further discussed below. In the case that a model had a process that represented more than one stage, a copy of the process was created in which inputs and outputs that represented another stage were eliminated. Sengupta required the use of a copied process to isolate raw material transport from the refinery process. Ecoinvent required the creation of a new unit process for the raw material acquisition stage by aggregating the crude inputs to the crude transport stage (stage 2). The USLCI model required modified processes to isolate both raw material acquisition and raw material transport from the refinery process as well as the use of a modified process to isolate refined product transport from the use-phase process of combustion. Acquiring the first two life cycle stages of the GREET model required us to combine multiple crude supply pathways and convert from energy content to mass using lower heating values.

For well-to-wheel context, we combine the above with common fuel use processes, the “Transport, passenger car, gasoline powered” process from the USLCI database. This provides a uniform set of use-phase combustion emissions to compare to the production phase. Since each of these models predicts the life cycle of multiple finished products, we later supplemented our analysis in Online Resource by repeating all steps to create similar life cycle characterizations of diesel fuel comparable to the original gasoline models.

Results and discussion

Quantitative comparison

Figure 1 contains comparative life cycle criteria pollutant emissions for a single kilogram of gasoline from each model. Online Resource Table 10 contains numerical output from the five models. No one model seems to predict greater or lesser emissions of all pollutants, and trends are difficult to identify. Predictions for some emissions, like carbon dioxide and nitrous oxide, seem to be relatively tight between models; for other emissions like SO _x, ammonia, particulates, and VOCs, there is a considerable spread. Differences over 50% are common and differences of 500% are not uncommon.

Fig. 1.

Comparative breakdown of criteria pollutants emitted during the well-to-pump life cycle of gasoline as reported by five major fuel models. Models show wide variance in almost all tracked emissions, with no clear trend between models across emissions

Figure 2 presents the results of the well-to-pump comparison of life cycle impact results for global warming potential, human health criteria, and smog formation. Global warming potential has a relatively tight spread with the lowest prediction only 73% of the highest prediction; the GREET model predicts the highest impact while the Sengupta model predicts the lowest. Human health criteria had a much larger spread, with the Sengupta model predicting 24% of impact than the ecoinvent model. In this case, the ecoinvent model predicted more than double the nearest competitor due to much higher particulate emissions reported in stages 1 and 3. There are particular stages in selected models such as the transport stages (2 and 4) in USLCI that differ substantially from other models and suggest that scrutiny of the underlying data is merited.

Fig. 2.

Comparative predictions of aggregate emissions in three major impact categories during the well-to-pump life cycle of gasoline as reported by five major fuel models. Models are relatively consistent in global warming potential predictions but otherwise show little trend

Figure 3 shows the impact results from a well-to-wheels perspective. Because combustion tends to dominate the fuel life cycle, it is important to keep perspective on how differences in pre-combustion models truly affect the impacts of greater fuel use models. Despite the normalizing effects of the large, uniform addition of combustion to the results, several important pollutants still show great variance. Global warming potential, post-combustion, shows only a 6% spread but human health criteria still shows a > 40% spread, and it is apparent with the ecoinvent model that more human health criteria pollutant impact comes from the well-to-pump stages than from combustion.

Fig. 3.

Comparison of predicted emissions between five major gasoline models from the well-to-wheel perspective

Table 3 contains the results of the allocation assessment. Despite showing the smallest impacts shown in Fig. 2, the Sengupta model allocated the most crude input to gasoline. The Sengupta model treats the refinery as a black box, monitoring only inputs and outputs with no tailoring for individual products. The refinery’s life cycle emissions are attributed to each refinery product based on their EIA market values.

Table 3.

Calculated allocation factors for CO₂ from five major fuel models

	Method	Sengupta	Ecoinvent	USLCI	GREET	NETL
Gasoline	kg crude/kg fuel	1.13	0.94	0.94	0.86	0.94
	kg CO2/kg fuel	0.31	0.42	0.21	0.52	0.39
Diesel	kg crude/kg fuel	1.00	0.97	0.99	1.00	0.96
	kg CO2/kg fuel	0.28	0.23	0.22	0.31	0.38

The ecoinvent processes’ allocation methods are complex and vary by information given for particular products and emissions. The ecoinvent processes breaks down the refinery into individual subprocesses of the complex petroleum refining system and allocates emissions from these subprocesses to the products and precursors formed by that subprocess. When subprocess-level data are not available, it allocates at the refinery level based on mass.

The USLCI model does not attempt to divide the refinery into subprocesses. Instead, it treats the entire refinery as a single unit and allocates the total impacts of the refinery to its products based upon their percent by mass of the total refinery output. The documentation states that energy content allocation would yield very similar results, while economic allocation would introduce too much complication from the fluctuations of supply and demand.

The GREET model attributed possibly the largest share of its life cycle emissions to gasoline products, but the documentation was not exactly clear about its allocation factors. Some GREET documentation claims that market value and energy content are statistically indistinguishable as allocation factors for refinery products, and appears to use the two terms almost interchangeably, while one of the most recent documents (Palou-Rivera et al. 2011) cites Bredeson et al. (2010) as the source of its allocation method, which uses both energy consumption and steam usage to create a hybrid energy intensity for allocation. The GREET model, like the ecoinvent model, subdivides the petroleum refinery into constituent subprocesses. It allocates emissions from these subprocesses based on the energy content of their products and on their proportion of steam use. When energy content is not available, GREET relies on market value or, in rare cases, volumetric throughput to make up the difference as an allocation factor.

The NETL model was clear in its allocation methods, but its exact gasoline allocation factor for CO₂ needed to be calculated from other results within the supporting documentation. Like the other models (except Sengupta), the NETL model includes subprocess level modeling and allocated among intermediate products. Like GREET, NETL bases allocation on both energy use in the process and hydrogen use by the product but uses volumetric throughput as a backup factor when necessary.

Qualitative comparison

Literature review

The Sengupta model was created to estimate impacts of average US petroleum refinery operations. The primary published description of the model is given in Sengupta et al. (2015) where only the products and air emissions and allocation factors are described. The full model contains a complete refinery process (with input data) as well as connections to upstream and downstream processes and was completed to support a study of gasoline and corn ethanol from “well-to-wheel” (Hawkins et al. 2012). This model is further described here and data are provided in Online Resource Tables 11, 12, and 13. For the refinery stage, the material input data and refined product output data are from the US Energy Information Administration, while the emissions data come from the EPA’s National Emissions Inventory (NEI) and the Toxic Release Inventory (TRI) a pair of databases controlled by the EPA which collect emission information from sources across the country. The model uses modified USLCI data for crude oil production but otherwise uses ecoinvent data for transport and storage processes with amounts of these processes (e.g., Transport distances) modified to reflect US circumstances.

The ecoinvent 2.2 life cycle inventory database is created and maintained by the Swiss Centre for Life Cycle Inventories. The database includes processes representing all major stages of gasoline production (Jungbluth 2007; Weidema et al. 2013). For the model representing the refinery stage, important Greenhouse Gas pollutants and mass balance data come from the IPCC 2001 report (Houghton et al. 2001), production quantities are taken from the International Energy Agency and subprocess energy usage comes from Barlow (1991). Ecoinvent uses data from Swiss and European petroleum supplies, refining technology, and production mixes.

The USLCI database contains life cycle inventory for US and North American locations. It is managed by the National Renewable Energy Laboratory (NREL), but the data are contributed by various private and public sources. The database contains processes representing the major stages of the pre-dispensing gasoline life cycle, but these processes are documented only as an aside in an appendix of a larger plastics manufacturing LCA (Franklin Associates 2011). Like other US models, data are from USEPA and Department of Energy (DOE) sources. The documentation lists four publications in particular as sources of emissions data. These documents are USEPA (2004, 2011), Energetics Incorporated (2007), and USEPA (2002)s. The model appears to draw water emissions from the TRI and air emissions from the other three sources, but the documentation is not specific about this.

Created by the Argonne National Laboratory, GREET is a modeling software designed to estimate US vehicle-related emissions. GREET can be used for both well-to-tank and well-to-wheels analyses (Argonne 2014). GREET is based on US data from a variety of sources. Supporting documentation for the current GREET model can be found in Cai et al. (2013), Elgowainy et al. (2014), Forman et al. (2014), and USEPA (2014). GREET updates its model each year based on the most recent refinery conditions in the USA, but in doing this may be susceptible to yearly fluctuations in the oil market. GREET uses EIA statistics on the inputs and outputs of refineries to generate energy efficiency statistics, but the methods and data used to calculate emission factors are unclear.

The National Energy Technology Laboratory has released its own petroleum cycle models in the form of spreadsheet-based unit processes. The NETL model is based primarily on publicly available US data. The NETL model relies heavily on data from the GaBi database for upstream inputs for all petroleum unit process. Supporting documentation for the NETL model can be found in NETL (2008). Notably the NETL model has been used as the petroleum baseline in evaluations of GHG emission reductions to determine qualification of renewable fuels under the Renewable Fuel Standards (USEPA 2015a).

Discussion of process-level data quality

Reliability

In terms of source reliability, four of five models use a combination of measurements and estimations in their core data. The Sengupta, USLCI, and NETL models all use, at least in part, the publicly available NEI and TRI emissions data, which are composed of verified, facility-reported emissions that may be based on measurements, calculations, and estimates. Ecoinvent uses reported data from various European sources as well as mass and energy balance approaches. The GREET model uses emission factors and energy balance calculations instead of emissions reporting data, making it less reliable. For this reason, we can find no reason to rank any model as significantly better than the others in terms of reliability. The exception is GREET, which we tentatively rate as slightly lower because we could not verify the source of their emission factors; a communication with a GREET developer suggested that emission factors were derived from AP-42 (USEPA 2015b), but this could not be confirmed. For the sake of this paper, GREET is assumed to derive refinery emission factors from the most recent AP-42 document, which is considered less reliable result than reported emissions.

Completeness

No model reported emissions in each TRACI category for all five stages; a summary of reporting by category and model is in Table 4. At the refining stage, the three models (Sengupta, ecoinvent, and USLCI) showed the most complete reporting. The most commonly overlooked impact categories were waste generation and land use. An array of all refining products created by each model can be found in Online Resource Table 14.

Table 4.

Completeness check of major models for use in a TRACI study with additional resource use and waste generation categories

Impact category	Sengupta	Ecoinvent	USLCI	GREET	NETL
Ozone depletion	Yesa,e	Yes	Yesa,b,d	No	No
Global warming	Yese	Yes	Yes	Yes	Yese
Acidification	Yesa,d,e	Yesa,e	Yesa	Yes	Yese
Eutrophication	Noa,d,e	Yese	Yes	No	Yesd,e
Smog formation	Yes	Yes	Yes	No	Yes
Human health criteria	Yesa,d	Yesa,e	Yesa	Yes	Yese
Human health: cancer	Yesb	Yesa	Yesb,d	No	No
Human health: noncancer	Yesb	Yesa	Yesb,d	No	No
Ecotoxicity	Yes	Yesa	Yesb,d	No	No
Fossil depletion	Yes	Yes	Yes	Yes	Yes
Land use	No	No	No	No	No
Water use	Yes	Yes	No	Yes	Yesd,e
Solid waste generation	No	No	No	No	Yesd,e
Waste water generation	No	No	No	No	Yesd,e

^aExcept in stage 1

^bExcept in stage 2

^cExcept in stage 3

^dExcept in stage 4

^eExcept in stage 5

Reproducibility

Reproducibility is an important quality in science, but there is no standard method to condense into a DQI. The USLCI model and the proposed Sengupta model are easily deconstructed and reproduced, with clearly defined interpretations and readily presented allocation methods based on plainly listed values. The NETL and ecoinvent models use a large amount of free, public data and publish relatively easy to digest material explaining their models; the usage of proprietary data to shape minutiae and the greater refinery model complexity make the models more difficult to reproduce and reduce transparency. The GREET model uses public data, but supplements it with private data from various sources; GREET’s documentation is technically dense and does not provide sufficient data to reproduce the results as the source of its emission factors and the manipulations of the original data are not clear.

Data access level

None of the models are entirely accessible with clear references or reproductions of primary data sources, algorithms and assumptions. The GREET model comes closest to being fully accessible by providing a unique software tool to view and configure many parameters in the petroleum life cycle. The Sengupta model provides much of the original data used to calculate refinery stage emissions, but the 2015 publication (Sengupta et al. 2015) does not provide full details on refinery inputs or other life cycle stages. The USLCI model is available in online, and the documentation is available upon request, but the underlying model and primary data used are not provided. The NETL model is available in an aggregated form, reducing access to the underlying data, in part because it relies on data from proprietary sources, including the GaBi database. Ecoinvent requires purchase of the data, but the documentation provides a high degree of detail on the sources of the data, though it does not provide the primary data for the most part.

Interoperability

Although interoperability has little to do with the models themselves, it may be an important aspect by which researchers choose their model. Since much of modern life cycle assessment is performed with the assistance of large databases, we find it important that a model be readily integrated with these databases and the software that use them. This allows researchers to run their model of choice in standard LCA software. In a similar manner to Ingwersen (2015), we have ranked them here in the order of most to least interoperable.

The ecoinvent and USLCI models are the most interoperable of the five. Both models are available in widely accepted and convertible formats for use with modern LCA Software. Additionally, these models interlink their gasoline processes with massive life cycle inventory databases, allowing full customizability of nearly any aspect of the studied life cycle. The NETL and Sengupta models are less interoperable than the ecoinvent and USLCI. NETL is provided only as a handful of excel files of inconsistent format potentially meant to be integrated with GaBi software and Sengupta as a publication meant to be integrated with USLCI and ecoinvent databases. Without using GaBi software to reconstruct the model, modifying the NETL model seems infeasible as it provides data at an aggregate level as a system process. The published processes currently only cover the first three of five life cycle stages in a high level of detail, while the detail level of the next two stages are much lower. GREET offers relatively little in the way of life cycle inventory, and its database is relatively small and focused entirely on the transport fuel life cycle. The user is unable to extract more advanced life cycle data such as inventory inputs and outputs, instead limited to a relatively small range of input processes and an output of criteria pollutant emissions.

Precision and uncertainty

Ecoinvent contains detailed lognormal uncertainty distributions in processes and their upstreams, allowing for rigorous monte-carlo assessments of the entire supply chain. The GREET model has some parameters with default distributions and allows users to add other parameters, specify distributions, and perform different types of stochastic simulations. The Sengupta model includes alternative estimation of refinery emissions that can be used in uncertainty assessment. The NETL and USLCI models do not appear to quantify uncertainty.

Temporal representativeness

The Sengupta model uses 2007–2009 data but includes unverifiable data of unknown date. Ecoinvent uses data from a wide range of years, going back as far as 1991, but almost all of which are older than the year 2000. USLCI’s data comes primarily from the early to mid2000s, but much of its emission data are older than that. Some of GREET’s data are taken from the most recent year possible, which currently appears to be 2011, but its emission factors are likely derived from data originally collected over 50 years ago. The NETL model’s documentation suggests the data best match the year 2005, almost all of the data sources reflect this period.

Geographical representativeness

The USLCI and NETL models have the strongest geographic representativeness, with data coming almost entirely from large-breadth surveys of the United States. The Sengupta model is similar but contains multiple instances of uncited data. The GREET model’s AP42 derived emission factors harm its geographical relevance by being derived in large part on studies from a small number of refineries around Los Angeles, a significantly smaller resolution. Ecoinvent reflects European conditions, which may greatly differ from the United States.

Technological correlation

Four models use United States data: Sengupta, USLCI, GREET, and NETL. Of these, Sengupta, USLCI, and NETL use the entire refining industry of the country, while GREET uses only incomplete samples of it. NETL, GREET, and potentially USLCI supplement their data with proprietary information, possibly to verify or enhance their models. This makes these four models very faithful to the profile of the United States petroleum production technology, with scoring differences caused primarily by the presence of unverified data. Ecoinvent uses European data. European refineries use different technologies, different crude sources, and different electric grids; they optimize different refinery products and they are subject to different regulations than US refineries.

Sampling correlation

The ecoinvent and GREET models use data from studies performed on relatively smaller sets of refineries. The Sengupta model again contains unverified data of indeterminate sample size. The NETL and USLCI models had the strongest sample sizes because of their use of EIA data.

Additional discussion

Refinery products are not single compounds but complex hydrocarbon mixtures. Gasoline, diesel, and other products are subject to federal, state, and local regulation and associated standards. Standards for the chemical composition (e.g., sulfur content) and physical properties (e.g., vapor pressure) of gasoline and diesel, in particular, have frequently changed over the last 20 years as a result of these regulations. Online Resource Table 15 summarizes US national-level fuel standards since 1998. These changes have primarily been intended to reduce combustion-related emissions of air pollutants. Implementation of these regulations have required changes in refining and blending that may also in turn affect refinery emissions. For the most part, the models reviewed here do not clearly define the characteristics of the refinery products coming from their models. Therefore, there may be differences in the “gasoline,” “diesel,” or other products being produced by these different models, and the products may not be perfectly comparable. The year of data collection provides the best indication of the characteristics of the petroleum products, as refineries are generally producing the same quality of products over the course of a given year to meet current standards.

The Sengupta refinery model takes data from the National Emission Inventory and the Toxic Release Inventory. This gives the dataset a large and comprehensive sample size from which to draw conclusions, but the production data are not reported by the same facilities, and therefore, a peer-reviewed estimation method is used to derive emission factors. Not all facilities report all pollutants, so this method assumes that the facilities that do report are representative of all facilities. This method may cause an overestimation in mandatory emissions by assuming that facilities emitting below the reporting threshold are actually emitting the industrial average, but it may cause an underestimation in voluntary emission estimates (like CO₂) by falsely assuming that facilities which voluntary disclose their (likely favorable) emissions are a representative sample. We did not discover how the other models using NEI and TRI data, USLCI and NETL, compensated for this data gap. The Sengupta model is more transparent than the USLCI and NETL models in the use of the public US data for refinery emissions and product allocation.

Ecoinvent uses a sample of over 100 refineries to generate its data, with information coming from a wide variety of sources. These data are similar to the US-specific data in that it is a wide range of data collected from many self-reporting sources using various estimation methods backed up occasionally by verifying measurement.

USLCI takes data from a variety of sources, particularly from the EPA and DOE. They take these data from a large sample size of relevant United States refineries, placing USLCI roughly on equal terms with other models in terms of United States relevance, but with slightly older data. The relative lack of model complexity and document detail is potentially problematic.

GREET is based on data collected from 43 US refineries representing roughly 70% of US refinery production; methods used to select these 43 refineries are unknown. Unlike other models, GREET does not rely on reported emissions, but instead bases its emission predictions on the energy efficiency and energy source of the process in question. GREET uses reported energy inputs and outputs, in forms ranging from purchased electricity and crude to exported fuels, to find an energy balance and, thus, an energy efficiency. GREET may devise its emission factors from public sources such as AP-42 to predict emissions based on the source and amount of energy lost, but this is unconfirmed.

The NETL model combines NEI and TRI data with proprietary data collected from private sources going back at least as far as 2002, but the data are primarily representative of the year 2005. NETL model data are overall high quality and comprehensive but embody the same uncertainties as other models in using NEI and TRI data. Supplementation with proprietary data may help increase reliability, but this is undocumented. Large differences in model prediction between Sengupta and NETL data are presumed to be caused by differences in third party data, differences in interpretation of NEI and TRI data, and in the allocation methods.

Conclusions

We intend the results to provide a LCA practitioner with important background information to better understand these models and inform choice of which model to use in an LCA model that demands US petroleum products somewhere in the supply chain, which we anticipate to be nearly all LCA models including US goods and services. Our recommendations are limited by the experience of a single user; we were unable to fully determine the linkages between differences in method and differences in result. Future exploration of these models to link differences of methodology to differences of results may be more successful with collaborative, in-depth assessments. If additional sources of model documentation can be found (or released), then the comparison process may become much more accurate. Since there is no clear methodology to compile all the findings into an objective final ranking of the models for use, we instead provide recommendations on each model below without providing an overall ranking.

The proposed Sengupta et al. (2015) refinery process is transparent in its emissions and product allocation derivations, which are derived from large, publicly available national databases. Data on some emissions have a low confidence because of the challenge of deriving emission factors on species that were not subject to mandatory reporting, like CO₂ emissions, but the emission factors are provided with variations that can be used in uncertainty calculations. The refinery process does not have the detailed subprocess descriptions of the other models, treating the entire refinery as a unit rather than the individual subunits of the refinery. Economic allocation is used in the refinery, but sufficient data are provided that another allocation method could be used. The processes representing the other stages in the model are largely derivations of existing processes from the USLCI (crude oil extraction) and ecoinvent (crude transport and refined product transport, storage and distribution) databases; the model requires these databases to function. The original unit process data are available through our Online Resource and the Sengupta et al. (2015) paper. We recommend the Sengupta processes for users who want to use more updated US-based refinery data and who have access to both ecoinvent v2 and USLCI. The refinery model could be more complete in terms of flows included, and the other life cycle stages could be tailored to use more US-specific data.

Ecoinvent has the greatest ease of use, readily inserted into existing LCA software and backed by a large and well-trusted life cycle inventory database with the highest completeness in coverage of inventory to support calculation of the most impact category indicators. The ecoinvent model, however, is based on European data and its documentation was only available in German. From what we gathered from this documentation, however, ecoinvent appears to have data as robust as the United States models, surveying a large number of refineries and collecting emission data across many sources in the literature. Because the life cycle of oil products is very different in a European setting, we recommend the use of the ecoinvent gasoline processes only to those decision makers researching European conditions and those who are capable of translating and verifying the supporting documentation for themselves.

The USLCI model relies on many of the same data sources as the Sengupta and NETL models. It is deconstructible with relative ease and sourced from public documentation but has what may be considered a lack of model complexity due to this. Like ecoinvent, it is readily integrated with a large database of LCA processes and can be plugged into new models and edited with relative ease. We recommend the model to users who need US-based data and are comfortable with the model’s methodology or who prefer to use data that can easily be complimented with the larger USLCI database.

GREET has ample data and complexity supporting its petroleum model. GREET takes data from a number of refineries representing roughly 70% of US refinery production, and its methods, relying on energy balance and emission factors rather than reported emissions, should give estimates that better satisfy mass and energy balances. GREET’s supporting documentation, however, is technically dense, disjointed, and relies on energy efficiency and engineering equations to predict outputs. GREET’s software is relatively easy to use and customize but is not compatible with projects utilizing a life cycle inventory and is less complete in its coverage of emissions. We recommend GREET to modelers who do not need to incorporate it into a standard LCA model and who need only to estimate transportation-related air emissions in the limited categories GREET offers.

NETL has one of the most robust methodologies and datasets, using the same national databases as the Sengupta and USLCI models but supplementing that with proprietary data to allow allocation that is more sophisticated. Its methods appear robust while still being relatively easy to understand. The NETL model yields very different predictions than the proposed Sengupta model despite their similar bases, and calculation details are less transparent, reducing its reproducibility. The NETL model is somewhat lacking in usability, with data expressed in excel files of varying formats and usefulnesses. We recommend NETL to decision makers seeking a US-based life cycle inventory who are prepared for the additional legwork of using the model, who do not need to model refinery products that are not transport fuels, and who do not mind the lack of customizability.

All the five models agree on the big picture: refinery emissions tend to dominate the pre-combustion life cycle while raw material extraction is still an important source of environmental impacts, but crude and refinery product transport cannot be ignored. The models do not agree, however, on the magnitude of emissions created during the gasoline pre-combustion life cycle, and the differences between them are often too large and too variable to explain with single simple parameters like allocation factors. The lack of agreement between these well-known models suggests that there are still important complexities in modeling the petroleum life cycle that the current models were not able to capture fully in their data and documentation. The small variance in CO₂ predictions within the group, however, suggests that decision makers can be relatively secure using any model for GHG related research, but they should still gravitate toward the model that best suits their situation.

All the models would be difficult to reproduce due to incompleteness or inaccuracies found in the documentation. All are missing key data source citations (especially transportation) in at least one stage of their life cycle. Petroleum is a critically important and highly complex system; researchers need to share their models in a way that can be readily understood and verified by both expert nonexpert users so that these models can be made more accurate and more complete. There are various options for proceeding to develop more harmonized, consistent models. One option would be to attempt to harmonize the results of these models to develop an average. We do not recommend the use of an average result of one or more of these models, as there is no clear indication that the variation of the results in the current models represents the variation of the actual petroleum LCA, and this further reduces transparency of the results. A more desirable solution would be for the model developers to collaborate to develop a common model that can serve multiple purposes, increasing flexibility of use, and increasing transparency. Many of the current models are developed or supported by US federal government agencies which are already beginning to collaborate through the Federal LCA Commons (USDA 2016). Collaboration with the petroleum industry and other industry groups, such as the American Chemistry Council, that have experience and interest petroleum LCA model development may increase data quality and model acceptance. Collaborative model building would help develop consensus, best practices, and reduce duplication of modeling efforts.

Regardless of how future US petroleum LCA modeling goals are pursued, we would recommend the following:

• Well-to-pump models be developed to include all major life cycle stages with transportation and distribution linking those stages.

• Models be developed as a series of unit processes consistent with LCA standards and best practices, including ISO 14044 (Finkbeiner et al. 2006) and the Global Guidance Principles for LCA databases (UNEP 2011).

• Processes be complete as possible, as understood by process completeness defined above. This includes the need for all emissions to water, land, and air in each life cycle stage to be included in the inventory and for grouped emissions (e.g., VOCs) to be speciated as much as possible to enable better impact characterization.

• Primary data be used as much as possible to represent emissions.

• Refinery models be developed to account for all refinery products, not just transportation fuels.

• Refinery models consist of a series of unit processes representing different refinery subprocesses in order to reduce the need for refinery-level allocation. When allocation is still necessary, make allocation flexible so that the model can be used with energy, mass, or economic allocation methods.

• The standards that refinery products meet be described in the metadata for the refinery processes.

• The model be put through a transparent review process by experts.

• Documentation be provided for the model as a whole and metadata be available for each unit process.

• The model be made available in an interoperable, standardized LCA format.