A Cost-Benefit Analysis Framework for Urban Methane Monitoring

Abstract

Despite recent advances in measurement and monitoring, methane (CH₄) emissions remain challenging to resolve at subcity jurisdictions as needed to inform mitigation progress. Here we develop an Observation System Simulation Experiment (OSSE) for New York City (NYC) to test whether a small number of strategically located, fixed in situ CH₄ measurement sites, together with existing sites, can effectively and accurately monitor NYC borough-level emissions. We identify candidate sites from tall, city-owned buildings screened for rooftop suitability and rank >300 network configurations by their ability to recover monthly borough-level emissions. Results show that the existing network improves coverage of Manhattan and Brooklyn relative to having no network at all, but performance remains limited in Staten Island and the Bronx. Adding a single site in either borough improves performance, while a three-site configuration spanning Staten Island, the Bronx, and Brooklyn captures most remaining gains, with little benefit beyond three sites. We also test a hypothetical set of observations from commercial aircraft profiles and find that these can provide annual city-scale constraints, especially when few or no urban sites exist, but are less effective than fixed sites for resolving month-to-month, borough-level variability. Together, the results demonstrate that borough-scale CH₄ monitoring is feasible with a compact, purpose-built network within the context of a wider set of existing and background tower observations, and that OSSEs provide a low-cost way to design such systems before deploying instruments.

Introduction

Methane (CH₄) emissions in cities arise from a complex mix of anthropogenic and biogenic sources, including natural gas distribution systems and usage, landfills, and wastewater treatment, and vary across space and time. , Despite major advances in urban methane research, − relatively few studies have resolved methane emissions at jurisdictional scales (e.g., city, county, or neighborhood) relevant to accountability and policy decision-making. Only a limited number of studies have begun to explore this challenge explicitly.

Atmospheric CH₄ observations provide data to enable quantification of urban emissions but are not inherently designed to deliver near-real time information at administrative boundaries. Instruments at stationary locations (e.g., on buildings or towers) offer continuous temporal coverage and have supported citywide emission estimates at monthly to seasonal scales, − yet their limited spatial density, coupled with rapid mixing of emissions in the atmosphere, complicates observation-derived emissions mapping to jurisdictional boundaries. Airborne and satellite observations ,− can provide complementary spatial coverage, but remain episodic and constrained by sampling and information-content limitations that restrict their ability to resolve jurisdictional-scale emission magnitudes and variability.

Bottom-up inventories therefore remain the primary tool for urban methane accounting. These inventories combine activity data with emission factors and can be resolved to subcity spatial scales, but they often rely on generalized assumptions that may not reflect city-specific infrastructure or operations, leading to spatial patterns that are difficult to evaluate. , Recent high-resolution urban inventories have improved representation of emissions within administrative boundaries, ,, yet they remain largely static in time and lack subannual variability. Continuous atmospheric observations have begun to demonstrate potential value for characterizing temporal variability and evaluating inventory-based estimates, − but this value has not yet been established for routine, jurisdictional-scale monitoring.

For atmospheric observations to contribute meaningfully beyond existing inventory approaches, their information content at jurisdictional scales must first be demonstrated, and the sampling strategies required to achieve that information content understood. Establishing observing networks that can robustly characterize emissions within administrative boundaries is therefore a necessary, but not sufficient, step toward integrating atmospheric data into urban methane monitoring to provide policy-relevant data.

Observation System Simulation Experiments (OSSEs) provide a framework for assessing the information content and network design prior to deployment. ,− Here, we apply an OSSE framework to New York City (NYC) to evaluate whether a limited number of strategically located fixed in situ sites can constrain borough-scale CH₄ emissions at monthly resolution. We first assess more than 300 network configurations using a Bayesian scaling-factor inversion and then examine the extent to which profile observations from instrumented commercial aircraft can complement such fixed urban measurements. New York City provides a good test case due to its dense infrastructure, strong spatial gradients in emissions, and clearly defined subcity administrative boundaries. NYC has also enacted ambitious policies for greenhouse gas emissions mitigation, motivating interest in jurisdictional-scale CH₄ information in NYC. The framework presented here is generalizable to other metropolitan regions seeking to balance observational cost, logistical constraints, and jurisdictional relevance.

Materials and Methods

Synthetic Data Framework

This study employs an Observation System Simulation Experiment (OSSE) to evaluate how well different configurations of fixed in situ atmospheric monitoring sites can constrain borough-scale CH₄ emissions across NYC. The OSSE provides a controlled modeling environment in which a prescribed set of “true” emissions is used to generate synthetic atmospheric observations. These are then analyzed using a scaling-factor inversion, in which scaling factors for each borough’s emissions are optimized in a Bayesian framework. The scaling factors are optimized to minimize the mismatch between concentrations simulated from the true emissions and those modeled from the assumed, or “initial,” emissions, under varying assumptions of model and observational error.

The model domain encompasses the NYC urban area and surrounding region (Figure a, purple boundary) to capture both urban and background CH₄ influences. The regional extent is chosen to ensure that the OSSE can distinguish borough-level emission signals from broader background enhancements originating outside the city, largely from upwind regions such as New Jersey and the Marcellus natural gas production area, which contribute significantly to the background CH₄ as observed from sites in NYC. Representing these external sources explicitly allows the inversion to more accurately attribute enhancements to local (borough-scale) versus regional (background) emissions.

1.

(a) Map of the regional modeling domain around the NYC metro area (purple box). Blue circles indicate existing “base” network site locations. (b) Map of NYC boroughs with base (blue circles) and candidate (gray triangles) additional network site locations simulated in this study. The purple star indicates location of Newark Liberty International Airport (EWR), where most flights considered here originated or landed. The underlying emissions map in both panels is the M3T inventory (Figure S1 and Supplementary Text S3) used as the true emissions in the OSSE scenarios, at 0.01-degree resolution (note different color scales, both truncated, in (a) and (b)). Site codes are defined in Table S2, with borough names shown in Figure a.

Lagrangian dispersion modeling was implemented to study the surface influence, or footprint, of observations from each site given meteorology for the calendar year 2023 (details are given in Supplementary Text S2). These influence functions (footprints) were aggregated to first quantify the relative influence of emissions from the five NYC boroughs on the current, or “base” observational network. This network includes several sites operated by Columbia University in or near NYC, along with additional sites located outside NYC operated by AEM, Inc. under contract with the New York State Energy Research and Development Authority (NYSERDA). Influence functions were also modeled for a hypothetical network of ten aircraft, based on flight paths associated with departures and arrivals at airports within the modeling domain. The modeled profiles generally occurred over Newark Liberty International Airport (EWR) due to the trajectories of the United Airlines aircraft tail numbers that were modeled, which were based on the specific Boeing model (737-900ER) that is certified for flight with the NOAA gas analyzer. Lastly, influence functions were modeled for sensors on the roofs of ten candidate buildings in NYC. Further details on site locations and aircraft observations are given in Supplementary Text S1 and Tables S2 and S3, and shown in Figure .

Two emission fields were defined: the “true” emissions, used to generate the synthetic “true” observations, and the “initial” emissions. For each of the five boroughs and the surrounding region, emissions were convolved with atmospheric transport footprints to produce a time series of simulated CH₄ enhancements from each borough and from the surrounding region outside NYC at each of the existing and candidate measurement sites. Different errors were then added to the “true” observations to simulate realistic scenarios where observations themselves include some uncertainty. The resulting “initial” time series and synthetic observations were used in the inverse model to scale initial emissions fields to best match the synthetic observations (Figure ).

2.

Flowchart of OSSE modeling framework and analysis steps. The scaling factors are initialized as a vector of ones. “Initial” emissions are derived from either unmodified or modified EPA/M3T fields (Figure S1), while “true” emissions used to construct the “true” observations are always taken from the M3T inventory (Table S1). Added Gaussian and transport errors for each scenario are summarized in Table S1. The inversion optimizes posterior scaling factors and their covariance estimates. Posterior emissions for each of the boroughs are the initial emissions multiplied by the optimized posterior scaling factors. Blue and green boxes denote workflow components used for site selection analysis; blue boxes indicate steps used for evaluating the addition of aircraft observations.

The OSSE was performed for seven different scenarios, each with a different choice of “true” and “initial” emissions and errors considered, moving from the simplest to the most complex (Table S1 and Supplementary Text S3). For each scenario, 386 possible network configurations were modeled, including only the “base” existing network, or the “base” network with one, two, three, or four additional sites in NYC chosen from the candidate sites, to assess the incremental returns. Additional inversions evaluated the impact of adding the commercial aircraft observations to the best-performing network configurations. Each inversion optimized borough and regional scaling factors to best reproduce the synthetic observations from the given network, yielding posterior scaling factors and corresponding emission estimates. These posterior estimates were compared with the known “true” emissions to evaluate how effectively each network configuration recovered borough-level CH₄ fluxes and assessed by various metrics (described below in the Evaluation Metrics section) to determine the optimal selection of sites.

Evaluation Metrics

Three metrics are used to evaluate each network configuration’s performance in retrieving the true monthly CH₄ fluxes for each borough. All metrics are based on root-mean-square error (RMSE) computed over 60 values (12 months × 5 boroughs). The first metric (Flux RMSE) quantifies the absolute difference between posterior and true fluxes (emissions per unit area) for each borough and month. It reflects how accurately the total flux is retrieved across all boroughs, giving greater weight to high-emission boroughs (e.g., Manhattan). The second (Percent Flux RMSE) measures the relative error in flux retrieval, i.e., the RMSE of the fractional difference between posterior and true fluxes. This metric normalizes for magnitude, providing a clearer sense of how well each borough’s emissions are retrieved in proportion to their true value. Third, Lambda RMSE represents how accurately the inversion recovers the multiplicative adjustments (scaling factors) applied to each region’s prior emissions.

In practice, the Flux RMSE emphasizes accuracy where absolute fluxes are largest, while the Percent Flux RMSE evaluates performance on a relative basis across all boroughs. The Lambda RMSE behaves similarly to the Percent Flux RMSE because it reflects the inversion’s ability to recover regional scaling relationships rather than total flux magnitudes. Although we show all three metrics, our final choice of recommended network is based on Flux RMSE because it prioritizes absolute CH₄ emission recovery by borough, but a different metric could be used depending on the goal.

Results and Discussion

Footprint Coverage Analysis

Analysis of the influence functions (footprints) from the transport and dispersion model (Supplementary Text S2) shows the regions that most influence the observations at the network sites. The existing (base) network (Figure a) lacks coverage in Staten Island and the Bronx while covering Manhattan relatively well. The NWK, ASR, and GIM sites contribute most to this coverage. These results suggest that in order to achieve accurate borough-level emissions estimates, it is likely that Staten Island and the Bronx would most benefit from additional coverage. Footprints from commercial aircraft profiles indicate lower influence from the city than the existing network (Figure b), but when combined with the existing network, they increase the influence of all five boroughs, and especially of Staten Island (Figure c).

3.

Annual sum footprint (ppb/(nmol/m²/s)) averaged across each borough from the existing tower network (a), 10 commercial aircraft (mostly landing and departing out of nearby Newark Liberty International Airport (magenta star)) (b), and the sum of the two (c). Stationary (tower or rooftop) sites in the existing network are indicated by orange triangles.

Scaling Factor Analysis for Optimal Network

Posterior scaling factors and flux estimates were evaluated against the three RMSE metrics for each of 386 combinations of sites for each scenario considered (Table S1). The OSSE results for all scenarios show close to linear improvement in RMSE metrics after adding the first and second sites to the base network, with marginal improvement after adding a third site, and little additional improvement with the addition of a fourth site (Figure shows the results for the three most challenging scenarios). Here we analyze networks depending on the number of sites added, so that a network can be evaluated against other networks with the same cost.

4.

Root-mean-square error (RMSE) over all 12 months and 5 boroughs for each chosen metric: flux (left), percent flux (middle), and scaling factor lambda (right), as the number of sites added in a network is increased (shown for scenarios 3, 5, and 6 as indicated by the legend). On the x-axis, 0 indicates the existing (base case) network with no added stations. The value shown is the lowest RMSE for a given number of added sites over all configurations for the given metric, i.e., the combination performing best with two added sites using the flux RMSE metric may not be the same as the one performing best for the percent flux metric.

Across all evaluation metrics and OSSE scenarios 5 and 6, the highest-performing single additions to the base network are consistently located in Staten Island and the Bronx, most notably ISA (Staten Island) and BCC (Bronx), with P58 (Staten Island) and BBH (Bronx) also ranking highly depending on the metric (Table ). The footprint analysis (Figure ) confirms that these boroughs are the least well covered by the existing network. The Flux RMSE metric tends to favor a Bronx site because of its higher fluxes, while the Percent Flux and Lambda RMSE metrics prioritize Staten Island, where relative errors are largest. When two additional stations are considered, the best-performing combinations consistently pair one site in Staten Island with one in the Bronx (e.g., ISA+BCC, ISA+PSC, P58+PSC), reinforcing that these boroughs are top priorities for network expansion.

1. Network Recommendations for Each Number of Added Sites .

	1 Added Site	Flux RMSE	2 Added Sites	Flux RMSE	3 Added Sites	Flux RMSE
Best combination	BCC	77	ISA+PSC	65	ISA+PSC+KCH	54
Alt. combination	BBH	81	ISA+BCC	66	P58+PSC+KCH	54
Alt. combination	PSC	85	P58+BCC	67	ISA+PSC+WHB	56
Alt. combination	ISA	94	ISA+BBH	68	ISA+BBH+WHB	57

^aRecommendations take into account combinations that perform well for all three metrics but Flux RMSE (units nmol/m²/s) for scenario 5 is shown for each network.

With three added stations, optimal configurations extend coverage into Brooklyn (either WHB or KCH) alongside Staten Island and the Bronx. The ISA+PSC+KCH combination achieves the lowest Flux RMSE, while ISA+PSC+WHB and P58+PSC+KCH perform robustly across all metrics and scenarios. Adding a fourth site produces only marginal improvement (Figure ), indicating diminishing returns beyond three stations. Overall, an expanded three-site network encompassing Staten Island, the Bronx, and Brooklyn offers the most cost-effective configuration for constraining borough-scale CH₄ emissions, further explored in the Discussion section. We note that given a specific number of stations, often the performance of different specific combinations within the top 5 or 10 is very similar (Figure S2), so logistical considerations may influence the final choices.

Monthly flux estimates by borough were derived from the scaling-factor inversion for the best-performing network with no added sites (“base”) and with three added sites (“base + 3”) (Figure S3). In several cases, the posterior fluxes show substantial improvement over the “initial” fluxes, even with the base network alone. However, performance varies among boroughs and months, reflecting differences in site sensitivity and temporal variability in transport. Adding three sites to the base network reduces this month-to-month scatter, yielding more consistent retrievals across boroughs and closer agreement with the true emissions, particularly for Staten Island and the Bronx. These results highlight that network expansion not only lowers mean bias but also improves monthly performance in borough-level flux estimates.

We note that while errors are reduced across all boroughs as sites are added, performance (as measured by the decrease in percent flux RMSE) is different in each borough. Figure S4 shows the decrease in percent flux RMSE separately by borough, showing a step reduction in RMSE as a site is added in a given borough, but also illustrating that the reductions are unequal; for example, Brooklyn’s RMSE remains higher than for other boroughs even after the third site is added. It is likely that the error in Brooklyn persists because the prior emissions map does not reflect the spatial distribution of high emitters in the true emissions map and the network lacks sensitivity to their locations. This is one compromise with a scaling factor inversion, in that it does not have the ability to scale individual pixels or sources without scaling the entire borough.

Assessment of Aircraft-Derived Constraints

Once the top-performing site combinations (Table ) were identified for each network size, we repeated the OSSEs while adding simulated observations from ten commercial aircraft. We evaluated the Flux RMSE for each configuration with and without aircraft data and found that the benefit of aircraft observations decreases as the number of stationary sites increases (Figure a). However, in the absence of any stationary network, aircraft sampling substantially reduces RMSE relative to the initial emissions (left-most bar in Figure a), providing meaningful constraints on total NYC-scale fluxes. When averaged across both time and space (12 months × 5 boroughs), fluxes derived from aircraft observations alone are broadly consistent with true emissions, while exhibiting larger variability (Figure b). This averaging effect highlights that aircraft measurements are most reliable for constraining city-wide annual means, while their ability to resolve borough-level or month-to-month variability remains limited in NYC (we note that specific results will differ depending on the city and the airport location relative to the flux domain). Overall, aircraft data provide a valuable complement to fixed in situ networks, especially where site coverage is limited, by strengthening NYC-scale constraints and characterizing background conditions.

5.

Impact of adding observations from 10 commercial aircraft on flux RMSE across the 5 boroughs and 12 months of the year, for scenario 5. (a) For each ground-based network ranging from no network (i.e., the error in the “initial” fluxes) to the top-performing network with three added sites, we show the flux RMSE with and without the aircraft observations added. (b) Averaged fluxes for the year over all five boroughs are estimated quite well with only the aircraft observations (light blue bar) and adding the ideal network with three added sites does not improve those annual means (dark blue bar) relative to the true emissions (black). Error bars show the standard deviation of the city-wide estimate over the 12 months, indicating that month-to-month variability of aircraft emissions estimates is large (actual initial and true emissions are both constant in this scenario).

These findings suggest that commercial aircraft observations could play an important role in extending CH₄ monitoring capabilities for cities lacking established in situ networks, particularly in regions where installing and maintaining ground stations is challenging. Beyond their use for estimating total urban fluxes, aircraft profiles can also help constrain regional background concentrations and vertical mixing, two factors that strongly influence urban emission estimates but were not explicitly varied in this OSSE. , Incorporating such measurements into inversion systems may therefore reduce biases in background specification and boundary-layer representation, improving overall accuracy even in networks with limited ground coverage. Future studies should examine how best to integrate aircraft data with fixed-site observations to optimize spatial coverage, reduce systematic errors, and support scalable, operational monitoring frameworks.

Discussion

Establishing an effective urban observing system requires balancing site/measurement cost against information gain, ensuring that added investments measurably reduce uncertainty in estimated emissions. Here, we applied an OSSE to evaluate whether atmospheric methane can be sampled to support routine monitoring of monthly CH₄ fluxes at the borough scale in New York City (NYC), a technically challenging and policy-relevant goal. The analysis assessed performance when adding candidate sites to an existing base network and examined the extent to which hypothetical commercial aircraft observations could further constrain borough-scale emissions, focusing on the quality of information content from different observational network configurations at jurisdictional scales, using NYC as a test case. The framework presented here can be modified as needed to match observational goals and specific city geography for implementation in other cities.

The OSSE results provide a quantitative basis for assessing cost-benefit trade-offs. Adding two or three strategically located sites to the existing base network produced the largest reductions in borough-level RMSE (Figure ), capturing most of the available improvement while avoiding diminishing returns from additional sites. These results indicate that network design should be guided not by site density alone but by the marginal information gained from individually added sites, relative to the monitoring objective, in this case, resolving monthly borough-level fluxes. Because bottom-up inventories already provide emission estimates at the borough scales using activity data and emission factors, demonstrating observational sensitivity at these same administrative boundaries represents a necessary first step for atmospheric observations to be relevant for urban monitoring.

Commercial aircraft observations offer a complementary pathway for constraining total city-wide emissions, particularly where long-term tower infrastructure is difficult to establish locally. In this analysis, aircraft profiles strengthen background characterization and provide credible city-scale annual estimates. However, they cannot replace the role of a stationary network (such as our base case) with sites within NYC for resolving intracity, borough-level variability. When aircraft observations were combined with one or two candidate towers (but without the existing base network), performance degraded substantially relative to the performance of the existing stationary network. In these experiments, a large fraction of 8-day inversion windows exhibited highly correlated posterior scaling factors across boroughs (Figure a), indicating insufficient independent information to distinguish borough-specific signals, and flux RMSE was larger than configurations that included the existing base network (Figure b). These results demonstrate that in NYC, such commercial aircraft data can complement, but cannot substitute for, a multisite surface network when the objective is subcity-scale monitoring.

6.

(a) Fraction of 8-day estimates in a given year that met the quality criterion based on correlation of scaling factors between boroughs, as described in Supplementary Text S4 (eq 10). Network combinations in which a large fraction of 8-day estimates was filtered (light blue bars) indicate that the model was not able to distinguish emissions from the different boroughs often and did not perform well overall. (b) Network combinations ranked by flux RMSE as a fraction of total NYC flux. Light blue points correspond to combinations shown in (a) in the same colors, indicating those combinations did not constrain fluxes well (explaining their large RMSE). The RMSE of the inventory (no network) is shown in the black circle. (c) Percent error improvement, or reduction, relative to the error in the initial inventory flux (blue) along with estimated cost (orange), of an observation network (scenario 5 shown), assuming each observation site added beyond the base costs $72,000/year. This panel omits the light blue points.

The optimal balance between cost and uncertainty reduction depends on monitoring objectives. City-wide totals may be constrained with sparse networks or aircraft observations, whereas resolving borough-level variability supporting sector-specific interpretation requires denser and sustained measurements. For this analysis, we estimate the incremental cost of network expansion at $72,000 per added site per year, based on other similar existing stations and assuming installation and capital costs are amortized over a three-year period. We emphasize that this represents the marginal cost of adding sites to the existing observational system; the baseline tower network, aircraft observations, and inventory development each carry substantial costs of their own that are not quantified here. While incremental costs increase approximately linearly as sites are added, the marginal benefit declines, indicating diminishing returns beyond three additional stations. In this context, OSSEs provide a framework for assessing whether proposed observing strategies are capable of delivering jurisdiction-scale information before long-term investments are made.

We also considered whether satellite observations could provide constraints comparable to those from commercial aircraft. Prior work shows that TROPOMI currently has very limited leverage over NYC, with low sensitivities, and only ∼36 days of valid retrievals over an entire year. Given this low information content and sparse sampling, satellite-based inversions cannot presently resolve borough-scale fluxes or monthly variability. While satellite observations of CH₄ remain an important component of the broader observing system, under current capabilities they provide substantially weaker constraints on city jurisdictional boundaries than commercial aircraft and were therefore not included in this analysis, but could be considered in a future similar analysis, as satellite coverage improves.

A broader implication of this cost-benefit analysis is that the absolute costs of expanded in situ networks, while small relative to total municipal budgets, may still exceed what many cities can sustain over the multiyear time scales required to independently resolve subcity emissions. Quantifying how much uncertainty is reduced per added site is therefore crucial for maximizing return on investment at any budget level. In such cases, observational assets supported by other (e.g., state or federal) entities, such as commercial aircraft measurements or in situ sites, can complement a small number of strategically placed local sites. This hybrid approach offers a scalable pathway toward continuous methane monitoring, leveraging the strengths of each observational component to deliver sustained subcity methane monitoring.

The Supporting Information is available free of charge at https://pubs.acs.org/doi/10.1021/acsestair.6c00084.

• Supporting Texts S1–S4; Tables S1–S3; and Figures S1–S4; details of the methodology used in the analysis (PDF)

The manuscript was written through contributions of all authors. All authors have given approval to the final version of the manuscript. K.L.M. and A.K. contributed equally.

The authors declare no competing financial interest.