Five Things to Know About New EDF Methane Study
The Environmental Defense Fund (EDF) has released a myriad of studies on natural gas system methane emissions over the past six years that have found low leakage rates between 1.2 and 1.5 percent of production. Five such studies are featured in the following EID graphic.
So the fact that a new EDF study released today finds methane leakage rates of 2.3 percent — well above what EDF-led research has previously found and “60 percent higher than the U.S. EPA inventory estimate,” according to the report — begs the question: What changed with regard to EDF’s methodology for this study that yielded a much higher leakage estimate than its past research has shown?
Turns out, quite a lot changed, and most of the changes raise red flags regarding the study’s conclusions. Not only did the authors of the new EDF study — which includes no new measurements and instead calculates national methane emissions based on past studies — opt not to use past EDF research as a basis for their emissions calculations, it relies exclusively on five far less comprehensive facility-level studies that lacked industry participation to arrive at its conclusion of higher U.S. emissions than previously reported. In contrast, an “alternative” calculation, based partially on EDF’s past studies, that finds emissions in line with current EPA estimates is buried in the study’s supplemental data and is not even mentioned in the report.
These are just two of several key issues regarding the manner in which EDF conducted this study that appear aimed at producing the most extreme emissions estimate possible ahead of the 27th annual World Gas Conference (WGC), which begins Monday in Washington, DC. Here is a deeper look at each issue.
#1. Exclusive Use of Facility-Scale Study Data Goes Against National Academy of Sciences’ Recommendations and Likely Exaggerates Emissions
This study’s national methane emissions estimate is based entirely on downwind, facility-based studies. From the report:
“In this work we integrate the results of recent facility-scale BU studies to estimate CH4 emissions from the U.S. O/NG supply chain, and then we validate the results using the TD [top-down] studies.”
However, a recent National Academy of Sciences (NAS) report aimed at improving national methane emissions inventories recommends a more comprehensive approach combining “bottom-up” measurements — both of the component- and facility-level variety — along with “top-down” measurements:
“Coordinated, contemporaneous top-down and bottom-up measurement campaigns, conducted in a variety of source regions for anthropogenic methane emissions, are crucial for identifying knowledge gaps and prioritizing emission inventory improvements. Careful evaluation of such data for use in national methane inventories is necessary to ensure representativeness of annual average assessments.”
EDF has conducted studies combining the comprehensive top-down/bottom-up methods recommended by NAS before. Zavala-Araiza et al. is the most notable example, and that study found a methane leakage rate of just 1.5 percent. Just as notably, a recent National Energy Technology Laboratory study based on Zavala-Araiza et al. data estimates national methane emissions at 1.65 percent. That report involved several of the co-authors of this most recent EDF study that reached much different conclusions.
The new EDF report argues that using facility-level measurements exclusively is appropriate because component-based studies can “under-sample abnormal operating conditions” such as malfunctions and large leaks. But this rationale ignores flaws with facility-level measurements that can lead to overestimation of emissions. For instance, facility-level measurements can capture episodic emissions, such as liquids unloading, and inaccurately characterize them as normal emissions that would be occurring 24 hours a day, seven days a week. The latter issue can be exacerbated when researchers lack a fundamental understanding of the facilities where they are taking measurements, which brings us to the next major issue with the study.
#2. Lack of Industry Collaboration Goes Against National Academy of Sciences’ Recommendations
With regard to the ground-based, facility-level studies used as the basis for estimating national emissions in this report, the report’s supplementary information document notes:
“Sites were reported to be sampled on a quasi-random basis without advance operator knowledge.”
Not only does EDF admit that some of the studies used did not conduct truly random sampling, it admits that industry wasn’t involved in these studies on any level. This again flies in the face of recommendations made in the NAS report, which states:
“[V]erifiability is the bedrock upon which inventories should be built if they are to be widely applicable to policy needs.”
The lack of industry participation is surprising, considering EDF’s past methane research is well known to have been a collaborative effort between EDF, academia and industry, a fact EDF has frequently touted. But even more surprising is EDF’s justification for excluding industry from participating in this particular study. From the report:
“Operator cooperation is required to obtain site access for emission measurements. Operators with lower-emitting sites are plausibly more likely to cooperate in such studies, and workers are likely to be more careful to avoid errors or fix problems when measurement teams are on site or about to arrive. The potential bias due to this ‘opt-in’ study design is very challenging to determine. We therefore rely primarily on site-level, downwind measurement methods with limited or no operator forewarning to construct our BU estimate.”
Not only does EDF fail to provide a single reference to back up this claim of “potential bias” that it claims necessitated it to use the methodology highlighted above, but none of the five co-authors of this report, who were also the lead authors of past EDF methane research that was conducted in close concert with industry ,have ever publicly claimed any “bias” whatsoever. Not once.
EDF’s assertion appears to be purely speculative in nature and also appears to be an excuse to use these studies as a basis for exaggerated national emission estimates.
#3. “Alternative” Emissions Estimate That Is In Line With EPA Greenhouse Gas Inventory (And Past EDF Research) Is Not Included In Report
In the supplemental materials document for this report, EDF includes the following “alternative” national emissions estimates based on source-based reports, several of which are past EDF studies.
Source: Alvarez et al. supplementary materials
This “alternative” estimate finds the national methane leakage rate is 1.4 percent, which (not surprisingly) not only aligns with past EDF studies, but also the EPA Greenhouse Gas Inventory.
Remarkably, the data from this “alternative” estimate isn’t mentioned at all in the actual report, even though EDF notes that an extensive list of source-based studies featured in the supplemental data of the report has “dramatically improved understanding of the sources and magnitude of CH4 emissions from the industry’s operations.”
EDF also argues that its “primary” estimate — which, again, is based solely on facility-level studies — is in line with aggregate average emissions found in the following nine “top-down” studies based on emission measurements largely collected via aircraft measurements.
“When the BU estimate is developed in this manner, direct comparison of BU and TD estimates of CH4 emissions in the nine basins for which TD measurements have been reported indicates agreement between methods…”
Source: Alvarez et al. supplementary materials
But this claim is a stretch on a couple levels. First, the cumulative data from the above “top-down” studies show a national leakage rate of 1.8 percent, well below the 2.3 percent leakage rate this new EDF study claims. Though that is within the study’s .5 percent uncertainty range, top-down studies typically overestimate oil and gas methane emissions due to the fact that emissions measurements from such studies are difficult to attribute to specific sources.
In other words, it is highly implausible that “bottom-up” methane emissions estimates would be higher than “top-down” estimates.
And in fact, a recent National Oceanic Administration (NOAA) study finds that top-down studies have likely overestimated emissions by mischaracterizing episodic emissions as normal emissions. Such emissions can also be detected and mischaracterized via facility-level measurements. So it’s not surprising that this EDF study tries to discredit that NOAA study.
#4. Attempts to Discredit Study That Finds Misrepresentation of Episodic Events Can Lead to Inflated Emissions Estimates Via Daytime Bias
Another factor that can lead to facility-scale measurements overestimating actual normal emissions is the fact that such methods are conducted in the daytime and, thus, can capture emissions from episodic events — such as liquids unloading — that are conducted during the day and inaccurately extrapolate them as if they are constant. This fact was further confirmed by a recent peer-reviewed NOAA study of the Fayetteville Shale covered by EID last year.
Perhaps anticipating that 2017 study would be used to call this new EDF report’s conclusions into question, EDF attempts to discredit the NOAA study in the paper:
“[W]e consider unlikely an alternative hypothesis that systemically higher emissions during day-time sampling cause a high bias in TD methods.”“[T]here is no reason to expect daytime bias in the kinds of abnormal operating conditions that are thought to characterize high-emitting production (and gathering) sites, which operate continuously. In fact, it is plausible that abnormal emissions could actually be higher at night because they are less likely to be found and corrected in the absence of operators.”
The above claim is directly contradicted by the following, which acknowledges the validity of the NOAA Fayetteville study, but claims it isn’t relevant to other basins.
“O/NG emissions are systematically higher during daytime hours when TD and BU measurements have been made, and lower at night. This situation was reported for the Fayetteville Shale but appears to be unique because the effect is caused by manual liquids unloadings, which represent a much higher fraction of total production emissions than in any other basin.”
The fact is, events such as liquid unloadings are common in other basins and downwind measurements, such as the ones used as the basis for this EDF analysis, do tend to be higher because they are conducted during the day.
#5. Despite EDF’s Alarmist Characterizations, Natural Gas’ Climate Benefits Remain Clear
The report claims the oil and natural gas development emissions level estimated in this report combined with carbon emissions from current natural gas combustion is having the same climate impact as coal in the short term (20-year timespan):
“Indeed, our estimate of CH4 emissions across the supply chain, per unit of gas consumed, results in roughly the same radiative forcing as does the CO2 from combustion of natural gas over a 20-year time horizon (31% over 100 years). Moreover, the climate impact of 13 Tg CH4/y over a 20-year time horizon roughly equals that from the annual CO2 emissions from all U.S. coal-fired power plants operating in 2016 (31% of the impact over a 100-year time horizon).”
But as alarming as that claim might be, it is essential to note that natural gas maintains clear climate benefits over other traditional sources even at much higher leakage rates than purported by this study.
A recent hydraulic fracturing issues brief published by Washington D.C.-based environmental think tank Resources for the Future (RFF) notes:
“If more than about 4% of the natural gas produced in the United States is emitted as methane (rather than being burned), the climate benefits of gas’s displacement of coal disappears over a 20-year time frame. If the time frame is 100 years, the leakage rate would have to be more than 8% for natural gas to be a climate loser relative to coal.”
The following International Energy Agency (IEA) graphic illustrates RFF’s point, showing natural gas maintains its climate benefits even at high leakage rates and regardless of time-frame considered.
Conclusion
This EDF study spends an inordinate amount of time explaining why its conclusions are plausible rather than explaining howit reached its conclusions. And it’s clear why — once one digs into the report’s supplemental information, it’s clear that the conclusions are based on some pretty shaky assumptions and speculation that runs counter to established and/or recommended best practices for such research.
But at the end of the day, the EDF study is not only an outlier in terms of the overall body of current methane research — it’s also an outlier with regard to EDF’s collective methane research, which has consistently found leakage rates between 1.2 and 1.5 percent. In the meantime, EPA data show oil and gas methane emissions have declined 14 percent since 1990 even as oil and natural gas production have skyrocketed. Combined with the fact that increased natural gas use has helped contribute to the best air quality of the modern era and the lowest carbon emissions in 25 years, it is clear that the shale revolution has been a win-win for the economy and environment.
Connect with us on Facebook and Twitter!
Follow @EnergyNewsBlog
