Using long temporal reference units to assess the spatial accuracy of global satellite-derived burned area products

In recent years, the growing availability of global satellite-derived burned area (BA) products has led to the development of methods and protocols to rigorously estimate their accuracy metrics. These protocols are based on design-based inference and provide unbiased estimators of various dimensions of accuracy. Current procedures consider the spatial and temporal dimension when obtaining the independent reference data used to assess accuracy, commonly based on the Landsat imagery archive as the basic source. The protocol in which the temporal dimension is addressed in the reference data impacts the accuracy metrics. For example, the 8–16-day sampling units usually recommended in Stage 3 BA validation protocols may result in confounding of spatial and temporal classification errors. However, both errors have different implications from a user's perspective, depending on whether the spatial detection or the temporal dating are relevant. While maintaining the fundamentals of current validation protocols, this study presents a new approach based on long temporal reference units (> 48 days) to diminish the influence of temporal reporting (i.e., dating errors) on the spatial accuracy estimates. This methodology is applied to estimate the accuracy of several global BA products for the period 2017–2019, including two European BA products, the FireCCI51 and C3SBA10, and NASA's standard BA product, the MCD64A1 collection 6 (MCD64C6). Global estimates showed similar performance for the three products; BA commission errors ranged from 17.2% ± 1.1% for C3SBA10 to 19.4% ± 1.1% for FireCCI51, and BA omission errors ranged from 43.1% ± 1.9% for FireCCI51 to 49.3% ± 2.2% for MCD64C6 (± values are one standard error). The total burned area was consistently underestimated in all products. These errors are much lower relative to those obtained in recent Stage 3 validation exercises based on short temporal reference units, which estimated global commission and omission errors greater than 40% and 70%, respectively. Thus, this study demonstrates that using long reference units provides a method to address the impact of BA product dating errors on estimates of spatial accuracy metrics, particularly for those products with lower temporal resolution or for areas with greater cloud coverage. Validation methods developed in this study may contribute to improving future protocols adopted by the Committee on Earth Observation Satellites (CEOS) Land Product Validation (LPV) subgroup.