U-Net Kalman Filter (UNetKF) - An Example of Machine Learning-assisted Ensemble Data Assimilation

Aug 4, 2026

Upload by Violet

Machine learning methods are applied to weather and climate data assimilation to strengthen ensemble Kalman filtering. The work uses U-Net convolutional neural networks to predict localized ensemble error covariances required by the Ensemble Kalman Filter (EnKF). Training is performed from EnKF data assimilation experiments in a 2-layer quasi-geostrophic model, then transferred to UNetKF experiments and compared with 3DVar, En3DVar, and EnKF.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages28

Reading Time1 h 11 min

AccessFree to download

Abstract

Plain Language Summary

Introduction

Ensemble Kalman Filter and variants

Localization and inflation for finite ensembles

Machine learning integration with data assimilation

U-Net Kalman Filter (UNetKF): An Example of Machine Learning-assisted Ensemble Data Assimilation Feiyu Lu1, 2, * 1University Cooperation of Atmospheric Research 2National Oceanic and Atmospheric Administration/Geophysical Fluid Dynamics Laboratory *Previously Princeton University Corresponding author: Feiyu Lu ([Feiyu.lu@noaa.gov](Feiyu.lu@noaa.gov)) Key Points: • Machine learning methods are used to improve traditional ensemble-based data assimilation methods. • U-Net is trained to predict the localized ensemble error covariances. • UNetKF outperforms EnKF with small to moderate ensemble sizes in a 2-layer quasigeostrophic model. Abstract Machine learning techniques have seen a tremendous rise in popularity in weather and climate sciences. Data assimilation (DA), which combines observations and numerical models, has great potential to incorporate machine learning and artificial intelligence (ML/AI) techniques. In this paper, we use U-Net, a type of convolutional neutral network (CNN), to predict the localized ensemble covariances for the Ensemble Kalman Filter (EnKF) algorithm. Using a 2-layer quasi-geostrophic model, U-Nets are trained using data from EnKF DA experiments. The trained U-Nets are then used to predict the flow-dependent localized error covariance matrices in U-Net Kalman Filter (UNetKF) experiments, which are compared to traditional 3-dimensional variational (3DVar), ensemble 3DVar (En3DVar) and EnKF methods. The performance of UNetKF can match or exceed that of 3DVar, En3DVar or EnKF. We also demonstrate that trained U-Nets can be transferred to a higher-resolution model for UNetKF implementation, which again performs competitively to 3DVar and EnKF, particularly for small ensemble sizes. Plain Language Summary Data assimilation has been widely used in weather and climate applications to combine information from observations and numerical model simulations. One common type of modern data assimilation method is the ensemble Kalman filter and its variants, which have been used in both research and operations in the fields of weather and climate. This study will incorporate deep learning methods to complement and improve the ensemble Kalman filter algorithm. More specifically, we will train a convolutional neural network to predict the error statistics of a model ensemble, which will reduce the computational costs ofthe ensemble Kalman filter. This approach is tested with a quasi-geostrophic dynamic model to demonstrate its feasibility in more advanced weather and climate applications. 1. Introduction Data assimilation (DA) has been used extensively in the fields of geosciences, including meteorology and climate science, to estimate present and historical states of the earth system components and provide initial conditions for numerical weather and climate predictions. Since its introduction (Evensen, 1994), Ensemble Kalman Filter (EnKF) and its variants have been widely used in weather and climate applications (Houtekamer & Mitchell, 1998; Anderson, 2001; Hunt et al., 2007; S. Zhang et al., 2007; Houtekamer & Zhang, 2016) . Ensemble-based Kalman filters improve upon the previous Optimal Interpolation (OI) or 3-dimentional variational (3DVar) DA methods by using an ensemble of model simulations to estimate the realtime flow-dependent model uncertainty and error statistics. The wide adoption of ensemblebased Kalman filters is facilitated by its minimal changes to existing model code and its flexibility in implementation for parallel computation (Anderson, 2003; S. Zhang et al., 2007; Anderson et al., 2009) . One persistent source of error for ensemble-based Kalman filters is the sampling errors due to limited ensemble sizes, especially for computationally expensive dynamic models. Numerous schemes to mitigate this issue have been proposed and refined over the decades, such as localization and inflation methods (F. Zhang et al., 2004; Buehner & Charron, 2007; Anderson, 2007, 2009) .

FAQs

What problem does UNetKF address in ensemble data assimilation?

UNetKF targets sampling errors caused by limited ensemble sizes, which affect the accuracy of ensemble-based Kalman filtering and related uncertainty estimates.

How is U-Net used in UNetKF?

A U-Net is trained to predict localized ensemble error covariance statistics, which are then used to drive the Ensemble Kalman Filter algorithm within UNetKF.

What models and methods are used to evaluate UNetKF performance?

UNetKF is tested using a 2-layer quasi-geostrophic model and compared against traditional 3DVar, ensemble 3DVar (En3DVar), and EnKF, with results showing competitive performance for small to moderate ensemble sizes.

Aug 4, 2026

Upload by Violet