Cerfacs Enter the world of high performance ...

Alban FARCHI (Researcher ECMWF)

Online model error correction with neural networks – from theory to the ECMWF forecasting system

Recent studies have shown that it is possible to combine machine learning (ML) with data assimilation (DA) to reconstruct the dynamics of systems that are partially and imperfectly observed. Such approach takes advantage of the strengths of both methods.
DA is used to estimate the system state from the observations, while ML computes a surrogate model of the dynamical system based on the estimated states. The surrogate model can be defined as a hybrid combination where a physical part based on experts' prior knowledge is enhanced with a statistical part estimated by a neural network. The training of the neural network is usually done offline, once a large enough dataset of state estimates is available.
Online learning has been investigated more recently. In this case, the surrogate model is updated and potentially improved each time a new system state estimate is computed.
Although online approaches still require a large dataset to achieve good performance, they naturally fit the sequential framework of numerical forecasting in the geosciences where new observations become available over time.
Going even further, we propose to merge the DA and ML steps. This is technically achieved by estimating, at the same time, the system state and the surrogate model parameters. This new method can be seen as a new variant of weak-constraint 4D-Var, and it has been illustrated using incrementally more complex low-order models. In this presentation, we show how to apply this method to the Integrated Forecasting System used for operational numerical weather prediction at ECMWF and we illustrate the resulting improvements by assessing forecast accuracy.