Cerfacs Enter the world of high performance ...

  Job offer & Post-Doc |  Parallel Algorithms |  Aerodynamics, High Performance Computing

Required Education : Ph.D.
Start date : 1 September 2023
Mission duration : 2 years minimum
Deadline for applications : 31 December 2023
Salary : 3300 gross/month

The European Center for Advanced Research and Training in Scientific Computing (CERFACS) was founded in 1987 in Toulouse, France, to develop a multidisciplinary expertise in numerical modeling and simulation of a wide range of scientific and engineering challenges. It hosts researchers focused on a variety of topics, including aerodynamics, combustion and propulsion, energy production from traditional and renewable sources, modeling for environment and safety, and climate modeling. The underlying mathematical, algorithmic, and computer science and engineering developments needed to sustain and advance the high-performance computing (HPC) requirements of these applications are pursued and shared between topics. These mutually beneficial interactions are CERFACS' core value to its shareholders: Airbus Group, Cnes, EDF, Météo France, Onera, Safran and Total.

CERFACS is an actor of the international HPC community. It often performs frontier computations through one-time allocations, notably via PRACE and INCITE. It also participates in European projects (via EuroHPC) and Centers of Excellence on HPC, and collaborates with many European HPC stakeholders (CEA, INRIA, JSC, BSC, LRZ). It has a special relationship with French HPC institutes such as CINES, IDRIS and TGCC, coordinated via GENCI. Its unique position helps to establish ties between the HPC community and the needs of its industrial partners. It plays a role in training students and engineers in HPC-related topics, in making academic HPC software available and low-friction for engineering applications, as well as exploring the effects of hardware and software evolutions in computing for them. This extends to future technologies and their potential to disrupt scientific computing, such as AI (associated to the ANITI institute) and quantum computing.

Context

CERFACS explores several innovative associations of physics and data-driven approaches in the context of both its “Data-Driven Modeling” and “Exascale” strategic axes. Notably, data science tools are of interest in the HELIOS academic workgroup, with a focus on approaches with a high impact potential for high-performance computational physics. In previous years, data-driven techniques that hybridize with existing CFD workflows and enable their acceleration or otherwise improvement have been very promising and a major focus of the high-fidelity simulation community. It was generally accepted that machine learning was good at producing fast approximators, but could not approach the accuracy of high-fidelity solvers on its own, and hybridization was needed. Recently however, there has been a sudden and impressive surge in the effectiveness of large, deep-learning-based surrogates for the specific domain of numerical weather prediction (NWP), with models like FourCastNet (Pathak et al. 2022), Pangu-Weather (Bi et al. 2022), and GraphCast (Lam et al. 2022). The reference NWP algorithm for this task, IFS, was progressively approached and finally beat on a number of accuracy metrics by these techniques, all while reducing the computational requirements by several orders of magnitude, a jump in performance unheard of in decades.

This evolution poses a number of questions for the CFD community as a whole. Notably, a key enabler for these innovations was the European Centre for Medium-Range Weather Forecasts (ECMWF)'s “ERA5” reanalysis archive, a 10 PB high-quality database with hourly estimates of a large number of atmospheric, land and oceanic climate variables between 1959 and the present day. There is no equivalent in general CFD of such a high quality and detailed dataset. What’s more, many parameters such as geometries and flow regimes vary widely between simulations, rendering aggregate training more difficult. But even if the application of these advances in NWP don’t straightforwardly apply to CFD, the massive benefits suggest that exploring how to adapt them is a timely and crucial endeavour, with the potential to obtain leaps in performance that computational physicists have long been seeking.

Work program

This postdoctoral research program seeks to explore the implications of these breakthroughs for general CFD beyond the field of atmospheric simulation. While the bibliography suggests that full-flow generation is achievable (and could replace high-fidelity CFD solvers entirely in the long run), it relies heavily on ERA-5, a thorough database of unmatched scale and precision in CFD. Therefore, as a first step, we propose to focus on a more limited task, namely emulating flows in limited regions. Two notable potential applications for such generators are turbulence injection (Yousif et al 2023), and near-wall flows (Dupuy et al 2023). The strategy rests on exploiting prior work which has led to gathering data from 4 different high-fidelity numerical solvers from European research centers, and focusing on a generative but limited task. If this investigation were to show that neural networks can effectively learn flow dynamics in this limited setup, this would set the stage for the progressive expansion of foundational flow models for CFD. Notably, a critical limitation of current physics-based AI techniques is the difficulty of generalizing outside the training dataset, a problem that the field of NLP has significantly mitigated by introducing foundational models. Developing foundational models for CFD is therefore a plausible and highly promising direction if learning-based models are to become relevant for high-fidelity CFD in the future, and has now been demonstrated for weather data (Nguyen et al. 2023).

Previous work at CERFACS set several important prerequisites for such an endeavour, such as aggregating a database of various CFD simulations performed with 4 different solvers (AVBP, Alya, PyFR, UniBG's DG code), and learning to train graph neural networks of the same type as those used by DeepMind (Pfaff et al. 2020). The goal is to train modern network architectures (such as multiscale MeshGraphNets, Vision Transformers, or Autoregressive Diffusion-based generators) to directly produce unsteady resolved 3D flow near walls, at lead times compatible with wall-modeled LES, as opposed to predicting only the wall shear stress ((Dupuy, Odier, and Lapeyre 2023), (Dupuy et al. 2023)), or possibly the wall heat flux (Dupuy, Odier, and Lapeyre under review). Numerous authors have shown that the logarithmic region close to the wall exhibits turbulent structures with a clear auto-similarity in both space and time, responsible for the wall shear stress ((Townsend 1980), (Marusic and Monty 2019), (Hwang 2013), (Deck et al. 2014), (Jiménez 2018), (Boxho et al. 2022) among others). It is thus expected that reconstructing this subgrid coherence would be highly beneficial for an accurate unsteady wall prediction. What's more, generating this flow region without incurring the cost of a wall-resolved LES could potentially circumvent most of the difficulties that arise in wall modeling. Indeed, correcting an insufficiently resolved flow near the wall simply by modulating wall fluxes has many limitations, which many methods seek to circumvent. This is true of hybrid LES/RANS approaches like Detached Eddy Simulation (DES, (Spalart et al. 1997)), and other techniques that delegate near-wall resolution to non-LES methods such as Thin-Boundary Layer Equations (TBLE) developed at CERFACS (Catchirayer et al. 2018).

Any developments of these innovative techniques would aim for concrete implementations in state-of-the-art High Fidelity Solvers. Building on pioneering open-source work performed at CERFACS, that led to the creation of the PhyDLL coupling library dedicated to in-solver neural network inference, the candidate would have the opportunity to test all developments against the best non-machine learning approaches from the literature on industrial-scale configurations.

Profile

For this position, we are looking for a postdoctoral fellow with a strong background in computational physics, and with some exposure to machine learning techniques. The position is open for a minimum of 2 years, with longer periods possible depending on the candidate profile and needs. This is a fixed-term contract, but in a field that is deeply strategic to CERFACS, and for which we fully expect longer term opportunities to emerge as the project progresses.

Contacts

Contacts for each laboratory are:

• Corentin Lapeyre (lapeyre@cerfacs.fr) at CERFACS
• Olivier Teste (teste@cerfacs.fr) at IRIT

Bibliography

Bi, Kaifeng, Lingxi Xie, Hengheng Zhang, Xin Chen, Xiaotao Gu, and Qi Tian. 2022. “Pangu-Weather: A 3D High-Resolution Model for Fast and Accurate Global Weather Forecast.”

Boxho, Margaux, M Rasquin, T Toulorge, G Dergham, G Winckelmans, and Koen Hillewaert. 2022. “Analysis of Space-Time Correlations to Support the Development of Wall-Modeled LES.” Flow, Turbulence and Combustion 109 (4): 1081–1109.

Catchirayer, M., J.-F. Boussuge, P. Sagaut, M. Montagnac, D. Papadogiannis, and X. Garnaud. 2018. “Extended Integral Wall-Model for Large-Eddy Simulations of Compressible Wall-Bounded Turbulent Flows.” Physics of Fluids 30 (6): 065106. https://doi.org/10.1063/1.5030859.

Deck, Sébastien, Nicolas Renard, Romain Laraufie, and Pierre-Élie Weiss. 2014. “Large-Scale Contribution to Mean Wall Shear Stress in High-Reynolds-Number Flat-Plate Boundary Layers up to 13650.” Journal of Fluid Mechanics 743: 202–48.

D. Dupuy, N. Odier, and C. Lapeyre, “Data-driven wall modeling for turbulent separated flows.” Journal of Computational Physics, Volume 487, 2023, 112173, ISSN 0021-9991, https://doi.org/10.1016/j.jcp.2023.112173.

D. Dupuy, N. Odier, and C. Lapeyre. “Using Graph Neural Networks for Wall Modelling in Compressible Anisothermal Flows.” under review.

Dupuy, D., Odier, N., Lapeyre, C., & Papadogiannis, D. (2023). Modeling the wall shear stress in large-eddy simulation using graph neural networks. Data-Centric Engineering, 4, E7. doi:10.1017/dce.2023.2

Hwang, Yongyun. 2013. “Near-Wall Turbulent Fluctuations in the Absence of Wide Outer Motions.” Journal of Fluid Mechanics 723: 264–88.Jiménez, Javier. 2018. “Coherent Structures in Wall-Bounded Turbulence.” Journal of Fluid Mechanics 842: P1.

Lam, Remi, Alvaro Sanchez-Gonzalez, Matthew Willson, Peter Wirnsberger, Meire Fortunato, Alexander Pritzel, Suman Ravuri, et al. 2022. “GraphCast: Learning Skillful Medium-Range Global Weather Forecasting.”

Marusic, Ivan, and Jason P Monty. 2019. “Attached Eddy Model of Wall Turbulence.” Annual Review of Fluid Mechanics 51: 49–74.

Nguyen, Tung, Johannes Brandstetter, Ashish Kapoor, Jayesh K. Gupta, and Aditya Grover. 2023. “ClimaX: A Foundation Model for Weather and Climate.”

Pathak, Jaideep, Shashank Subramanian, Peter Harrington, Sanjeev Raja, Ashesh Chattopadhyay, Morteza Mardani, Thorsten Kurth, et al. 2022. “FourCastNet: A Global Data-Driven High-Resolution Weather Model Using Adaptive Fourier Neural Operators.”

Pfaff, Tobias, Meire Fortunato, Alvaro Sanchez-Gonzalez, and Peter W Battaglia. 2020. “Learning Mesh-Based Simulation with Graph Networks.” arXiv Preprint arXiv:2010.03409.

Spalart, Philippe R, W-H Jou, Michael Strelets, and Steven Allmaras. 1997. “Comments on the Feasibility of LES for Wings and on the Hybrid RANS/LES Approach.” In Proceedings of the First AFOSR International Conference on DNS/LES, 1997, 137–47.Townsend, AAR. 1980. The Structure of Turbulent Shear Flow. Cambridge university press.

Yousif, M., Zhang, M., Yu, L., Vinuesa, R., & Lim, H. (2023). A transformer-based synthetic-inflow generator for spatially developing turbulent boundary layers. Journal of Fluid Mechanics, 957, A6. doi:10.1017/jfm.2022.1088