Offre en lien avec l’Action/le Réseau : – — –/– — –

Laboratoire/Entreprise : Sorbonne Universite – Institut des Systèmes Intell
Durée : 36 mois
Contact : patrick.gallinari@sorbonne-universite.fr
Date limite de publication : 2024-12-30

Contexte :
Physics-aware deep learning aims at investigating the potential of AI methods to advance scientific research for the modeling of complex natural phenomena. This is a fast-growing research topic with the potential to boost scientific progress and to change the way we develop research in a whole range of scientific domains. An area where this idea raises high hopes is the modeling of complex dynamics characterizing natural phenomena occurring in domains as diverse as climate science, earth science, biology, fluid dynamics. A diversity of approaches is being developed including data-driven techniques, methods that leverage first principles (physics) prior knowledge coupled with machine learning, neural solvers that directly solve differential equations. Despite significant advances, this remains an emerging topic that raises several open problems in machine learning and application domains. Among all the exploratory research directions, the idea of developing foundation models for learning from multiple physics is emerging as one of the fundamental challenges in this field. This PhD proposal is aimed at exploring different aspects of this new challenging topic.

Sujet :
Foundation models have become prominent in domains like natural language processing (GPT, Llama, Mistral, etc) or vision (CLIP, DALL-E, Flamingo, etc). Trained with large quantities of data using self-supervision, they may be used or adapted for downstream tasks while benefiting through pre-training from large amounts of training data. Initial attempts at replicating this framework in scientific domains is currently being investigated in fields as diverse as protein, molecule, weather forecasting. Is the paradigm of foundation models adaptable to more general physics modeling such as the complex behavior of dynamical systems? Large initiatives are emerging on this fundamental topic (http://micde.umich.edu/SciFM24, https://iaifi.org/generative-ai-workshop). This high stake, high gain setting might be the next big move in the domain of data-driven spatio-temporal dynamics modeling. The objective of the PhD is to explore different directions pertaining to the topic of foundation models for physics, focused on the modeling of dynamical systems.

**Solving parametric PDEs

A first step is to consider solving parametric partial differential equations (PDEs), i.e. PDEs from one family with varying parameters including initial and boundary conditions, forcing functions, or coefficients. Current neural solvers operate either on fixed conditions or on a small range of parameters with training performed on a sample of the parameters. A first direction will be to analyze the potential of representative NN solvers to interpolate and extrapolate out of distribution to a large range of conditions when learning parametric solutions. A key issue is then the development of training techniques allowing for fast adaptation on new dynamics.

**Tackling multiple physics

The foundation approach is particularly interesting in the case of scarce data, provided physics primitive could be learned from related but different PDE dynamics that are available in large quantities and then transferred to the case of interest. Learning from multiple PDEs raises algorithmic challenges since they operate on domains with different space and time resolutions, shapes and number of channels. We will consider an Encode-Process-Decode framework so that the commonalities between the dynamics are encoded and modeled in a shared latent space and the encoding-decoding process allows to project from and to the observation space for each PDE. This framework will be evaluated with selected backbones.

**Generalization and few shot capabilities

Generalization to new dynamics is the core problem motivating the development of foundation models in science. This is a key issue for the adoption of data-driven methods in physics and more generally in any context were the data is scarce. We will consider the general framework of few shot learning aiming at fine tuning pre-trained models for downstream tasks. In this context the objective will be to develop frameworks for the fast adaptation of foundation models to target tasks. Different strategies will be analyzed and developed including parameters sampling, meta-learning for adaptation and strategies inspired from the developments in semantics and language applications like in-context.

Profil du candidat :
Computer science or applied mathematics. Good programming skills.

Formation et compétences requises :
Master degree in computer science or applied mathematics, Engineering school. Background and experience in machine learning.

Adresse d’emploi :
Sorbonne Université (S.U.), Pierre et Marie Campus in the center of Paris. The candidate will integrate the MLIA team (Machine Learning and Deep Learning for Information Access) at ISIR (Institut des Systèmes Intelligents et de Robotique).

Document attaché : 202404261431_2024-04-20-PhD-Description-Foundation-models-Physics.pdf