Offre en lien avec l’Action/le Réseau : – — –/Doctorants
Laboratoire/Entreprise : Laboratoire ICube, Strasbourg
Durée : 36 mois
Contact : seo@unistra.fr
Date limite de publication : 2024-06-12
Contexte :
Spatiotemporal data provides a comprehensive view of the evolution of phenomena or organic structures over time and space, with a wide range of applications in various domains, such as environmental monitoring and climate analysis, transportation and traffic management, and longitudinal analysis of disease progression. The Transformer architecture, initially conceived and demonstrated as a powerful model for natural language processing tasks, has been adapted to become a formidable tool for spatiotemporal analysis, demonstrating superior parallelization and scaling capabilities to RNNs, in part due to its attention mechanism. Graph neural networks (GNNs), used in some studies to extract spatial information, further enhance spatiotemporal analyses by exploiting their adaptability to irregularly distributed data and their flexibility in integrating various sources of information.
Sujet :
In this study, we aim to develop a novel model specifically tailored for generating spatiotemporal data. Our strategy involves leveraging diffusion probabilistic models for denoising (DDPM) while exploring other innovative theories with the aim of comprehensively incorporating spatiotemporal conditions, possibly from different modalities. DDPM has demonstrated remarkable generative capability in the text-image generation task, and possesses inherent advantages for dealing with the many-to-many distribution fitting problem, making it a superior candidate for spatiotemporal data generation. However, diffusion models typically rely on U-Net, which is not explicitly crafted for sequential data but excels in feature extraction. Although some researchers have attempted to incorporate Transformers into diffusion models, there remains a notable absence of tailor-made adaptations to meet the demands of spatiotemporal data.
Recognizing the limitations of existing model architectures, we intend to transition to a new model capable of exploiting spatial information while exploring temporal relationships in depth. At the same time, we aim to establish a reliable latent space for the representation of spatiotemporal data, with potential benefits for several downstream tasks such as anomaly detection or longitudinal analysis. Our specific aims are as follows:
1. Generative model for the generation of spatiotemporal data with spatiotemporal conditioning. The advantages of probabilistic diffusion models for denoising (DDPM) will be deployed, with advanced spatiotemporal encoding capabilities. The underlying spatiotemporal constraints will be modeled, such as time-varying tasks or stimuli given to the subject at the time of biometric data acquisition.
2. Extension to multimodality. Taking inspiration from the multi-model diffusion model, we will design a joint denoising process so as to efficiently generate multimodal data consistent across modalities. Going beyond existing works that deal with at most two modalities, we will tackle the challenging cases involving more than three modalities, where relying on paired modeling can be suboptimal.
The project will encompass several applications, including but not limited to, boat surveillance and research on neurodegenerative diseases.
Profil du candidat :
− Master student in Computer Science, Electronic & Electrical Engineering, or in Applied Mathematics
Formation et compétences requises :
− Solid programming skills: Python/Pytorch/C++
− Proficiency in Deep Learning techniques
− Good communication skills
Adresse d’emploi :
Université de Strasbourg
4 Rue Blaise Pascal
67081 Strasbourg
Document attaché : 202405120934_Generative model for spatio-temporal data.pdf