Offre en lien avec l’Action/le Réseau : TIDS/– — –

Laboratoire/Entreprise : Laboratoire ICube, Strasbourg
Durée : 6 mois
Contact : seo@unistra.fr
Date limite de publication : 2025-03-31

Contexte :
Action recognition from video is highly important for assistive care robots, as it enables them to understand and respond appropriately to the needs and activities of the people they assist. Recent DL models for action recognition are moving toward more data-efficient, interpretable, and computationally optimized frameworks: The combination of transformer architectures, spatio-temporal attention, multimodal fusion, and self-supervised learning, just to mention a few. Meanwhile, the recent emergence of large-scale pre-trained vision-language models (VLMs) has demonstrated remarkable performance and transferability to different types of visual recognition tasks, thanks to their generalizable visual and textual representations. It has been confirmed by our recent study, where our developed model learns and improves visual, textual, and numerical representations of patient gait videos based on a large-scale pre-trained Vision Language Model (VLM), for several classification tasks.

Sujet :
Motivated by these recent successes, we will extend our previous developed model and the multimodal representation for a new classification task – action recognition from video. Similarly to our previous method, we will adopt the prompt learning strategy, keeping the pre-trained VLM frozen to preserve its general representation and leverage the pre-aligned multi-modal latent space the prompt’s context with learnable vectors, which is initialized with domain-specific knowledge.

Profil du candidat :
− Solid programming skills in Python/C++
− Experience in Deep Learning (Transformer, CLIP, etc.)
− Good communication skills

Formation et compétences requises :

Adresse d’emploi :
2 Rue Marie Hamm
67000 Strasbourg

Document attaché : 202411071346_Stage-ActionRecognition.pdf