LIPS

Language Interaction Speech and Signe (LIPS)

The LIPS team, made up of researchers in linguistics and language processing, conducts multidisciplinary research into oral -spoken and signed- languages. It cooperates extensively with other teams in the STL department, as well as with other departments in the laboratory.

The team’s scientific challenges concern oral, spoken and signed, languages, with the aim of linguistic description and modelling. The team brings together researchers in natural language processing and linguists to focus on the situated dimension of language: we use a variety of data, of different sizes and from different sources, illustrating linguistic variation in all its dimensions, from minimal units to meaning. Multimodal processing involving the written and aural variety of spoken languages as well as other visual information (e.g. occulometry), or owritten and aural varieties of different languages (e.g. sign language videos with French subtitles), is also at the heart of our concerns. Our work gives rise to a variety of applications: speech and sign language recognition and synthesis, dialogue systems. Our research is interdisciplinary by nature and requires skills in signal processing, linguistics and computer science. Our research is interdisciplinary by nature and requires skills in signal processing, linguistics and computer science.

The team’s activities are organised around three themes:

Information retrieval in dialogues

Work on multimodal and conversational information retrieval is centered around two main pillars: incorpo-
rating multimodality into information retrieval systems and studying dialogic interactions. In more detail, this
research is focused on how to represent multimodal data, taking into account contexts and various multi-
modal aspects in the developed representations, and addressing the challenge posed by the scarcity of avail-
able data. The artificial intelligence methods implemented also tackle issues related to handling degraded
data, continuous and interactive learning, while aiming to make model predictions understandable, with an
eye towards explainability.

Sign language modeling and processing

Sign languages, which are poorly endowed languages, have a linguistic system resulting from their visuo-gestural nature: a large amount of information is expressed simultaneously and organized spatially, and iconicity
plays a central role. Computer modeling of SL requires the design of representations with little
available data, and where pre-existing models, which are essentially linear, have been developed for written
or spoken languages and do not cover all aspects of LS. Through projects and PhD theses and in collaboration with signers of these languages (e.g. deaf translators and journalists), we are tackling the following research question: How can SL be analysed, represented and processed? How can we take into account the linguistic specifics linked to their visual-gestural nature (multilinearity, spatialization, iconicity)? What types of approach are possible with little LSF data? Current projects are detailed on this page.

Speech processing and multilingual variation modeling

Research in this theme aims to understand the variation phenomena that underlie temporal and spatial
changes in spoken language and to develop models for use in automatic speech processing. One of our objectives is to structure the information in audio documents by developing models and algorithms
that rely on diverse information sources and can serve to detect the presence of speech, to identify the lan-
guage being spoken and to characterize the speaker(s), to transcribe the speech into text in the same or a
different language or identify specific entities or acoustic events. Concerning speech recognition, our research aims to complete the word sequence with punctuation and with paralinguistic information such as hesitations, laughter or breath noises. We also study frugal learning techniques and applied them to speech recognition for low e-resourced languages and tasks.

News

All News Articles

Coordination

Team members

Publications

  • Pré-publication, Document de travail

    Mathilde Aguiar, Pierre Zweigenbaum, Nona Naderi. Am I eligible? Natural Language Inference for Clinical Trial Patient Recruitment: the Patient’s Point of View. 2025. ⟨hal-04992084⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Chapitre d'ouvrage

    Mathieu Constant, Marie Candito, Yannick Parmentier, Carlos Ramisch, Agata Savary. Construction, exploitation et exploration de ressources linguistiques pour le traitement automatique des expressions polylexicales en français : le projet PARSEME-FR. Lidia Becker; Julia Kuhn; Christina Ossenkop; Claudia Polzin-Haumann; Elton Prifti. Digitale romanistische Sprachwissenschaft: Stand und Perspektiven, Narr Francke Attempto Verlag GmbH + Co. KG, pp.219-250, 2023, Romanistisches Kolloquium, 978-3-8233-8506-6. ⟨hal-04995189⟩

    ILES, STL

    Year of publication

    HAL publication

  • Thèse

    Rémi Uro. Détection et caractérisation des interruptions dans les interactions orales pour la description du comportement des femmes et des hommes dans les contenus audiovisuels. Informatique et langage [cs.CL]. Université Paris-Saclay, 2024. Français. ⟨NNT : 2024UPASG055⟩. ⟨tel-04994439⟩

    STL, STL

    Year of publication

    Available in free access

    HAL publication

  • Communication dans un congrès

    Amel Fraisse, Patrick Paroubek, Ramit Goyal, Nassreddine Znaidi. Measuring Multilingualism in Online Public Access Catalogs. The ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL), Dec 2024, Hong Khong, China. ⟨hal-04986773⟩

    ILES, STL

    Year of publication

    HAL publication

  • Communication dans un congrès

    Manon Scholivet, Agata Savary, Louis Estève, Marie Candito, Carlos Ramisch. SELEXINI – a large and diverse automatically parsed corpus of French. Building and Using Comparable Corpora (BUCC), Jan 2025, Abu DHABI, United Arab Emirates. ⟨hal-04978746⟩

    ILES, STL

    Year of publication

    Available in free access

    HAL publication

  • Thèse

    Hui-Syuan Yeh. Prompt-based Relation Extraction for Pharmacovigilance. Computation and Language [cs.CL]. Université Paris-Saclay, 2024. English. ⟨NNT : 2024UPASG097⟩. ⟨tel-04968043⟩

    STL, STL

    Year of publication

    Available in free access

    HAL publication

  • Rapport

    Sylvain Bouveret, Aurélie Bugeau, Frenoux Emmanuelle, Julien Lefevre, Laurent Lefèvre, et al.. Quiz sur les impacts environnementaux du numérique. EcoInfo. 2025, pp.1-5. ⟨hal-04960328v2⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Thèse

    Camille Challant. Représentation formelle avec AZee et contraintes grammaticales pour la langue des signes française. Théorie et langage formel [cs.FL]. Université Paris-Saclay, 2024. Français. ⟨NNT : 2024UPASG086⟩. ⟨tel-04957486⟩

    STL, STL

    Year of publication

    Available in free access

    HAL publication

  • Article dans une revue

    Zheng Zhang, Brian Denton, Xiaolan Xie. Branch and Price for Chance-Constrained Bin Packing. INFORMS Journal on Computing, 2020, 32 (3), pp.547-564. ⟨10.1287/ijoc.2019.0894⟩. ⟨hal-04941861⟩

    ILES, STL

    Year of publication

    HAL publication

  • Communication dans un congrès

    Simon Devauchelle, David Doukhan, Lucas Ondel Yang, Benjamin Élie, Albert Rilliard. Estimation automatique de caractéristiques acoustiques pour l’étude diachronique du français oral dans les médias. Atelier DAHLIA: DigitAl Humanities and cuLtural herItAge: data and knowledge management and analysis, Claudia Marinica; Fabrice Guillet; Florent Laroche, Jan 2025, Strasbourg, France. ⟨hal-04938377⟩

    STL, STL

    Year of publication

    Available in free access

    HAL publication

  • Article dans une revue

    Rémi Uro, David Doukhan. Pendant le confinement, le temps de parole des femmes a baissé à la télévision et à la radio. La revue des médias, 2020. ⟨hal-04906221⟩

    STL, TLP

    Year of publication

    HAL publication

  • Communication dans un congrès

    Fanny Ducel, Nicolas Hiebel, Olivier Ferret, Karën Fort, Aurélie Névéol. “Women do not have heart attacks!” Gender Biases in Automatically Generated Clinical Cases in French. Annual Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics, Apr 2025, Albuquerque, United States. ⟨hal-04938811⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Article dans une revue

    Clément Bernard, Guillaume Postic, Sahar Ghannay, Fariza Tahi. RNA-TorsionBERT: leveraging language models for RNA 3D torsion angles prediction. Bioinformatics, 2025, 41 (1), pp.btaf004. ⟨10.1093/bioinformatics/btaf004⟩. ⟨hal-04911519⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Article dans une revue

    Marion Ficher, Tom Bauer, Anne-Laure Ligozat. A comprehensive review of the end-of-life modeling in LCAs of digital equipment. International Journal of Life Cycle Assessment, 2024, 30 (1), pp.20-42. ⟨10.1007/s11367-024-02367-x⟩. ⟨hal-04924691⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Thèse

    Atilla Kaan Alkan. Natural Language Processing for Analyzing Messages of Astrophysical Observations. Artificial Intelligence [cs.AI]. Université Paris-Saclay, 2024. English. ⟨NNT : 2024UPASG114⟩. ⟨tel-04928511⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Pré-publication, Document de travail

    Clément Bernard, Guillaume Postic, Sahar Ghannay, Fariza Tahi. Has AlphaFold3 achieved success for RNAs?. 2025. ⟨hal-04911522⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Thèse

    Léa-Marie Lam-Yee-Mui. Modélisations pour la reconnaissance de la parole à données contraintes. Traitement du signal et de l’image [eess.SP]. Université Paris-Saclay, 2024. Français. ⟨NNT : 2024UPASG075⟩. ⟨tel-04918814⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Article dans une revue

    Clément Bernard, Guillaume Postic, Sahar Ghannay, Fariza Tahi. Has AlphaFold 3 achieved success for RNA?. Acta crystallographica Section D : Structural biology [1993-..], 2025, 81 (2), pp.49–62. ⟨10.1107/S2059798325000592⟩. ⟨hal-04919467⟩

    STL

    Year of publication

    HAL publication

  • Chapitre d'ouvrage

    Philippe Boula de Mareüil, Plínio A. Barbosa. Picos melódicos pretônicos em final de enunciado no português brasileiro: um estudo quantitativo. Dermeval da Hora; Ángela Helmer. Interseções Linguísticas: Estudos Diversos, Líquido Editorial, pp.71-85, 2023, ALFAL, 9786599924804. ⟨hal-04893646⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Pré-publication, Document de travail

    Douglas Teodoro, Nona Naderi, Anthony Yazdani, Boya Zhang, Alban Bornet. A Scoping Review of Artificial Intelligence Applications in Clinical Trial Risk Assessment. 2025. ⟨hal-04913991⟩

    STL

    Year of publication

    HAL publication

  • Pré-publication, Document de travail

    Omar Adjali, Olivier Ferret, Sahar Ghannay, Hervé Le Borgne. Entity-aware cross-modal pretraining for Knowledge-Based Visual Question Answering. 2024. ⟨cea-04910767⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Thèse

    Paritosh Sharma. Sign Language synthesis by a decreasing granularity system from AZee. Computation and Language [cs.CL]. Université Paris-Saclay, 2024. English. ⟨NNT : 2024UPASG092⟩. ⟨tel-04908078⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Communication dans un congrès

    Laetitia Biscarrat, David Doukhan, Cyril Grouin. De Loft Story aux Marseillais à Dubaï : apport des méthodes d’analyse automatique pour la description des évolutions du dispositif télévisuel. Colloque ”La téléréalité, entre média, événement et société”, part of 89e Congrès de l’Association canadienne-française pour l’avancement des sciences (ACFAS), Association canadienne-française pour l’avancement des sciences (ACFAS), 2022, Montreal, Canada. ⟨hal-04906923⟩

    STL

    Year of publication

    HAL publication

  • Communication dans un congrès

    Laetitia Biscarrat, David Doukhan, Cyril Grouin. De Loft Story aux Marseillais à Dubaï : 20 ans de télé-réalité, 20 ans de sexisme ? Apport des méthodes d’analyse automatique pour une approche comparative. Première journée d’études de l’Arcom, ARCOM, Nov 2022, Paris, France. ⟨hal-04905959⟩

    STL, STL

    Year of publication

    Available in free access

    HAL publication

  • Communication dans un congrès

    Rémi Uro, Marie Tahon, David Doukhan, Albert Rilliard. Comprendre les phénomènes permettant la gestion des tours de parole dans les contenus de médias audiovisuels. Journée commune AFIA-TLH / AFCP – “Extraction de connaissances interprétables pour l’étude de la communication parlée”, Corinne Fredouille; Maëva Garnier; Olivier Perrotin; Marie Tahon, Dec 2023, Avignon, France. ⟨hal-04906679⟩

    STL, TLP

    Year of publication

    HAL publication

  • Autre publication scientifique

    Louis Estève, Kaja Dobrovoljc. A new pipeline for measuring diversity across various linguistic levels. 2025. ⟨hal-04886792⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Communication dans un congrès

    Leticia Rebollo Couto, Albert Rilliard. Variação Pragmática e Diminutivização: intensificação e atenuação de atos expressivos e diretivos para a dublagem de animação em português, espanhol e francês. IV Colloque International VariaR 2024, Université Paul-Valéry Montpellier 3, Jun 2024, Montpellier, France. pp.43-44, ⟨10.3726/978-3-0351-0740-1⟩. ⟨hal-04874595⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Thèse

    Sofiya Kobylyanskaya. Towards multimodal assessment of L2 level : speech and eye tracking features in a cross-cultural setting. Computation and Language [cs.CL]. Université Paris-Saclay, 2024. English. ⟨NNT : 2024UPASG111⟩. ⟨tel-04900961⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Poster de conférence

    Leticia Rebollo Couto, Albert Rilliard. Variación pragmática y expresividad negativa: análisis multimodal en datos de doblaje. LingCor2024: Workshop on Spoken Corpus Linguistics, Jul 2024, Vienna, Austria. . ⟨hal-04874470⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Communication dans un congrès

    Clémentine Bleuze, Fanny Ducel, Karën Fort, Maxime Amblard. Vers la création d’une super-intelligence » : un corpus pour étudier les revendications des articles de TALTraitement Automatique des langues. Journées de lancement LIFT 2, Nov 2024, Orléans, France. ⟨hal-04880335⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Communication dans un congrès

    Ayoub Hammal, Benno Uthayasooriyar, Caio Corro. Few-Shot Domain Adaptation for Named-Entity Recognition via Joint Constrained k-Means and Subspace Selection. COLING 2025 – 31st International Conference on Computational Linguistics, Jan 2025, Abu Dhabi, United Arab Emirates. pp.1-15. ⟨hal-04877776⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Communication dans un congrès

    Simon Devauchelle, Albert Rilliard, David Doukhan, Lucas Ondel Yang. Describing voice in French media archives: age and gender effects on pitch and articulation characteristics. XX Convegno Nazionale AISV, LFSAG (Laboratorio di Fonetica Sperimentale “Arturo Genre”) Dipartimento di Lingue e Letterature Straniere e Culture Moderne Università degli Studi di Torino, Feb 2024, Turin (Italie), Italy. ⟨hal-04874662⟩

    STL

    Year of publication

    HAL publication

  • Communication dans un congrès

    Donna Erickson, João Antônio De Moraes, Albert Rilliard. Dimensões das atitudes prosódicas entre culturas. V Seminário Internacional de Fonologia, Universidade Federal do Rio de Janeiro, Nov 2024, Rio de Janeiro (BR), Brazil. ⟨hal-04874627⟩

    STL

    Year of publication

    HAL publication

  • Communication dans un congrès

    Khanh-An C Quan, Camille Guinaudeau, Shin’Ichi Satoh. Evaluating VQA Models’ Consistency in the Scientific Domain. Multimedia Modelling 2025, Jan 2025, Nara, Japan. ⟨hal-04860239⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Communication dans un congrès

    Saumya Yadav, Elise Lincker, Caroline Huron, Stéphanie Martin, Camille Guinaudeau, et al.. Towards Inclusive Education: Multimodal Classification of Textbook Images for Accessibility. Multimedia Modelling 2025, Jan 2025, Nara, Japan. ⟨hal-04860245⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Communication dans un congrès

    Delphine Bernhard, Myriam Bras, Anne-Laure Ligozat, Aleksandra Miletic, Jean Sibille, et al.. L’avenir numérique des langues minoritaires : bilan du projet RESTAURE pour l’alsacien, l’occitan et le picard. Colloque « Langues minoritaires » : quels acteurs pour quel avenir ?, Groupe d’Etudes sur le Plurilinguisme européen (EA1339 LiLPa), Nov 2019, Strasbourg, France. ⟨hal-04864670⟩

    ILES, ILES, STL

    Year of publication

    HAL publication

  • Article dans une revue

    Cyril Grouin, Natalia Grabar. Year 2023 in Biomedical Natural Language Processing: A Tribute to Large Language Models and Generative AI. IMIA Yearbook of Medical Informatics, 2024. ⟨hal-04865083⟩

    STL, STL

    Year of publication

    HAL publication

  • Communication dans un congrès

    Natalia Grabar, Thierry Hamon. Study of the propaganda techniques occurring in Russian newspaper titles in 2022. METAPOL, université de Liège, Nov 2024, Liège (Belgique), Belgium. ⟨hal-04865074⟩

    STL

    Year of publication

    HAL publication

  • Article dans une revue

    Angèle Gayet-Ageron, Khaoula Ben Messaoud, Mark Richards, Cyril Jaksic, Julien Gobeill, et al.. Gender and geographical bias in the editorial decision-making process of biomedical journals: a case-control study. BMJ Evidence-Based Medicine, 2024, pp.bmjebm-2024-113083. ⟨10.1136/bmjebm-2024-113083⟩. ⟨hal-04865134⟩

    STL

    Year of publication

    HAL publication

  • Communication dans un congrès

    Omar Adjali, Olivier Ferret, Sahar Ghannay, Hervé Le Borgne. Multi-Level Information Retrieval Augmented Generation for Knowledge-based Visual Question Answering. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Nov 2024, Miami, United States. pp.16499-16513, ⟨10.18653/v1/2024.emnlp-main.922⟩. ⟨hal-04852275⟩

    STL

    Year of publication

    Available in free access

    HAL publication

All Publications