LIPS

Language Interaction Speech and Sign (LIPS)

The LIPS team, made up of researchers in linguistics and language processing, conducts multidisciplinary research into oral -spoken and signed- languages. It cooperates extensively with other teams in the STL department, as well as with other departments in the laboratory.

The team’s scientific challenges concern oral, spoken and signed, languages, with the aim of linguistic description and modelling. The team brings together researchers in natural language processing and linguists to focus on the situated dimension of language: we use a variety of data, of different sizes and from different sources, illustrating linguistic variation in all its dimensions, from minimal units to meaning. Multimodal processing involving the written and aural variety of spoken languages as well as other visual information (e.g. occulometry), or owritten and aural varieties of different languages (e.g. sign language videos with French subtitles), is also at the heart of our concerns. Our work gives rise to a variety of applications: speech and sign language recognition and synthesis, dialogue systems. Our research is interdisciplinary by nature and requires skills in signal processing, linguistics and computer science. Our research is interdisciplinary by nature and requires skills in signal processing, linguistics and computer science.

The team’s activities are organised around three themes:

Information retrieval in dialogues

Work on multimodal and conversational information retrieval is centered around two main pillars: incorpo-
rating multimodality into information retrieval systems and studying dialogic interactions. In more detail, this
research is focused on how to represent multimodal data, taking into account contexts and various multi-
modal aspects in the developed representations, and addressing the challenge posed by the scarcity of avail-
able data. The artificial intelligence methods implemented also tackle issues related to handling degraded
data, continuous and interactive learning, while aiming to make model predictions understandable, with an
eye towards explainability.

Sign language modeling and processing

Sign languages, which are poorly endowed languages, have a linguistic system resulting from their visuo-gestural nature: a large amount of information is expressed simultaneously and organized spatially, and iconicity
plays a central role. Computer modeling of SL requires the design of representations with little
available data, and where pre-existing models, which are essentially linear, have been developed for written
or spoken languages and do not cover all aspects of LS. Through projects and PhD theses and in collaboration with signers of these languages (e.g. deaf translators and journalists), we are tackling the following research question: How can SL be analysed, represented and processed? How can we take into account the linguistic specifics linked to their visual-gestural nature (multilinearity, spatialization, iconicity)? What types of approach are possible with little LSF data? Current projects are detailed on this page.

Speech processing and multilingual variation modeling

Research in this theme aims to understand the variation phenomena that underlie temporal and spatial
changes in spoken language and to develop models for use in automatic speech processing. One of our objectives is to structure the information in audio documents by developing models and algorithms
that rely on diverse information sources and can serve to detect the presence of speech, to identify the lan-
guage being spoken and to characterize the speaker(s), to transcribe the speech into text in the same or a
different language or identify specific entities or acoustic events. Concerning speech recognition, our research aims to complete the word sequence with punctuation and with paralinguistic information such as hesitations, laughter or breath noises. We also study frugal learning techniques and applied them to speech recognition for low e-resourced languages and tasks.

News

All News Articles

Coordination

Team members

Publications

  • Article dans une revue

    Clémentine Bleuze, Karën Fort, Vincent P Martin, Aurélie Névéol. Large Language Models for Mental Health Prediction: Scoping Review of Bias and Clinical Utility Documentation in 2019-2024. JMIR AIArtificial Intelligence, 2026, 5, pp.e88082-e88082. ⟨10.2196/88082⟩. ⟨hal-05717650⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Communication dans un congrès

    Marco Naguib, Christel Gérardin, Victor Beaucoté, Cyril Charron, Adrien Joseph, et al.. Evaluating the Retrieval Component in a Retrieval-Augmented Summarization System for Patient Records in French. The Fifteenth Language Resources and Evaluation Conference (LREC 2026), May 2026, Palma, France. pp.57-65, ⟨10.63317/4cy8xxinjw7z⟩. ⟨hal-05716360⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Communication dans un congrès

    Aygalic Jara-Mikolajczak, Thomas Lavergne, Christophe Servan, Sophie Rosset. Robustesse des LLM dans les contextes longs, hallucinations et détection sur questions-réponses séquentielles. 21e Conférence en Recherche d’Information et Applications (CORIA) 19e Rencontres Jeunes Chercheurs en RI (RJCRI) 33e Conférence sur le Traitement Automatique des Langues Naturelles (TALN) 28e Rencontre des Étudiants Chercheurs en Informatique pour le Traitement Automatique des Langues (RÉCITAL), Jun 2026, Nantes, France. pp.512-523. ⟨hal-05708368⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Communication dans un congrès

    Zhongjie Li, Rim Abrougui, Guillaume Lechien, Elisabeth Savatier, Benoît Laurent, et al.. Sem-G-RAG, combiner sémantique symbolique à base de graphes et LLM pour le RAG. 21e Conférence en Recherche d’Information et Applications (CORIA) 19e Rencontres Jeunes Chercheurs en RI (RJCRI) 33e Conférence sur le Traitement Automatique des Langues Naturelles (TALN) 28e Rencontre des Étudiants Chercheurs en Informatique pour le Traitement Automatique des Langues (RÉCITAL), Jun 2026, Nantes, France. pp.544-563. ⟨hal-05708371⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Communication dans un congrès

    Eve Sauvage, Cyril Grouin, Julien Tourille. Tous les tokens sont-ils utiles pour les modèles de langues ?. 21e Conférence en Recherche d’Information et Applications (CORIA) 19e Rencontres Jeunes Chercheurs en RI (RJCRI) 33e Conférence sur le Traitement Automatique des Langues Naturelles (TALN) 28e Rencontre des Étudiants Chercheurs en Informatique pour le Traitement Automatique des Langues (RÉCITAL), Jun 2026, Nantes, France. pp.595-609. ⟨hal-05708373⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Communication dans un congrès

    Oumaima El Khettari, Virgile Barthet, Guillaume Hocquet, Joconde Weller, Emmanuel Morin, et al.. Is Clinical Text Enough? A Multimodal Study on Mortality Prediction in Heart Failure Patients. LREC 2026 – 15th Language Resources and Evaluation Conference, May 2026, Palma, Spain. pp.194-206, ⟨10.63317/47hsfchk79n6⟩. ⟨hal-05709146⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Communication dans un congrès

    Lounès Kebdi, Lubin Longuépée, Mathilde Aguiar, Pierre Zweigenbaum, Nona Naderi. Impact de l’affinage de modèles génératifs pour l’inférence en langue naturelle appliquée aux essais cliniques : comparaison avec des approches de *few-shot learning. 21e Conférence en Recherche d’Information et Applications (CORIA) 19e Rencontres Jeunes Chercheurs en RI (RJCRI) 33e Conférence sur le Traitement Automatique des Langues Naturelles (TALN) 28e Rencontre des Étudiants Chercheurs en Informatique pour le Traitement Automatique des Langues (RÉCITAL), Jun 2026, Nantes, France. pp.303-321. ⟨hal-05708357⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Proceedings/Recueil des communications

    Vincent Claveau, Nicolas Diniz, Juliane Flament, Nihel Kooli, Jose Moreno, et al.. Actes de l’atelier sur l’évaluation des modèles génératifs (LLM) et challenges (EvalLLM 2026). 21e Conférence en Recherche d’Information et Applications (CORIA) 19e Rencontres Jeunes Chercheurs en RI (RJCRI) 33e Conférence sur le Traitement Automatique des Langues Naturelles (TALN) 28e Rencontre des Étudiants Chercheurs en Informatique pour le Traitement Automatique des Langues (RÉCITAL), Jun 2026, Nantes, France. ATALA, 2026. ⟨hal-05708467⟩

    STL

    Year of publication

    HAL publication

  • Communication dans un congrès

    Khanh-an C. Quan, Camille Guinaudeau, Shin’Ichi Satoh. Évaluation de la cohérence des modèles vision-langage pour la tâche de question-réponse visuelle. 21e Conférence en Recherche d’Information et Applications (CORIA) 19e Rencontres Jeunes Chercheurs en RI (RJCRI) 33e Conférence sur le Traitement Automatique des Langues Naturelles (TALN) 28e Rencontre des Étudiants Chercheurs en Informatique pour le Traitement Automatique des Langues (RÉCITAL), Jun 2026, Nantes, France. pp.64-74. ⟨hal-05708471⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Communication dans un congrès

    You Zuo, Kim Gerdes, Éric de la Clergerie, Benoît Sagot. Sparse Coverage: Semantic Center Representations for Patent Prior-Art Retrieval. CORIA-TALN 2026 – 21e Conférence en Recherche d’Information et Applications (CORIA), Jun 2026, Nantes, France. ⟨hal-05707237⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Communication dans un congrès

    Younes Djemmal, Olutola Oloruntobi Paul, Kim Gerdes. Au-delà des résumés : Apprentissage des représentations d’articles scientifiques à partir de fenêtres de texte intégral. 21e Conférence en Recherche d’Information et Applications (CORIA) 19e Rencontres Jeunes Chercheurs en RI (RJCRI) 33e Conférence sur le Traitement Automatique des Langues Naturelles (TALN) 28e Rencontre des Étudiants Chercheurs en Informatique pour le Traitement Automatique des Langues (RÉCITAL), Jun 2026, Nantes, France. pp.31-50. ⟨hal-05708484⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Communication dans un congrès

    Marc Evrard, Rémi Uro, Nicolas Hervé, Béatrice Mazoyer. French Tweet Corpus for Automatic Stance Detection. Proceedings of the Twelfth International Conference on Language Resources and Evaluation (LREC 2020), ELRA, May 2025, Marseille, France. pp.6317-6322. ⟨hal-05682655⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Communication dans un congrès

    Lorena de la Garza, Julie Halbout, Julie Lascar, Niels Martinez-Guevara, Arturo Curiel, et al.. Extracting Signs from Weakly Aligned Sign Language Corpora: A Study on LSF and LSM. 12th Workshop on the Representation and Processing of Sign Languages: Language in Motion (LREC 2026), May 2026, Palma De MaJorque, Spain. pp.174-183, ⟨10.63317/38kfot52b4dz⟩. ⟨hal-05688151⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Communication dans un congrès

    Lucía Catalán, Kim Gerdes. On the difficulty of producing good linguistic lies. Atelier sur l’Analyse et la Recherche de Textes Scientifiques 2026 (ARTS), Jun 2026, Nantes, France. ⟨hal-05688347⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Communication dans un congrès

    Idrissa Mahamoudou Dicko, Nona Naderi. Synergizing Domain-Specific Masked Language Models and Instruction-Tuned LLMs for Chemical NER. Atelier IAIntelligence Artificielle et santé, Jun 2026, Arras, France. ⟨hal-05679910⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Communication dans un congrès

    Jose Felipe Espinosa Orjuela, Philippe Boula de Mareüil, Marc Evrard. Speech synthesis for Walloon, an under-resourced minority language. 13th edition of the Speech Synthesis Workshop, Aug 2025, Leeuwarden, Netherlands. pp.189-195, ⟨10.21437/SSW.2025-29⟩. ⟨hal-05682647⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Communication dans un congrès

    Benedictus Kent Rachmat, Thomas Gerald, Zheng Zhang, Cyril Grouin. QA Analysis in Medical and Legal Domains: A Survey of Data Augmentation in Low-Resource Settings. ACL 2025 – 63rd Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop), ACL, Jul 2025, Vienna, Austria. pp.1132-1144, ⟨10.18653/v1/2025.acl-srw.89⟩. ⟨hal-05683004⟩

    AO, STL

    Year of publication

    Available in free access

    HAL publication

  • Communication dans un congrès

    Laura Ascone, Lucie Gianola, Julien Longhi, Laurène Renaut. La linguistique forensique pour l’analyse du discours : anticiper les risques, aider à la décision, répondre aux menaces. Colloque R2DIP, « Les notions de risques, société et sécurité dans les discours institutionnels et politiques », CY Cergy Paris Université, Dec 2017, Cergy, France. ⟨hal-05682141⟩

    ILES, STL

    Year of publication

    HAL publication

  • Proceedings/Recueil des communications

    Nina Khairova, Victoria Vysotska, Natalia Grabar, Thierry Hamon, Nina Rizun. Preface: computational linguistics workshop. Computational Linguistics Workshop at 9th International Conference on Computational Linguistics and Intelligent Systems, May 2025, Kharkiv, Ukraine. Vol-3976, 2025. ⟨hal-05682500⟩

    STL

    Year of publication

    HAL publication

  • Autre publication scientifique

    Théophile Lenoir, Ana Valdivia, Aurélie Bugeau, Anne-Laure Ligozat. Beyond the Energy Efficiency Directive. Observatory on the Environmental Footprint of AIArtificial Intelligence, 2026. ⟨hal-05680820⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Communication dans un congrès

    Iskandar Boucharenc, Eve Sauvage, Thomas Gerald, Julien Tourille, Sabrina Campano, et al.. Using Syntax for the Semantic Representation of Sentences. SLiDE 1st Workshop on Structured Linguistic Data and Evaluation at the 2026 Language Resources and Evaluation Conference (LREC 2026), May 2026, Palma de majorque, Spain. pp.169–179, ⟨10.63317/4gtinxarm3dd⟩. ⟨hal-05669816⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Communication dans un congrès

    Clément Morand, Aurélie Névéol, Anne-Laure Ligozat. The Rising Unsustainability of AIArtificial Intelligence Graphics Cards Production. LIMITS 2026: 12th Workshop on Computing within Limits, Jun 2026, Online, France. ⟨hal-05666542⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Communication dans un congrès

    Kim Gerdes. The Grammar Does the Work: Functional vs. Lexical Dependency Length Minimization Across Universal Dependencies. UDW 2026 – Ninth Workshop on Universal Dependencies, May 2026, Palma De MaJorque, Spain. pp.163-173, ⟨10.63317/4akqrtsv7i65⟩. ⟨hal-05676925⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Communication dans un congrès

    Iskandar Boucharenc, Sahar Ghannay, Christophe Servan, Laure Soulier, Sophie Rosset. Étude de l’adaptation des gros modèles de langues par retour visuel. Journée Visu, GdR IG-RV, Jun 2023, Orsay, France. ⟨hal-05670004⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Communication dans un congrès

    Emmett Strickland, Marc Evrard, Valentina Fedchenko. Transfer Learning for Creole TTS: A Pilot Study on Whether Substrate Phonologies or Lexifier Vocabularies Matter More. Towards Inclusivity and Equality: Language Resources and Technologies for Under-Resourced and Endangered Languages, SIGUL 2026 Joint Workshop with ELE, EURALI, and DCLRL, May 2026, Palma De Majorque, Spain. ⟨10.63317/5d5qjmokuvmc⟩. ⟨hal-05617449⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Communication dans un congrès

    Clémentine Bleuze, Bruno Guillaume, Aurélie Névéol, Karën Fort. Omniprésents et anthropomorphisés : analyse lexico-syntaxique des discours sur les LLM. TALN 2026 – 33e Conférence sur le Traitement Automatique des Langues Naturelles, Jun 2026, Nantes, France. ⟨hal-05670834⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Communication dans un congrès

    Clémentine Bleuze, Karën Fort, Vincent P. Martin, Aurélie Névéol. Grands modèles de langue pour prédire la santé mentale : une revue exploratoire de la documentation des biais et de l’utilité clinique. TALN 2026 – 33e Conférence sur le Traitement Automatique des Langues Naturelles, Jun 2026, Nantes, France. ⟨hal-05670826⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Article dans une revue

    Clément Morand, Aurélie Névéol, Rosy Tsopra, Anne-Isabelle Tropeano, Sophie de Chambine, et al.. Prospectively Evaluating the Environmental Impacts of Digital Health Applications : A Case Study and Recommendations. Journal of the American Medical Informatics Association, 2026, ⟨10.1093/jamia/ocag091⟩. ⟨hal-05628404⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Communication dans un congrès

    Thomas Gerald, Sahar Ghannay, Julie Lascar, Paul Lerner, Anne Vilnat. Can Multimodal LLMs Generate Pedagogical Questions?. LREC 2026, May 2026, Palma, Spain. ⟨10.63317/4z4gj3h8jmc7⟩. ⟨hal-05658326⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Communication dans un congrès

    Thierry Hamon. Description of the LISN system for extracting terms. DEfinition and Term Extraction CHallenge 2026 (DETECH 2026), Jun 2026, Zadar, Croatia. ⟨hal-05669893⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Communication dans un congrès

    Marie Schmit, Melvin Selim Atay, Khalid Belhajjame, Ulysse Le Clanche, Emmanuel Coquery, et al.. ShareFAIR-KG, a centralised knowledge base of scientific workflows. JOBIM 2026 – Journées Ouvertes en Biologie, Informatique et Mathématiques, Jun 2026, Strasbourg, France. ⟨hal-05666980⟩

    BioInfo, STL

    Year of publication

    Available in free access

    HAL publication

  • Pré-publication, Document de travail

    Louis Estève, Marie-Catherine de Marneffe, Nurit Melnik, Agata Savary, Olha Kanishcheva. A survey of diversity quantification in natural language processing: The why, what, where and how. 2026. ⟨hal-05661565⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Pré-publication, Document de travail

    Alexandre Genadot, Nicolas Guilliot, Philippe Boula de Mareüil. Introduction to the book “Cartographier les Langues de Nouvelle-Aquitaine: entre Grammaire et Société”. 2026. ⟨hal-05662837⟩

    STL

    Year of publication

    HAL publication

  • Communication dans un congrès

    Agata Savary, Manon Scholivet, Carlos Ramisch, Takuya Nakamura, Eric Bilinski, et al.. PARSEME 2.0 Multilingual Corpus of Multiword Expressions. LREC 2026 – 15th biennial Language Resources and Evaluation Conference, ELRA Language Resources Association, May 2026, Palma De MaJorque, Spain. pp.4819-4834, ⟨10.63317/2iy5qf38yhay⟩. ⟨hal-05661505⟩

    ILES, STL

    Year of publication

    Available in free access

    HAL publication

  • Communication dans un congrès

    Julie Halbout, Annelies Braffort, Michèle Gouiffès, Diandra Fabre, Julie Lascar. Learning to Spot Signs from Named Entities. A study on French Sign Language. LREC 2026 – 12th Workshop on the Representation and Processing of Sign Languages: Language in Motion, May 2026, Palma de Majorque, Spain. ⟨10.63317/26i8n4zuyzyx⟩. ⟨hal-05636077⟩

    AMIArchitectures et modèles pour l'Interaction, STL

    Year of publication

    Available in free access

    HAL publication

  • Communication dans un congrès

    Damien Lacroux, Aurélie Bugeau, Anne-Laure Ligozat. The indirect rebound effects of AIArtificial Intelligence as undone science: philosophical reflection on two structural causes. Undone Computer Science, Mar 2026, Luxembourg, Luxembourg. ⟨hal-05624399⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Communication dans un congrès

    Benedictus Kent Rachmat, Thomas Gerald, Zheng Zhang, Cyril Grouin. Les données de calibration comptent-elles vraiment pour LoRA?. EvalLLM2026 : Atelier sur l’évaluation des modèles génératifs (LLM), le RAG et challenges, Jul 2026, Nantes (France), France. ⟨hal-05633638⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Communication dans un congrès

    Mathilde Aguiar, Pierre Zweigenbaum, Nona Naderi. Assessing the Difficulty of Inference Types in Natural Language Inference for Clinical Trials. The Fifteenth Language Resources and Evaluation Conference (LREC 2026), May 2026, Palma, France. pp.5290-5300, ⟨10.63317/359toazp33g8⟩. ⟨hal-05652719⟩

    STL

    Year of publication

    HAL publication

  • Communication dans un congrès

    Jenny Copara, Nona Naderi, Gilles Falquet, Douglas Teodoro. MeSH Concept Relevance and Knowledge Evolution: A Data-Driven Perspective. 12th International Conference on Information Management and Big Data. Communications in Computer and Information Science, Oct 2025, Lima (Pérou), Peru. pp.280-299, ⟨10.1007/978-3-032-20322-9_20⟩. ⟨hal-05625658⟩

    STL

    Year of publication

    Available in free access

    HAL publication

  • Communication dans un congrès

    Clément Morand, Aina Rasoldier, Paul Gay. Not up to its critical perspective on digitalization: A Descriptive Analysis of How Sustainability is Approached in the ICT4S Conference. ICT4S, Jun 2026, Berne, France. ⟨hal-05615744⟩

    STL

    Year of publication

    Available in free access

    HAL publication

All Publications