BEGIN:VCALENDAR VERSION:2.0 PRODID:-//wp-events-plugin.com//7.4.2//EN TZID:Europe/Paris X-WR-TIMEZONE:Europe/Paris BEGIN:VEVENT UID:0-361@lisn.upsaclay.fr DTSTART;TZID=Europe/Paris:20260220T100000 DTEND;TZID=Europe/Paris:20260220T130000 DTSTAMP:20260206T133103Z URL:https://www.lisn.upsaclay.fr/evenements/generation-of-facial-nonverbal -behavior-for-socially-interactive-agents-a-convolutional-generative-adver sarial-approach/ SUMMARY:Generation of Facial Nonverbal Behavior for Socially Interactive Ag ents: A Convolutional Generative Adversarial Approach DESCRIPTION:Jury\nStefan Kopp – Professor\, Bielefeld University (Reviewe r)\nJonas Beskow – Professor\, KTH Royal Institute of Technology (Review er)\nCatherine Pelachaud – Professor\, Sorbonne University (Examiner)\nR achel McDonnell – Professor\, Trinity College Dublin (Examiner)\nZerrin Yumak – Associate Professor\, Utrecht University (Examiner)\nStéphane A yache – Professor\, Aix-Marseille University (Guest member)\nSébastien Biaudet – Chief Technology Officer\, DAVI\, The Humanizers (Guest memb er)\nAbstract\nTo communicate\, humans naturally combine gestures\, gaze\, head movements\, and facial expressions during face-to-face interaction. Socially Interactive Agents (SIAs) aim to reproduce these multimodal behav iors to facilitate human-machine communication. Among nonverbal cues\, fac ial behaviors are particularly critical: they contribute to intelligibilit y\, naturalness\, affective expression\, and impression formation\, but ca n also trigger uncanniness when inappropriate or poorly synchronized. This thesis focuses on the automatic generation of believable facial nonverbal behaviors for SIAs\, encompassing head movements\, gaze direction\, and f acial expressions. \;\nSeveral challenges must be addressed to achieve this goal\, beginning with the joint generation of facial modalities in a manner consistent with their natural coordination in human communication. The first contribution of this thesis is FaceGen\, an encoder-decoder mod el based on convolutional generative adversarial networks\, designed to jo intly synthesize head motion\, gaze\, and FACS-based facial expressions du ring speaking phases. FaceGen generates facial nonverbal signals directly from the speech signal\, which are then used to animate a virtual agent. T rained and evaluated on the TRUENESS corpus\, featuring professional actor s enacting dyadic interactions involving ordinary sexism and racism\, the model is validated using both objective and subjective evaluations. Result s show that our modeling choices significantly enhance the perceived belie vability of the agent and its coordination with speech. \;\nA second c hallenge lies in modeling how affective and interactional factors shape fa cial behavior\, while enabling explicit control over the affective attitud e expressed by the agent. The thesis addresses this through FaceAttGen\, a n extension of FaceGen formulated as a conditional generative model that p roduces affective facial nonverbal behaviors during both speaking and list ening phases. FaceAttGen is original in its ability to be conditioned on s ocial attitudes while generating facial behaviors that remain affectively appropriate to the unfolding interaction context. Using a semisupervised l earning strategy\, the model learns to reproduce two contrasted social att itudes: hot anger and conciliation. Objective evaluations validate the arc hitectural extensions introduced in this model\, and subjective studies co nfirm its ability to shape the affective variability of the generated beha viors. \;\nA further contribution of the thesis concerns the developme nt of an objective evaluation framework that better aligns with human perc eptual judgments of believability and appropriateness than the commonly us ed objective metrics in the field. To this end\, we propose an evaluation methodology that combines multiple metrics into a composite score. Results from a perceptual study\, compared with the objective measures\, show tha t this composite framework correlates more strongly with human judgments t han existing metrics and supports the optimization of model architectures and hyperparameters. \; Finally\, the ethical dimension of generated b ehaviors is examined\, with a particular focus on gender bias. After demon strating the persistence of such biases both in real data and in the outpu ts of generative models\, the thesis introduces FairGenderGen\, a model th at generates facial nonverbal behaviors from speech while attenuating gend er bias through gradient-reversal domain adaptation. CATEGORIES:IaH,Thèses et HDR LOCATION:Amphithéâtre Hexagone, Campus Universitaire de Luminy 1309 Marse ille\, France X-APPLE-STRUCTURED-LOCATION;VALUE=URI;X-ADDRESS=Amphithéâtre Hexagone, Ca mpus Universitaire de Luminy 1309 Marseille\, France;X-APPLE-RADIUS=100;X- TITLE=:geo:0,0 END:VEVENT BEGIN:VTIMEZONE TZID:Europe/Paris X-LIC-LOCATION:Europe/Paris BEGIN:STANDARD DTSTART:20251026T020000 TZOFFSETFROM:+0200 TZOFFSETTO:+0100 TZNAME:CET END:STANDARD END:VTIMEZONE END:VCALENDAR