# Expressive Furhat: Generating Real-Time Facial Expressions for Human-Robot Dialogue with LLMs

- Authors: Giulio Antonio Abbo, Ruben Janssens, Seppe Van de Vreken and Tony Belpaeme
- Date: 2026-03-16
- Type: conference paper
- Venue: Companion Proceedings of the 21st ACM/IEEE International Conference on Human-Robot Interaction (HRI '26 (LBR))
- DOI: https://doi.org/10.1145/3776734.3794436
- Code: https://github.com/giubots/expressive-furhat
- ResearchGate: https://www.researchgate.net/publication/402209762_Expressive_Furhat_Generating_Real-Time_Facial_Expressions_for_Human-Robot_Dialogue_with_LLMs
- PDF: https://giubots.net/publications/expressive-furhat-generating/expressive-furhat-generating.pdf
- Research line: [Collabs](https://giubots.net/research/collabs/)
- URL: https://giubots.net/publications/expressive-furhat-generating/


## TL;DR

In this paper, we present Expressive Furhat, a system that generates real-time facial expressions for the Furhat robot using large language models based on the dialogue context.


In this paper, we present Expressive Furhat, a system that generates real-time facial expressions for the Furhat robot using large language models based on the dialogue context.

This python library easily integrates with the Furhat API and allows for the generation of new gestures on the fly, as well as gaze aversion while the robot is speaking. We use a cache to store previously generated expressions and improve the system's response time; to avoid mixing emotions, we flush the cache when necessary. We conducted user studies to evaluate the system, and the results show significant improvements in user perception compared to a baseline system. Participants praised the system's emotional responsiveness and naturalness.

## Abstract

Enabling natural robot communication through dynamic, context-aware facial expressions remains a key challenge in human-robot interaction. The field lacks a system that can generate facial expressions in real time and can be easily adapted to different contexts. Early work in this area considered inherently limited rule-based systems or deep learning-based models, requiring large datasets. Recent systems using large language models (LLMs) could not yet generate context-appropriate facial expressions in real time. This paper introduces Expressive Furhat, an open-source algorithm and Python library that leverages LLMs to generate real-time, adaptive facial gestures for the Furhat robot. Our modular approach separates gesture rendering, new gesture generation, and gaze aversion, ensuring flexibility and seamless integration with the Furhat API. User studies demonstrate significant improvements in user perception over a baseline system, with participants praising the system's emotional responsiveness and naturalness.

## How to cite

Giulio Antonio Abbo, Ruben Janssens, Seppe Van de Vreken, and Tony Belpaeme. 2026. Expressive Furhat: Generating Real-Time Facial Expressions for Human-Robot Dialogue with LLMs. In Companion Proceedings of the 21st ACM/IEEE International Conference on Human-Robot Interaction (HRI Companion '26). Association for Computing Machinery, New York, NY, USA, 456–460. https://doi.org/10.1145/3776734.3794436

```bibtex
@inproceedings{10.1145/3776734.3794436,
author = {Abbo, Giulio Antonio and Janssens, Ruben and Van de Vreken, Seppe and Belpaeme, Tony},
title = {Expressive Furhat: Generating Real-Time Facial Expressions for Human-Robot Dialogue with LLMs},
year = {2026},
isbn = {9798400723216},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3776734.3794436},
doi = {10.1145/3776734.3794436},
booktitle = {Companion Proceedings of the 21st ACM/IEEE International Conference on Human-Robot Interaction},
pages = {456–460},
numpages = {5},
keywords = {embodied conversational agent, facial expression generation, gesture generation, human-robot interaction, multimodal dialog},
location = {Edinburgh, Scotland, UK},
series = {HRI Companion '26}
}
```