picture

Allo Papa Noël

type
Experience
client
Bouygues Telecom
date
2021-12
role
  • UX & UI design
  • Development
Techs
  • Python

Millions of messages for WhatsApp Santa

To appeal to the imagination of users of all ages with an Artificial Intelligence feature developed for WhatsApp: such is the promise that we lived up to, enabling people all over France to easily get a personalized video message from Santa.

picture
The WhatsApp Bot was developed by combining deep-learning, computer vision and text-to-speech technologies. We will explain its step-by-step development:
Step #01

Turn a WhatsApp message into an audio message from Santa Claus

The messages sent to Santa are processed by text-to-speech technology. First the text is transformed into an mp3 audio file featuring a synthetic voice specially designed for the operation.

Step #02

Create Santa’s voice

We recorded the voice of a real-life actor, and we had him read a whole list of key words which provide the meansto recreate as many sounds of the French language as possible.A specific algorithm developed by Voxygène was then usedto synthesize the voice.

Step #03

Lip sync Santa Claus

It requires the development of three models: the first one detects a face in a video, the second generates a set of reference points based on a sound sequence, and thethird recreates a new, personalized face animation. -Model 1: We created a first algorithm and trained it to detect human faces on video. -Model 2: We generated a second algorithm able to analyze the phonemes on the audio recordings of these videos and to generate moving reference points accordingly. The objective of that process was to create a dataset and to train a third model. -Model 3: A third algorithm was developed and trained to use the reference points thus created to animate another face with very realistic movements. These models were then applied to the Santa videos produced in-house, and were used to personalize the movements of Santa’s mouth to fit the content of user messages.

picture
2021 results
2,034,000 generated videos
3,213,000 sent messages
2,806,00 messages received
made by
Maxime HuygheProduction Director
Lara Jane LelièvreExecutive Producer
Laure GenthonDigital Producer
Alexis GalbourdinTech lead
Benjamin VergneTech consultant
Dimitri UmimonPython dev
VonageAPI of communication
VoxygenSynthesis voice
UnlikelyBack development
We use various technologies such as cookies to produce statistics and improve your experience on the website. Please click the button below to give your consent. You can change your mind and modify your choices at any time.
Accept