Amplifying a Sense of Emotion toward Drama- Long Short-Term Memory Recurrent Neural Network for dynamic emotion recognition

Amplifying a Sense of Emotion toward Drama- Long Short-Term Memory Recurrent Neural Network for dynamic emotion recognition

本篇論文探討放大觀眾欣賞戲劇時候的情緒，使用 Long Short-Term Memory Recurrent Neural Network 來當作主要的模型，建立和預測連續動態的情緒。建立好模型後，我們將結果移植到視覺程式上，我們可以達成兩個不同的情緒視覺放大器，分別有 RGB 和 Vignette 版本，RGB 版本是控制影像中的 RGB 像素數值，及時隨著模型結果（Arousal 和 Valence 值），及時調整紅色、綠色和藍色數值，讓觀眾隨著影像中演員的情緒起伏；同樣也是，我們視覺化換成 Vignette，隨著情緒辨識維度（Arousal 和 Valence），來產生情緒視覺化畫面。而我們使用資料庫是 NNIME，NNIME 是最近收集的多模態互動情緒資料庫，總共有 44 位演員，約 11 個小時的連續資料，包含語音、影像和生理訊號，這個嚴謹和工程化收集的資料庫，提供給我們很好研究情緒的機會，讓我們得以完成此篇研究。

This paper tried to amplify a sense of emotion toward drama. Using Long Short-Term Memory Recurrent Neural Network to model and predict dynamic emotion(Arousal and Valence) recognition. After building model, we transplant whole framework and take results from it on visualizing. We have two demo version: RGB version and Vignette version. RGB version is to modulate the RGB value of frame in video. The Vignette one is to add the vignette effect. Both version all are to amplify a sense of emotion toward drama. Let people have more fun during watching videos. The database we used is NNIME (The NTHU-NTUA Chinese Interactive Multimodal Emotion Corpus). NNIME is a newlycollected multimodal corpus. This database includes recordings of 44 subjects engaged in spontaneous dyadic spoken interaction. The length of data is about 11 hours containing audio, video and electrocardiogram. The database is also completed with a rich set of emotion annotations on continuous-in-time annotation by four annotators. This carefullyengineered data collection and annotation processes provide us to create amplify framework.