應用沉浸式擴增實境音像互動於現場音樂表演之研究與創作

A Study of Audio-visual Interaction Based on Immersive Augmented Reality for Live Music Performance

現場音樂表演不論任何形式，多半都離不開視聽的呈現。從舊有的音樂表演來看，傳統音樂會的視覺呈現主要是透過表演者的肢體動作、服裝造型、臉部表情等來表達，藉以加強音樂中的情緒，然而隨著科技和舞臺技術的進步，使得當代的音樂表演之視覺呈現已不局限於表演者本身。當代的音樂表演尚可透過燈光、投影、煙霧等元素，皆能強化表演的視覺效果，從而加強觀眾的觀賞感受。本研究嘗試透過沉浸式擴增實境的技術設計一件音像互動的作品，以頭戴式顯示器與手機的硬體裝置，在現實世界當中與虛擬世界結合並產生互動，藉此強化現場音樂表演視覺之呈現，並針對音樂與聲響特性，設計出相對應的粒子效果，利用擴增實境技術使視覺元素在空間中懸浮，創造符合音樂表演的視覺情境。透過此系統的建置，得以虛擬的視覺體驗，強化觀賞者在聆賞音樂表演的沉浸感。研究前期經由文獻蒐集及相關案例探討，歸納出沉浸式擴增實境音像互動的設計原則，並依據這些理論基礎開發出沉浸式擴增實境音像互動作品，研究後期將透過作品之展演進行實驗測試，檢視本研究之設計成果。最後回顧本研究之歷程，針對相關研究成果進行歸納探討，總結具體建議。

In all forms of live music performance, audio-visual presentation is indispensable. In traditional music performances, the visual presentation mainly involves expression through the performers' body movements, costume modeling, and facial expressions, among others, to emphasize the emotions in the music. However, with the advancement of technology, especially stage technology, the visual presentations in contemporary music performances are no longer limited to the performers. The visual effects in contemporary music performances can also be enhanced through elements such as lights, projections, and fog machines, thereby magnifying the audience's viewing experience. This research attempts to design an audio-visual interactive project using immersive augmented reality technology, including a head-mounted display and a mobile device, to combine and interact with the virtual environment, thereby enhancing the visual presentation in a live music performance. In addition, based on the characteristics of music and sound, a corresponding particle effect was designed, and the visual elements were floating in the space through augmented reality technology to create a visual situation consistent with the music performance. Through the construction of such a system, a virtual visual experience can be improved to enhance the viewer's immersion in a live music performance. During the early stage of the research, through the collection of related literature and case discussions, the design principles of audio-visual interaction based on immersive augmented reality were summarized. This theoretical foundation was used to develop an immersive augmented reality audio-visual interactive work. Later in this study, tests will be conducted on the performance of the project to review the findings of this research. To summarize, the researcher reviewed the course of this study, which summarizes and discusses related research results and makes specific suggestions.