台灣手語中視覺性為主與操作性為主手語之研究

Vision- and Manipulation-based Signs in Taiwan Sign Language

本研究欲探討手語中的視象性(iconicity)的表徵方式，以及這樣的表徵方式如何影響語言及腦部的運作。視象性的手語指的是手語中能清楚反應出所欲呈現該事物本身的動作、外型或特性。在台灣手語中，視象性手語包含有以視覺表徵為主的手語及操作表徵為主的手語等兩類，手語使用者在運作這兩類手語時，由於不同的表徵方式，因而會有不同的語言及腦部的運作。在本研究中，實驗中呈現黑白圖片給聾人手語使用者看，並要求聾人手語使用者以台灣手語或是中文在心裡默想該圖片的名稱，同時，以功能性磁振造影(fMRI)記錄腦部的活動情形。首先，分析結果顯示聾人手語使用者在以台灣手語和中文默想圖片名稱時，腦部活動激發的區域相當一致，主要包含背側視覺路徑(dorsalvisualroute)及Sylvian裂周邊的區域(perisylvianareas)；此外，過去的研究中發現表徵物體外型和動作會有不同的神經激發型態，這主要是在辨識這些事物時，負責處理運作的感官知覺管道比重不同所致，由本實驗結果也顯示，手語的表徵特性（視覺性為主與操作性為主）所激發的神經型態，也會與腦部不同區域的功能有關（例如視覺與運動路徑），這顯示腦部的神經運作不僅會反映出不同的語言學類別（語法和語意類別），同時也會因為不同的型態及概念表徵影響，動態地調整腦部神經運作型態。

An fMRI experiment was conducted to investigate how iconic elements are represented in sign languages and how such representation affects the linkage between language and brain. Iconic signs are those that clearly resemble the actions, objects, or characteristics they represent. In Taiwan Sign Language (TSL), expressions involve two kinds of signs: vision-based signs and manipulation- based signs. It is proposed that these two kinds of signs impose different processing demands on the signers. In our experiment, deaf signers viewed line drawings and were asked to covertly produce the names of the pictures either in TSL or in Chinese. Brain activities were measured with an fMRI technique. Analyses of the neural images indicated that the neural systems involved in the production of TSL and Chinese words were remarkably similar, in that the dorsal visual route and perisylvian areas were important in processing both languages for our deaf participants. On the other hand, the neural substrates mediating the representation of objects and/or action took different dynamically distributed forms, depending on which sensory/perceptual channels were required to weight the input information for recognition. In addition, results also suggested that the characteristics of signs (i.e., vision-based vs. manipulation-based) correlated with functional arrangements (i.e., visual vs. motor pathway) of the cerebral cortex, indicating that the brain not only represents the structural knowledge of the linguistic categories (syntactic and/or the semantic categories), but also involves the dynamic processes which characterize the interaction between modality and conceptual representation.