Liudui is a unique Hakka volunteer organization in Taiwan’s social development in the Qing Dynasty, and the Loyalty Pavilion was the spiritual core, which are our collective memory. However, the vicissitude of the Loyalty Pavilion suggests a major qualitative changes in Liudui. As a result, to redefine the history of Liudui is necessary because to reveal the true meaning is the core of cultural heritage reservation. After research, it is found that "Liudui" is a collective diachronic designation. It includes multi-ethnic groups of volunteer, militia, and finally water resources organization. The geographical location is in accordance with the water system of the initial migrants, and due to their different classes, each region represents different cultural style. Xishi was the core of the power construction at that time because of the inherent advantages in water resource, therefore, and became the location of Loyalty Pavilion. Later, due to the development of water canal, land transportation, and the settled society, the spiritual symbol is transformed from the imperial dependence of a single Hakka ethnic group to Mazu belief of the multi-ethnic, that Neipuzhuang became the leader, while the organizational link elements were transferred from the external needs of survival and safety, industry and transportation to the higher level need of social status. The industry organizational structure of Liudui was formally constructed during the Tongzhi period. Because the historical meaning that the spatial location of Loyalty Pavilion reveals the fundamental change in Liudui, as a result, the concept of "loyalty Pavilion" should be viewed as an important intangible cultural heritage in Hakka history to reserve.