This article tries to explore the holy images of Jerusalem Latin Christendom and Islamic world. This study traces back to the Crusades movements in the medieval period, and discusses the religious ideologies, actions, and interactions of both faiths. The discussions start from the Abraham religion, which is the origin of the sacredness of Jerusalem and the common origin of both religions. Then it analyzes religious concepts that are inherited and developed in Latin Christendom and the Islamic world. Further, the study focuses on the formation and process of the Crusades, and points out how similar concepts such as Holy War and Jihad could be appropriated, and how this process strengthens the sanctification of Jerusalem again. In addition, this study explores perspectives of heaven-faith and earthworld about Jerusalem respectively, then points out that, in fact, the concept of “Holy City” carries different perspective and meanings in Latin Christendom and the Islamic world. Finally, this article presents the complex factors of the sacredness of Jerusalem and indicates that it is necessary to understand and compare the religious concepts of both faiths through historical retrospection.