本研究提出一種基於電腦視覺和人機互動/協作輔助從模型資訊到工業機器人於現場自主物件辨識與組裝的方法。其發展機器人建造技術於現場施工的可能性與相對應用，結合多學科的資源及技術來實現機器人施工的可能性與發展。本文將人工智慧、計算機運算與機器人組裝一體化流程，結合建築資訊模型和視覺辨識系統。開發了一種基於BIM輔助自主機器人辨識與姿態預測系統（BIM-assisted Autonomous Robotic Recognition and Pose Estimation System, ARROS）。將設計端的數位物件從電腦端輸出數據、資料與其特徵，透過電腦視覺引導與即時抓取姿態系統來抓取/操作現場的目標物件。使現場機械臂能夠自主處理建築構件，建造不受物件材料、形狀、色澤和環境影響的特殊設計。為了讓現場機器人能夠根據物理環境的即時資訊做出決策，本研究開發一種低成本且開源的電腦視覺引導閉環式控制系統。該辨識系統分為三個階段： (1) BIM輔助虛擬物件掃描 (2) 物體深度辨識系統、 (3) 現場物件夾取姿態辨識系統。以手在眼(Eye-in-hand)的配置搭載深度相機，並整合於人機協作使用介面，即時偵測的資訊回饋於用戶端，具有修正錯誤或是人工操作的功能，藉此增加整體流程的靈活性和應用性。以此視覺辨識方法進一步探討施工現場不同環境與物理條件下的測試與討論，並適用於半結構化的場域。本文開發的系統與配置目標於研究開發機器人自動化現場組裝的可能性，探討智慧營建的新形態應用。

This research proposes a method that integrates computer vision and human-robot collaboration to assist industrial robots in autonomous on-site object recognition and assembly, bridging the gap from model information to physical construction tasks. It explores the possibilities and practical applications of robotic construction technology at the construction site by integrating interdisciplinary resources and techniques. The proposed approach combines artificial intelligence, computational techniques, robotic assembly methods, Building Information Modeling (BIM), and visual recognition systems into an integrated workflow. The paper developed a BIM-assisted Autonomous Robotic Recognition and Pose Estimation System (ARROS), which converts digital design data and object characteristics into actionable instructions at the construction site. Utilizing real-time visual guidance and pose estimation techniques, ARROS enables robotic manipulators to handle construction elements autonomously, irrespective of variations in material, shape, color, or environmental conditions. To facilitate real-time decision-making based on dynamic site conditions, this study presents a low-cost, open-source, visually guided closed-loop control system. The recognition process includes three primary stages： (1) BIM-assisted virtual object scanning, (2) object depth recognition, and (3) on-site object grasp pose estimation. A depth camera is mounted in an eye-in-hand configuration, and an interactive human-robot collaboration interface provides real-time feedback to users, allowing immediate corrections or manual interventions, thereby enhancing the system’s overall flexibility and usability. Further testing and discussions of this visual recognition method were conducted across various environmental and physical conditions on construction sites, particularly emphasizing applicability in semi-structured scenarios. Ultimately, the system and methods developed in this research aim to advance robotic automation for on-site assembly, exploring novel applications within the context of intelligent construction.