Existing computer use agent focus on general-purpose desktop automation tasks, with limited exploration of their application in highly specialized domains. In particular, the 3D building modeling process in the Architecture, Engineering, and Construction sector involves open-ended design tasks and complex interaction patterns within Building Information Modeling authoring software. In this thesis, we propose BIMgent, an agentic framework powered by multimodal large language models, designed to enable autonomous BIM modeling via graphical user interface operations. BIMgent automates the architectural building modeling process, including multimodal input for conceptual design, planning of software-specific workflows, and efficient execution of the authoring GUI actions. We evaluate BIMgent on real-world building modeling tasks, including both text-based conceptual design generation and reconstruction from existing building design. The design quality achieved by BIMgent was found to be reasonable. Its operations achieved a 32% success rate, whereas all baseline models failed to complete the tasks (0% success rate). Results demonstrate that BIMgent effectively reduces manual workload while preserving design intent, highlighting its potential for practical deployment in real-world architectural modeling scenarios.     «

Existing computer use agent focus on general-purpose desktop automation tasks, with limited exploration of their application in highly specialized domains. In particular, the 3D building modeling process in the Architecture, Engineering, and Construction sector involves open-ended design tasks and complex interaction patterns within Building Information Modeling authoring software. In this thesis, we propose BIMgent, an agentic framework powered by multimodal large language models, designed to e...     »