Single Image to Semantic BIM: Domain-Adapted 3D Reconstruction and Annotations via Multi-Task Deep Learning

The digitization and semantic enrichment of built environments traditionally rely on costly and labor-intensive processes, which hinder scalability, adaptability, and real-time deployment in real-world applications. This research presents a novel, fully automated approach that transforms single RGB images directly into semantically enriched, Building Information Modeling (BIM)-compatible 3D representations via an innovative domain adaptation and multi-task learning pipeline. The proposed method simultaneously leverages depth estimation and semantic segmentation from single-image inputs, using high-capacity 2D neural networks, thereby enabling accurate 3D mesh reconstruction and semantic labeling without manual annotation or specialized sensors. The developed pipeline segments and reconstructs both common architectural elements and previously unrepresented object classes, such as stairs, balustrades, railings, people, and furniture items, expanding the coverage of existing 3D indoor datasets. Experimental evaluations demonstrate remarkable reconstruction precision, with an RMSE as low as 0.02 and a per-point semantic accuracy of 81.89% on the TUM CMS Indoor Point Clouds dataset. The resulting 3D models are directly exportable to BIM, OBJ, and CAD formats, supporting a wide range of applications including digital documentation, asset management, and digital twins. By achieving high accuracy and semantic richness with minimal input, the proposed framework offers a scalable, efficient, and automated solution for the rapid digitization of complex built environments, addressing critical limitations in traditional scan-to-BIM workflows and setting new performance standards for future research in the field.     «

The digitization and semantic enrichment of built environments traditionally rely on costly and labor-intensive processes, which hinder scalability, adaptability, and real-time deployment in real-world applications. This research presents a novel, fully automated approach that transforms single RGB images directly into semantically enriched, Building Information Modeling (BIM)-compatible 3D representations via an innovative domain adaptation and multi-task learning pipeline. The proposed method...     »