In the realm of 3D reconstruction pipelines, 2D conflict maps that indicate the presence openings in façades such as windows and doors, represent an intermediate output [1]. However, these maps often fall short of completeness due to insufficient coverage or occlusions caused by objects such as vegetation. This research delves into the exploration of deep learning strategies to address this limitation by inpainting unseen façade objects into the 2D conflict maps. The central focus of this study revolves around deploying the Stable Diffusion inpainting model [2, 3] and the LaMa GAN [4] for this purpose. Specifically, I investigate in the potential of personalizing a pre-trained Stable Diffusion inpainting model with Dreambooth [5] to facilitate its application in inpainting unseen façade objects within 2D conflict maps. Simultaneously, I undergo training for the LaMa GAN with a similar objective. By utilizing synthetic conflict maps that are derived from randomly generated semantic city models [6] and such that are derived from databases of annotated optical façade images [7] as data for training the LaMa GAN and personalisation a pre-trained Stable Diffusion with Dreambooth, I investigate if such data sources can facilitate the deployment of deep learning models. My results demonstrate the general capability of deep learning based methods for inpainting unseen objects into the 2D conflict maps. I find that the personalisation with Dreambooth yields improvements regarding the behaviour of diffusion based models when considering tree-shaped masks. This, and the successful training of the LaMa GAN demonstrate the utility of synthetic conflict maps and such derived from annotated images. The insights gained in this paper can be applied to existing pipelines for the reconstruction of LOD3 models. This way my work contributes to improving the reconstruction accuracy of such approaches. It also serves as the basis for further exploration.     «

In the realm of 3D reconstruction pipelines, 2D conflict maps that indicate the presence openings in façades such as windows and doors, represent an intermediate output [1]. However, these maps often fall short of completeness due to insufficient coverage or occlusions caused by objects such as vegetation. This research delves into the exploration of deep learning strategies to address this limitation by inpainting unseen façade objects into the 2D conflict maps. The central focus of this study...     »