Application of Deep Learning for 3D building generalization

Abstract. The generalization of 3D buildings is a challenging task, which needs to consider geometry information, semantic content and topology relations of 3D buildings. Although many algorithms with detailed and reasonable designs have been developed for the 3D building generalization, there are still cases that could be further studied. As a fast-growing technique, Deep Learning has shown its ability to build complex concepts out of simpler concepts in many fields. Therefore, in this paper, Deep Learning is used to solve the regression (generalization of individual 3D building) and classification problems (classification of roof type) simultaneously. Firstly, the test dataset is generated and labelled with the generalization results as well as the classification of roof types. Buildings with saddleback, half-hip, and hip roof are selected as the experimental objects since their generalization results can be uniformly represented by a common vector which aims to meet the compatible representation of Deep Learning. Then, the pre-trained ResNet50 is used as the baseline network. The optimal model capacity is searched within an extensive ablation study in the framework of the building generalization problem. After that, a multi-task network is built by adding a branch of classification to the above network, which is in parallel with the generalization branch. In the process of training, the imbalance problems of tasks and classes are solved by adjusting their donations to the total loss function. It is found that less error rate is obtained after adding a classification branch. For the final results, two improved metrics are used to evaluate the generalization performance. The accuracy of generalization reached over 95% for horizontal information and 85% for height, while the accuracy of classification reached 100% on the test dataset.