Dense prediction of label noise for learning building extraction from aerial drone imagery

Label noise is a commonly encountered problem in learning building extraction tasks; its presence can reduce performance and increase learning complexity. This is especially true for cases where high resolution aerial drone imagery is used, as the labels may not perfectly correspond/align with the actual objects in the imagery. In general machine learning and computer vision context, labels refer to the associated class of data, and in remote sensing-based building extraction refer to pixel-level classes. Dense label noise in building extraction tasks has rarely been formalized and assessed. We formulate a taxonomy of label noise models for building extraction tasks, which incorporates both pixel-wise and dense models. While learning dense prediction under label noise, the differences between the ground truth clean label and observed noisy label can be encoded by error matrices indicating locations and type of noisy pixel-level labels. In this work, we explicitly learn to approximate error matrices for improving building extraction performance; essentially, learning dense prediction of label noise as a subtask of a larger building extraction task. We propose two new model frameworks for learning building extraction under dense real-world label noise, and consequently two new network architectures, which approximate the error matrices as intermediate predictions. The first model learns the general error matrix as an intermediate step and the second model learns the false positive and false-negative error matrices independently, as intermediate steps. Approximating intermediate error matrices can generate label noise saliency maps, for identifying labels having higher chances of being mis-labelled. We have used ultra-high-resolution aerial images, noisy observed labels from OpenStreetMap, and clean labels obtained after careful annotation by the authors. When compared to the baseline model trained and tested using clean labels, our intermediate false positive-false negative error matrix model provides Intersection-Over-Union gain of 2.74% and F1-score gain of 1.75% on the independent test set. Furthermore, our proposed models provide much higher recall than currently used deep learning models for building extraction, while providing comparable precision. We show that intermediate false positive-false negative error matrix approximation can improve performance under label noise.