FFT-based Appearance Adaptation for the Semantic Segmentation of Small-scale Aerial Image Datasets

Modern semantic segmentation models, which enable useful applications based on aerial image analysis such as tracking deforestation of the Amazon rainforest over time, rely on access to large amounts of training data. However, due to the cost of manually labeling data, segmentation datasets, including aerial image datasets, are often relatively small. In many cases, no labels are available at all and domain adaptation from a related domain has to be performed. Since many domain adaptation approaches are specialized for the adaptation of street scene segmentation, we propose an alternate domain adaptation pipeline taking the properties of aerial image datasets into account. We propose a novel appearance adaptation technique using statistics (mean and covariance) of coefficients in the frequency domain and transferring the statistics from one domain to another to copy its look/style. In an experiment with 30 domain pairs, we achieved an average improvement of 2.5% mean F1 score. We improve upon the state of the art by 13.3%.