Scribble-Based Weakly Supervised Deep Learning for Road Surface Extraction from Remote Sensing Images

Overview

We propose a novel scribble-based weakly supervised deep learning approach (called ScRoadExtractor) for road surface extraction from remote sensing images under the weak supervision of centerline-like scribble annotations.

• A road label propagation algorithm is proposed to propagate the semantic information from scribbles to unlabeled pixels by utilizing both the buffer-based prop­erties of roads and the local and global dependencies between graph nodes built on super-pixels to generate the proposal mask.
• We design a dual-branch encoder-decoder network (DBNet), which is trained with the proposal mask and boundary prior information detected from images, and outputs a road surface segmentation map that approaches to a map predicted from a densely supervised method.
• The experimental results on diverse road data sets across the world demonstrate that our approach possesses high-performance and powerful generalization ability and also outperforms state-of-the-art scribble-supervised segmen­tation methods.

At present, very few works have explored weakly supervised semantic segmentation for extracting road surfaces from remote sensing images. Our method is a step on a journey that will ultimately bring us closer to automatic road extraction from remote sensing images with very little manual annotating required.

If you find it is helpful in your research, cite our paper (Wei, Y., & Ji, S. Scribble-Based Weakly Supervised Deep Learning for Road Surface Extraction from Remote Sensing Images. IEEE Transactions on Geoscience and Remote Sensing 2021). Please note that the code can be used in any academic study however the commercial application is not allowed as the system is protected by a Chinese Patent with Number 202010771919.6 as well as a software copyright with Number 2020SR1190370.

Code

Download the network code. Click HERE to Download

Dependencies

• Python == 3.6.7
• PyTorch == 1.1.0
• OpenCV == 4.1.0
• Numpy == 1.16.3
• PyMaxflow == 1.2.12
• Scipy == 1.2.1

Usage

• 1. Download dataset and prepare for the code

The scribbles can be obtained from OpenStreetMap centerlines, GPS traces, or manually annotation through ArcGIS or other software. Also, you can generate skeletonized road lines by thinning road segmentation maps (skimage.morphology.thin).

With respect to the implementation of HED Boundary detector, you can refer to the folder boundary_detect. To generate HED masks, download the pre-trained model [network-bsds500.pytorch] by download.bash and run boundary_detect/run.py. We also provide the pre-trained model [network-bsds500.pytorch] using the link below. https://pan.baidu.com/s/1AMNnmo7YAk1X3_m8Ky1arw (pwd:0HED)

• 2. Road label propagation

Run road_label_propagation.py to derive proposal masks.

• 3. DBNet

Run DBNet/train.py for training and run DBNet/test.py for testing.

Modify the parameters before running. More details, please contact weiyao@whu.edu.cn.