"Multi-view Spectral Polarization Propagation for Video Glass Segmentation"
IEEE/CVF International Conference on Computer Vision (ICCV), 2023
Abstract
In this paper, we present the first polarization-guided video glass segmentation propagation solution (PGVS-Net) that can robustly and coherently propagate glass segmentation in RGB-P video sequences. By leveraging spatiotemporal polarization and color information, our method combines multi-view polarization cues and thus can alleviate the view dependence of single-input intensity variations on glass objects. We demonstrate that our model can outperform glass segmentation on RGB-only video sequences as well as produce more robust segmentation than per-frame RGB-P single-image segmentation methods. To train and validate PGVS-Net, we introduce a novel RGB-P Glass Video dataset (PGV-117) containing 117 video sequences of scenes captured with different types of camera paths, lighting conditions, dynamics, and glass types.
Download
Supplementary Material
Bibtex
IEEE/CVF International Conference on Computer Vision (ICCV), 2023
AbstractIn this paper, we present the first polarization-guided video glass segmentation propagation solution (PGVS-Net) that can robustly and coherently propagate glass segmentation in RGB-P video sequences. By leveraging spatiotemporal polarization and color information, our method combines multi-view polarization cues and thus can alleviate the view dependence of single-input intensity variations on glass objects. We demonstrate that our model can outperform glass segmentation on RGB-only video sequences as well as produce more robust segmentation than per-frame RGB-P single-image segmentation methods. To train and validate PGVS-Net, we introduce a novel RGB-P Glass Video dataset (PGV-117) containing 117 video sequences of scenes captured with different types of camera paths, lighting conditions, dynamics, and glass types.
Download
- Qiao_ICCV2023.pdf (9.42Mb)
Supplementary Material
- Document: Qiao_ICCV2023_supplemental.pdf (1.06Mb)
- Video: dynamic_door_daytime2.mp4 (13.17Mb)
- Video: dynamic_door_inclined_dusk3.mp4 (23.73Mb)
- Video: HDR_building_inclined3.mp4 (16.66Mb)
- Video: high-light_building_circle2.mp4 (16.66Mb)
- Video: outside_building_daytime_inclined2.mp4 (16.66Mb)
- External Supplemental Material (link)
Bibtex
@conference{Qiao:2023:MVS,
author = {Qiao, Yu and Dong, Bo and Jin, Ao and Fu, Yu and Beak, Seung-Hwan and Heide, Felix and Peers, Pieter and Wei, Xiaopeng and Yang, Xin},
title = {Multi-view Spectral Polarization Propagation for Video Glass Segmentation},
year = {2023},
booktitle = {IEEE/CVF International Conference on Computer Vision (ICCV)},
}
author = {Qiao, Yu and Dong, Bo and Jin, Ao and Fu, Yu and Beak, Seung-Hwan and Heide, Felix and Peers, Pieter and Wei, Xiaopeng and Yang, Xin},
title = {Multi-view Spectral Polarization Propagation for Video Glass Segmentation},
year = {2023},
booktitle = {IEEE/CVF International Conference on Computer Vision (ICCV)},
}