"Relighting Neural Radiance Fields with Shadow and Highlight Hints"
ACM SIGGRAPH 2023 Conference Proceedings, Article 73, August 2023
Abstract
This paper presents a novel neural implicit radiance representation for free viewpoint relighting from a small set of unstructured photographs of an object lit by a moving point light source different from the view position. We express the shape as a signed distance function modeled by a multi layer perceptron. In contrast to prior relightable implicit neural representations, we do not disentangle the different light transport components, but model both the local and global light transport at each point by a second multi layer perceptron that, in addition, to density features, the current position, the normal (from the signed distance function), view direction, and light position, also takes shadow and highlight hints to aid the network in modeling the corresponding high frequency light transport effects. These hints are provided as a suggestion, and we leave it up to the network to decide how to incorporate these in the final relit result. We demonstrate and validate our neural implicit representation on synthetic and real scenes exhibiting a wide variety of shapes, material properties, and global illumination light transport.
Download
Supplementary Material
Code
Bibtex
ACM SIGGRAPH 2023 Conference Proceedings, Article 73, August 2023
AbstractThis paper presents a novel neural implicit radiance representation for free viewpoint relighting from a small set of unstructured photographs of an object lit by a moving point light source different from the view position. We express the shape as a signed distance function modeled by a multi layer perceptron. In contrast to prior relightable implicit neural representations, we do not disentangle the different light transport components, but model both the local and global light transport at each point by a second multi layer perceptron that, in addition, to density features, the current position, the normal (from the signed distance function), view direction, and light position, also takes shadow and highlight hints to aid the network in modeling the corresponding high frequency light transport effects. These hints are provided as a suggestion, and we leave it up to the network to decide how to incorporate these in the final relit result. We demonstrate and validate our neural implicit representation on synthetic and real scenes exhibiting a wide variety of shapes, material properties, and global illumination light transport.
Download
- Zeng_SIGGRAPH2023.pdf (13.97Mb)
Supplementary Material
- Document: Zeng_SIGGRAPH2023_supplemental.pdf (2.61Mb)
- Video: Zeng_SIGGRAPH2023_video.mp4 (227.81Mb)
- ArXiv version (arxiv)
Code
- Code Repository (github)
Bibtex
@conference{Zeng:2023:RNR,
author = {Zeng, Chong and Chen, Guojun and Dong, Yue and Peers, Pieter and Wu, Hongzhu and Tong, Xin},
title = {Relighting Neural Radiance Fields with Shadow and Highlight Hints},
month = {August},
year = {2023},
articleno = {73},
booktitle = {ACM SIGGRAPH 2023 Conference Proceedings},
doi = {https://doi.org/10.1145/3588432.3591482},
}
author = {Zeng, Chong and Chen, Guojun and Dong, Yue and Peers, Pieter and Wu, Hongzhu and Tong, Xin},
title = {Relighting Neural Radiance Fields with Shadow and Highlight Hints},
month = {August},
year = {2023},
articleno = {73},
booktitle = {ACM SIGGRAPH 2023 Conference Proceedings},
doi = {https://doi.org/10.1145/3588432.3591482},
}