"ScribbleLight: Single Image Indoor Relighting with Scribbles"
CoRR, abs/2411.17696, November 2024
Abstract
Image-based relighting of indoor rooms creates an immersive virtual understanding of the space, which is useful for interior design, virtual staging, and real estate. Relighting indoor rooms from a single image is especially challenging due to complex illumination interactions between multiple lights and cluttered objects featuring a large variety in geometrical and material complexity. Recently, generative models have been successfully applied to image-based relighting conditioned on a target image or a latent code, albeit without detailed local lighting control. In this paper, we introduce ScribbleLight, a generative model that supports local fine-grained control of lighting effects through scribbles that describe changes in lighting. Our key technical novelty is an Albedo-conditioned Stable Image Diffusion model that preserves the intrinsic color and texture of the original image after relighting and an encoder-decoder-based ControlNet architecture that enables geometry-preserving lighting effects with normal map and scribble annotations. We demonstrate ScribbleLight's ability to create different lighting effects (e.g., turning lights on/off, adding highlights, cast shadows, or indirect lighting from unseen lights) from sparse scribble annotations.
Download
Supplementary Material
Related Publications
Bibtex
CoRR, abs/2411.17696, November 2024
AbstractImage-based relighting of indoor rooms creates an immersive virtual understanding of the space, which is useful for interior design, virtual staging, and real estate. Relighting indoor rooms from a single image is especially challenging due to complex illumination interactions between multiple lights and cluttered objects featuring a large variety in geometrical and material complexity. Recently, generative models have been successfully applied to image-based relighting conditioned on a target image or a latent code, albeit without detailed local lighting control. In this paper, we introduce ScribbleLight, a generative model that supports local fine-grained control of lighting effects through scribbles that describe changes in lighting. Our key technical novelty is an Albedo-conditioned Stable Image Diffusion model that preserves the intrinsic color and texture of the original image after relighting and an encoder-decoder-based ControlNet architecture that enables geometry-preserving lighting effects with normal map and scribble annotations. We demonstrate ScribbleLight's ability to create different lighting effects (e.g., turning lights on/off, adding highlights, cast shadows, or indirect lighting from unseen lights) from sparse scribble annotations.
Download
- Choi_ARXIV2024.pdf (19.03Mb)
Supplementary Material
- External Supplemental Material (link)
Related Publications
-
"ScribbleLight: Single Image Indoor Relighting with Scribbles",
IEEE International Conference on Computer Vision and Pattern Recognition,
Nashville (TN), USA,
June 2025,
@misc{Choi:2024:SLS,
author = {Choi, Jun Myeong and Wang, Annie and Peers, Pieter and Bhattad, Anand and Sengupta, Roni},
title = {ScribbleLight: Single Image Indoor Relighting with Scribbles},
month = {November},
year = {2024},
howpublished = {CoRR, abs/2411.17696},
doi = {https://arxiv.org/abs/2411.17696},
}
author = {Choi, Jun Myeong and Wang, Annie and Peers, Pieter and Bhattad, Anand and Sengupta, Roni},
title = {ScribbleLight: Single Image Indoor Relighting with Scribbles},
month = {November},
year = {2024},
howpublished = {CoRR, abs/2411.17696},
doi = {https://arxiv.org/abs/2411.17696},
}