"Appearance-from-Motion: Recovering Spatially Varying Surface Reflectance under Unknown Lighting"
Yue Dong, Guojun Chen, Pieter Peers, Jiawan Zhang, and Xin Tong

ACM Transactions on Graphics, Volume 33, Issue 6, Article 193, December 2014
Abstract
We present "appearance-from-motion", a novel method for recovering the spatially varying isotropic surface reflectance from a video of a rotating subject, with known geometry, under unknown natural illumination. We formulate the appearance recovery as an iterative process that alternates between estimating surface reflectance and estimating incident lighting. We characterize the surface reflectance by a data-driven microfacet model, and recover the microfacet normal distribution for each surface point separately from temporal changes in the observed radiance. To regularize the recovery of the incident lighting, we rely on the observation that natural lighting is sparse in the gradient domain. Furthermore, we exploit the sparsity of strong edges in the incident lighting to improve the robustness of the surface reflectance estimation. We demonstrate robust recovery of spatially varying isotropic reflectance from captured video as well as an internet video sequence for a wide variety of materials and natural lighting conditions.


Download
Supplementary Material
Bibtex
@article{Dong:2014:AFM,
author = {Dong, Yue and Chen, Guojun and Peers, Pieter and Zhang, Jiawan and Tong, Xin},
title = {Appearance-from-Motion: Recovering Spatially Varying Surface Reflectance under Unknown Lighting},
month = {December},
year = {2014},
articleno = {193},
journal = {ACM Transactions on Graphics},
volume = {33},
number = {6},
doi = {http://dx.doi.org/10.1145/2661229.2661283},
}