Physically-inspired Deep Light Estimation from a Homogeneous-Material Object for Mixed Reality Lighting

Cited 0 time in webofscience Cited 0 time in scopus
  • Hit : 152
  • Download : 0
In mixed reality (MR), augmenting virtual objects consistently with real-world illumination is one of the key factors that provide a realistic and immersive user experience. For this purpose, we propose a novel deep learning-based method to estimate high dynamic range (HDR) illumination from a single RGB image of a reference object. To obtain illumination of a current scene, previous approaches inserted a special camera in that scene, which may interfere with user's immersion, or they analyzed reflected radiances from a passive light probe with a specific type of materials or a known shape. The proposed method does not require any additional gadgets or strong prior cues, and aims to predict illumination from a single image of an observed object with a wide range of homogeneous materials and shapes. To effectively solve this ill-posed inverse rendering problem, three sequential deep neural networks are employed based on a physically-inspired design. These networks perform end-to-end regression to gradually decrease dependency on the material and shape. To cover various conditions, the proposed networks are trained on a large synthetic dataset generated by physically-based rendering. Finally, the reconstructed HDR illumination enables realistic image-based lighting of virtual objects in MR. Experimental results demonstrate the effectiveness of this approach compared against state-of-the-art methods. The paper also suggests some interesting MR applications in indoor and outdoor scenes.
Publisher
IEEE COMPUTER SOC
Issue Date
2020-03
Language
English
Article Type
Article; Proceedings Paper
Citation

IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, v.26, no.5, pp.2002 - 2011

ISSN
1077-2626
DOI
10.1109/TVCG.2020.2973050
URI
http://hdl.handle.net/10203/273944
Appears in Collection
CS-Journal Papers(저널논문)GCT-Journal Papers(저널논문)
Files in This Item
There are no files associated with this item.

qr_code

  • mendeley

    citeulike


rss_1.0 rss_2.0 atom_1.0