Shape Reconstruction from Shadows and Reflections

Citation

Savarese, Silvio (2005) Shape Reconstruction from Shadows and Reflections. Dissertation (Ph.D.), California Institute of Technology. doi:10.7907/FH0V-3M10. https://resolver.caltech.edu/CaltechETD:etd-05242005-162056

Abstract

Measuring automatically the shape of physical objects in order to obtain corresponding digital models has become a useful, often indispensable, tool in design, engineering, art conservation, computer graphics, medicine and science. Machine vision has proven to be more appealing than competing technologies. Ideally, we would like to be able to acquire digital models of generic objects by simply walking around the scene, while filming with a handheld camcorder. Thus, one of the main challenges in modern machine vision is to develop algorithms that: i) are inexpensive, fast and accurate; ii) can handle objects with arbitrary appearance properties and shape; and iii) need little or no user intervention.

In this thesis, we address both issues. In the first part, we present a novel 3D reconstruction technique which makes use of minimal and inexpensive equipment. We call this technique "shadow carving". We explore the information contained in the shadows that an object casts upon itself. An algorithm is provided that makes use of this information. The algorithm iteratively recovers an estimate of the object which i) approximates the object’s shape more and more closely; and ii) is provably an upper bound to the object's shape. Shadow carving is the first technique to incorporate "shadow" information in a multi-view shape recovery framework. We have implemented our approach in a simple table-top system and validated our algorithm by recovering the shape of real objects.

It is well known that vision-based 3D scanning systems handle specular or highly reflective surfaces only poorly. The cause of this deficiency is most likely not intrinsic, but rather due to our lack of understanding of the relevant cues. In the second part of this thesis, we focus on how to promote mirror reflections from "noise" to "signal". We first present a geometrical and algebraic characterization of how a patch of the scene is mapped into an image by a mirror surface of given shape. We then develop solutions to the inverse problem of deriving surface shape from mirror reflections in a single image. We validate our theoretical results with both numerical simulations and experiments with real surfaces.

A third goal of this thesis is advancing our understanding of human perception of shape from reflections. Although the idea of perception of shape from different visual cues (e.g., shading, texture, etc.) has been extensively discussed in the past, little is known to what extent highlights and specular reflections carry useful information for shape perception. We use psychophysics to study this capability. Our goal is to provide a benchmark, as well as inspire possible technical approaches, for our computational work. We find that surprisingly, humans are very poor at judging the shape of mirror surfaces when additional visual cues (i.e., contour, shading, stereo, texture) are not visible.

Thesis Files