Ordinary videos capture a surprising amount of hidden, visually imperceptible information. For instance, videos of peoples' faces may capture color changes in the skin and artery motion from heartbeats, while videos of mechanical systems can capture subtle vibrations indicating imminent failure. Algorithms can extract and exaggerate these signals for visualization on top of the original videos. In particular, Eulerian magnification algorithms sidestep the need to track hidden motions directly and instead devise multiscale bandpass filters to amplify signals in local spatial regions. In this work, we extend these techniques beyond color videos to geometric video data captured by 3D depth cameras such as the Microsoft Kinect. In our framework, we can spatially amplify a "bulging of the neck" during a heartbeat or the expansion of a chest/abdomen during a breath. We then exaggerate and display these signals as evolving 3D shapes. We explore a pipeline based on an implicit surface representation, in which we reconstruct the object in 3D space using multiple camera angles captured by one camera moving around the subject. We discuss the merits, drawbacks, and challenges of this representation compared to ordinary color videos.
Boccuto, Thomas; Kutai, Seraiah; Mosley, Kristen; and Kirk, Samuel, "Amplification of Hidden Periodic Motions in 3D Videos" (2021). Mathematics Summer Fellows. 14.
Available for download on Sunday, May 01, 2022
Available to all.