Time-of-flight 3D Cameras

Time-of-flight range sensors with on-chip continuous-wave correlation of radio frequency modulated signals are increasingly popular. These 3D cameras, which are mostly known under the brand names Swiss Ranger or Photonic Mixer Device (PMD), have the potential to be applied to many problems such as obstacle detection, industrial surveillance or human-computer interaction.
They simultaneously deliver depth maps and intensity images with noise and systematic errors that are unique for this particular kind of data. We investigate the various error sources and try to develop techniques for enhancing the quality of the depth images.


In [1] and [4] we provide an analysis of the range measurement process. In [1] we show that the range follows a (quantized) offset-normal distribution and derive the influence of the signal amplitude to the reliability of the range estimates.
Using this insight on the quality of the range data, we develop a fast adaptive filtering algorithm to enhance the 3D maps [2]. Our filter is a weighted normalized convolution that adapts its kernel size to the local quality of the pixels. The signal amplitude serves as a confidence measure. Our code is available here.

In [3], we use a 3D camera to monitor the operation of a robot arm. After learning the correct operation of the robot from a few unperturbed cycles, our method is able to detect abnormal states of the machine such as, for instance, obstacles in the operation space. Click here (1M/6M/19M) to see more.


Relevant publications:
  1. Mario Frank, Matthias Plaue, Holger Rapp, Ullrich Köthe, Bernd Jähne and Fred A. Hamprecht. "Theoretical and experimental error analysis of continuous-wave time-of-flight range cameras". In Optical Engineering (Vol. 48, No. 1), SPIE 2009.
    [ bib | pdf | doi | abstract ]
  2. Mario Frank, Matthias Plaue and Fred A. Hamprecht. "Denoising of continuous-wave time-of-flight depth images using confidence measures". In Optical Engineering (Vol. 48, No. 7), SPIE 2009.
    [ bib | pdf | code | doi | abstract ]
  3. Mario Frank and Fred A. Hamprecht. "Image-Based Supervision of a Periodically Working Machine". In Pattern Analysis and Applications, Springer 2011.
    [ bib | pdf | abstract | video | doi ]
  4. Holger Rapp, Mario Frank, Fred A. Hamprecht and Bernd Jähne. "A theoretical and experimental investigation of the systematic errors and statistical uncertainties of Time-Of-Flight-cameras". In International Journal of Intelligent Systems Technologies and Applications (Vol. 5, No. 3/4), pages 402-413, Inderscience 2008.
    [ doi ]