A benchmark collection for computer vision and 3d reconstruction of real underwater scenarios
Benjamin GEHMLICH 1 | Marco BLOCK-BERLITZ1 | Dennis WITTCHEN1 | Benjamin DUCKE2
(1 HTW Dresden, Dresden, Germany | 2 German Archaeological Institute, Berlin, Germany)
Keywords: 3d reconstruction, computer vision, underwater archaeology, videogrammetry, benchmark
Often, the drivers of scientific research are improvements to existing technical solutions, rather than new solutions. On the one hand, the use of acknowledged benchmark tests with high diversity are necessary to fairly compare current and new approaches. On the other hand, only open access benchmark databases can open up the research field to more people.
Existing ground truth datasets for trajectory or two-view disparity estimation focused on real-time 3d reconstruction are available (Tsukaba , TUM RGB-D , KITTI  or NYU Dataset ) and widely used, e.g. to compare different SLAM (simultaneous localization and mapping) algorithms. In case of 3d reconstruction in underwater scenarios, no benchmark database for academic research is freely available. The Archaeoteam of HTW Dresden, in cooperation with Freie Universität Berlin and the German Archaeological Institute, wants to fill this gap with ARCHAEO-BASE and collects useful datasets (photo sets as well as videos) from several campaigns with high diversity in camera systems used, recorded structures and quality of water. To be oriented towards real-world scenarios, the benchmark represents a broad range of perfect to really challenging image sets. The benchmark allows researchers to test and compare algorithms under realistic and reproducible conditions. The main aspects are repeatability, independency, and unambiguity.
Beside original videos and photo sets, the benchmark database ARCHAEO-BASE includes results of our own solutions for undistortion, keyframe selection, image enhancement, feature extraction and 3d reconstruction. This allows to open up the field of research and supports the improvement of current solutions. We still work on a synthetic image and video database (like VaFRIC  for evaluation of visual odometry, 3D reconstruction and SLAM algorithms) including noise and physical interaction (absorption and reflection) between particles and light sources. For evaluating the accuracy of a 3d reconstructions framework, the free software CloudCompare  can be used to align the given ground truth model with the reconstructed model. ARCHAEO-BASE is freely available at the project webpage .
Relevance conference / Relevance session:
Archaeo-Base is a novum for underwater archaeology.
No benchmark for computer vision and 3d reconstruction for underwater scenarios is available.
- Peris M., Martull S., Maki A., Ohkawa Y., Fukui K.: “Towards a simulation driven stereo vision system”, Pattern Recognition (ICPR 2012), 21st International Conference on, pp. 1038–1042, IEEE, 2012
- Sturm J., Engelhard N., Endres F., Burgard W., Cremers D.: “A benchmark for RGB-D SLAM evaluation”, Proceedings of the IEEE/RSJ Conference on Intelligent Robots and Systems (IROS), 2012
- Geiger A., Lenz P., Urtasun R.: “Are we ready for autonomous driving? the KITTI vision benchmark suite”, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3354–3361, IEEE, 2012
- Silberman N., Hoiem D., Kohli P., Fergus R.: “Indoor segmentation and support inference from RGBD images”, Computer Vision (ECCV 2012), 2012
- Handa A., Whelan T., McDonald J., Davison A.J.: “A Benchmark for RGB-D Visual Odometry, 3D Reconstruction and SLAM”, IEEE Intl. Conf. on Robotics and Automation, ICRA, 2014