• Author: Vadim Pisarevsky
• Link:The feature request
• Status:Draft
• Platforms:All
• Complexity: a few man-months

In opencv_contrib there isrgbd module that does some point cloud processing. Besides, there is well-knownPoint Cloud Library under BSD license, which contains a lot of good stuff that can be ported to OpenCV. While we do not need to move every single piece of the code around the world into OpenCV, it makes sense to build a better foundation for 3D vision and robotics in OpenCV.

Create a dedicatedpclouds module in the main repository with some assorted algorithms fromrgbd,calib3d and maybePCL. Some initial set of functionality may include:

• algorithm to compute surface normals
• RGBD/visual odometry (compute camera trajectory based on the sequence of RGBD frames it observes)
• registration of (establishing correspondence between) RGB data and depth data coming from different sensors
• 3D filtering (smooth surfaces)
• 3D feature detection (corners)
• maybe kinect fusion (?)
• maybe DTAM
• ICP or such
• surface registration (there is dedicated module in opencv_contrib, but it implements a patented algorithm, maybe there are non-patented alternatives)
• 3D shape fitting
• structure from motion? (there is dedicatedsfm module in opencv_contrib for that)
• building blocks for SLAM (or make it a separate module?)
• ...

Overall, this is a loosely covered part of CV in OpenCV, so it's unlikely that many people use the existingrgbd module.

Perhaps, it should be more than 1 module and perhapssfm and some other parts should stay in opencv_contrib for now.

TBD