System information (version)

• OpenCV = 3.2
• Operating System / Platform => Linux 64 bit
• Compiler = GCC

Detailed description

I found out that with certain input interest points, the fundamental matrix output by OpenCV is wrong. I used a set with many hundreds of interest points, and with both RANSAC and the LMEDS outlier removal option with various outlier thresholds. It was consistently giving the wrong results.

Wiggling those interest points by adding random noise made it give the right result, but that is not a good solution.

This appears to be a long-standing problem, described in the documentation at:

https://docs.opencv.org/master/da/de9/tutorial_py_epipolar_geometry.html

and with no good solution there.

The solution which worked for me was based on the following observation. If the fundamental matrix is found correctly using given interest points, and the two images are rectified using that matrix, for example using stereoRectifyUncalibrated(), then the y component of the disparity between the rectified images must be close to 0. That is the same as saying that if the rectification matrices are applied to the interest points, each pair of transformed interest points must have the y difference very close to 0.

Based on this, I implemented a RANSAC algorithm for finding the fundamental matrix where the error metric is precisely the above. If the fundamental matrix is F, and the rectification matrices are H1 and H2, then for given interest point matches P1 and P2, the error is abs( (H1P1).y - (H2P2).y ).

The existing code in the OpenCV repository uses RANSAC too, but I think its error metric is not as good. I found the code here:

It seems to encode the fundamental matrix property m1 * F * m2 = 0, where m1 and m2 are the interest points, and I think this is not robust enough.

Any thoughts?