Finding matching points in stereo images for 3D position calculation
10 views (last 30 days)
Show older comments
I am a student with limited knowledge on computer vision currently developing a simple 3D tracker for flies using a fully calibrated stereoscopic camera setup as a project at a group in my university.
The 2D detections of the flies in the individual frames are not a problem, I've got a pretty solid handle on that. The calculation of the 3D position based on matching 2D points in the respective camera frames is also not a problem, I've tested that. To get a better idea of the data I'm working with, here's a frame of cam1 and cam2 with lots of walking and one flying fly (marked red).

My current problem is identifying which of the 2D detections in the camera frames are the same fly, i.e. which of the points are matching as there may be multiple flies in the same frame. The simple (non-)solution is to take all combinations, pretend they're all valid points and hope that the following tracking algorithm takes care of all the false-positives. This is a waste of resources though as only min(m,n) 3D points are actually valid, and m*n points are generated with this approach. So I'd like to either perfectly determine matching 2D points or at least minimize the false positives.
For the latter approach of minimizing false positives, given that I have a fully calibrated stereo system with intrinsic and extrinsic data I figured I could take the detections of say cam1 and for each of them project a ray / line / curve across the frame of cam2 where the matching point ought to be on or close to said line. DLTdv uses such an approach:

Does Matlab offer a similar functionality, or does anyone know how to go about to implement such an approach? A cursory search through the documentation led to no results.
During the search I have found the functionality to rectify images in Matlab, whereupon the same features are along the same row, thus making it fairly simple to reduce the search space for matching 2D points:

I would like to avoid having to rectify all the image pairs as it is computationally expensive and I'd have to rewrite all my 2D detection code to work with the rectified images, so is there a possibility to take either distorted or undistorted image coordinates and transform them into this rectified image space instead? That way I'd only have to check if the y coordinate of candidates is say, within 5px of the point I'm looking at to find potential matched pairs and cut down the number of false-positives.
Finally, if there is an even better approach available I'd be happy to hear about that, too. Note though that the cameras do not cover the entire space and thus flys may appear or disappear and cannot be used as a constant feature.
0 Comments
Answers (0)
See Also
Categories
Find more on MATLAB Support Package for USB Webcams in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!