Optical Flow and Depth Estimation

First, computed the spatiotemporal derivatives for every pixel of the given image. Each time the directions orthogonal to the differentiation direction are smoothed. Then computed the two flow components (u, v) for each pixel as well as a confidence value.

Assumed that the optical flow is constant in a local neighbourhood of 5×5 pixels, so the flow can be computed as a 25×2 linear system consisting of 25 equations Ix*u+Iy*v+It =0 where(Ix,Iy,It) are the spatiotemporal derivatives at every pixel. The smallest singular value smin of the system is used as a confidence measure. 

Flow vectors for Threshmin = 1

Flow vectors for Threshmin = 10

Flow vectors for Threshmin = 30

After using RANSAC and thresholding the flow vectors with higher than smin confidence level, the epipoles are found.

Epipoles for Threshmin = 1

Epipoles for Threshmin = 10

Epipoles for Threshmin = 30

The epipoles are in the direction of the optical flow vectors. So, the number of inliers decreases as we increase the threshold.

Depth at every pixel for which flow exists is computed using the given pixel flow, confidence, epipole, and intrinsic parameters.

Depth for Threshmin = 1

Depth for Threshmin = 10

Depth for Threshmin = 30

Code: click here

Go back to Projects