%% Image Rectification
% Rectification is the process of transforming stereo images, such that the
% corresponding points have the same row coordinates in the two images. It 
% is a useful procedure in stereo vision, as the 2-D stereo correspondence
% problem is reduced to a 1-D problem when rectified image pairs are used.
% Image rectification is a prerequisite step for the 
% <videostereo.html Stereo Vision Demo>.
%
% This demo uses the Computer Vision System Toolbox(TM) to compute the
% rectification of two uncalibrated images where the camera intrinsics are
% unknown. 
%
%  Copyright 2004-2010 The MathWorks, Inc.
%  $Revision: 1.1.6.5 $Date: 2011/05/09 03:44:01 $

%% Step 1. Read Stereo Image Pair
% Here we read in two color images of the same scene taken from different
% positions. We convert them to grayscale since colors are not necessary
% for the matching process. 
%
% Below we show both images side by side, and we produce a color composite
% to demonstrate the pixel-wise differences between them. There is an
% obvious offset between the images in both dimensions. The goal of
% rectification is to transform the images, aligning them such that
% corresponding points in each image will appear on the same row.

I1 = im2double(rgb2gray(imread('yellowstone_left.png')));
I2 = im2double(rgb2gray(imread('yellowstone_right.png')));

cvexShowImagePair(I1, I2, 'Left image', 'Right image');
cvexShowMatches(I1, I2, [], [], 'Left image', 'Right image');

%% Step 2. Collect Salient Points from Each Image
% The rectification process requires a set of point correspondences between
% the two images. To generate these correspondences, we collect points of 
% interest from both images, then choose potential matches between them.
%
% In this step we produce these candidate points for each image. To have
% the best chance that these points will have correspondences in the other
% image, we want points around salient image features such as corners. We
% use the |CornerDetector| System object with its default detection method
% of Harris corners. The detected points in both images are shown below.
% Observe how many of them cover the same image features, as desired.

hcornerdet = vision.CornerDetector('MaximumCornerCount', 500, ...
    'CornerThreshold', 0.001, 'NeighborhoodSize', [21 21]);
  
cornerPoints1 = step(hcornerdet, I1);

cornerPoints2 = step(hcornerdet, I2);

cvexShowImagePair(I1, I2, 'Corners in left image', ...
    'Corners in right image', 'SingleColor', cornerPoints1, cornerPoints2);

%% Step 3. Select Correspondences Between Points
% Next we determine correspondences between the left and right image
% points. We use the |extractFeatures| and |matchFeatures| functions for
% this purpose. For each point, we extract a 9-by-9 block from the image
% centered around it. We use the normalized cross-correlation (normxcorr)
% between these image regions as the matching cost between points. Each
% point in either image is matched with the points in the other image with
% the lowest cost. There is no uniqueness constraint, so points from either
% images can participate in multiple correspondences.

[features1, validPoints1] = extractFeatures(I1, cornerPoints1, ...
    'BlockSize', 9);
[features2, validPoints2] = extractFeatures(I2, cornerPoints2, ...
    'BlockSize', 9);

[indexPairs, match_metric] = matchFeatures(features1, features2, ...
    'Metric', 'normxcorr', 'MatchThreshold', 5);

% Retrieve locations of matched points for each image
matchedPoints1 = validPoints1(indexPairs(:, 1), :);
matchedPoints2 = validPoints2(indexPairs(:, 2), :);

%%
% The image below is another color composite. Points from the left image
% are shown as red, and points from the right image are shown as green.
% Yellow lines are drawn between the points to show the correspondences
% selected by this alignment procedure. Most of these are correct, but
% there are also some outliers.

cvexShowMatches(I1, I2, matchedPoints1, matchedPoints2, ...
  'Matched points in I1', 'Matched points in I2');

%% Step 4. Remove Outliers Using Random Sample Consensus (RANSAC) Algorithm
% Some of the point correspondences obtained in the previous step are
% incorrect. We cannot proceed to rectification with erroneous point
% correspondences as above, so we must first discard bad matches (i.e.,
% outliers). A standard way to do this is with the Random Sample Consensus
% (RANSAC) algorithm [2,3,4]. We use the |estimateFundamentalMatrix|
% function with the Method set to 'RANSAC'.

[fMatrix, inliers] = estimateFundamentalMatrix(...
    matchedPoints1, matchedPoints2, 'Method', 'RANSAC', ...
    'NumTrials', 1000, 'DistanceThreshold', 0.1, 'Confidence', 99.99);

inlierPoints1 = matchedPoints1(inliers, :);
inlierPoints2 = matchedPoints2(inliers, :);

cvexShowMatches(I1, I2, inlierPoints1, inlierPoints2, ...
    'Inlier points in I1', 'Inlier points in I2');

%% Step 5. Rectify Images
% The rectification procedure transforms the images such that the
% corresponding points will appear on the same rows in both images. We use
% the |estimateUncalibratedRectification| function to compute the
% rectification transformation matrix for each of the images. We then use
% the |cvexShowMatches| and |cvexShowStereoImages| functions to apply the
% rectification transformations and to display the images.

if isEpipoleInImage(fMatrix, size(I1)) ...
    || isEpipoleInImage(fMatrix', size(I2))
    error(['For the rectification to succeed, the epipoles must be ',...
        'outside the images.']);
end

[t1, t2] = estimateUncalibratedRectification(fMatrix, ...
    inlierPoints1, inlierPoints2, size(I2));

%%
% Rectify the images using projective transformations, t1 and t2, and show
% a color composite of the rectified images demonstrating point
% correspondences.

cvexShowMatches(I1, I2, inlierPoints1, inlierPoints2, ...
    'Inlier points in I1', 'Inlier points in I2', 'RC', t1, t2);

%%
% Show the rectified images and the epipolar lines corresponding to the
% features points used during the rectification process. Note that the
% corresponding lines are in the same rows demonstrating that the
% rectification was successful.

cvexShowStereoImages('Rectified image 1', 'Rectified image 2',...
    I1, I2, inlierPoints1, inlierPoints2, fMatrix, t1, t2);

%% References
% [1] Trucco, E; Verri, A. "Introductory Techniques for 3-D Computer Vision."
% Prentice Hall, 1998.
% 
% [2] Hartley, R; Zisserman, A. "Multiple View Geometry in Computer Vision."
% Cambridge University Press, 2003.
% 
% [3] Hartley, R. "In Defense of the Eight-Point Algorithm." IEEE(R)
% Transactions on Pattern Analysis and Machine Intelligence, v.19 n.6, June
% 1997.
%
% [4] Fischler, MA; Bolles, RC. "Random Sample Consensus: A Paradigm for
% Model Fitting with Applications to Image Analysis and Automated
% Cartography." Comm. Of the ACM 24, June 1981.


displayEndOfDemoMessage(mfilename)
