Open videotrafficof.m in the Editor
Run in the Command Window

# Tracking Cars Using Optical Flow

This demo uses System objects to illustrate how to detect and track cars in a video sequence using optical flow estimation.

## Contents

- Introduction
- Initialization
- Stream Processing Loop
- Release
- Summary

## Introduction

The demo uses optical flow estimation technique to estimate the motion vectors in each frame of the video sequence. By thresholding and performing morphological closing on the motion vectors, the binary feature images are produced. In this demo, you locate the cars in each binary feature image using blob analysis. Then you draw a rectangle around the cars that pass beneath a reference line. You use a counter to track the number of cars in the region of interest.

## Initialization

Use these next sections of code to initialize the required variables and System objects.

Create a System object to read video from avi file.

    filename = 'viptraffic.avi';
    hvfr = vision.VideoFileReader(filename, ...
                                  'ImageColorSpace', 'RGB',...
                                  'VideoOutputDataType', 'single');

Create color space converter System object to convert the image from RGB to intensity format.

    hcsc = vision.ColorSpaceConverter('Conversion', 'RGB to intensity');

Create a System object to estimate direction and speed of object motion from one video frame to another using optical flow.

    hof = vision.OpticalFlow( ...
        'OutputValue', 'Horizontal and vertical components in complex form', ...
        'ReferenceFrameDelay', 3);

Create and configure two System objects, one for calculating the overall mean of the velocities and the other for calculating the running mean of the velocities.

    hmean1 = vision.Mean;
    hmean2 = vision.Mean('RunningMean', true);

Create a 2-D median filtering System object to filter out slight speckle noise introduced due to segmentation.

    hmedianfilt = vision.MedianFilter;

Create a morphological closing System object used to remove gaps within the blobs.

    hclose = vision.MorphologicalClose('Neighborhood', strel('line',5,45));

Create a blob analysis System object to segment cars in the video.

    hblob = vision.BlobAnalysis( ...
                        'CentroidOutputPort', false, ...
                        'AreaOutputPort', true, ...
                        'BoundingBoxOutputPort', true, ...
                        'OutputDataType', 'double', ...
                        'MinimumBlobAreaSource', 'Property', ...
                        'MinimumBlobArea', 250, ...
                        'MaximumBlobAreaSource', 'Property', ...
                        'MaximumBlobArea', 3600, ...
                        'MaximumCount', 80);

Create a morphological erosion System object to thin out portions of the road and objects other than cars which are also produced as a result of segmentation.

    herode = vision.MorphologicalErode('Neighborhood', strel('square',2));

Create and configure two System objects that insert shapes, one for drawing the bounding box around the cars and the other for drawing the motion vector lines.

    hshapeins1 = vision.ShapeInserter( ...
                'BorderColor', 'Custom', ...
                'CustomBorderColor', [0 1 0]);
    hshapeins2 = vision.ShapeInserter( ...
                'Shape','Lines', ...
                'BorderColor', 'Custom', ...
                'CustomBorderColor', [255 255 0]);

Create and configure a System object to write the number of cars being tracked.

    htextins = vision.TextInserter( ...
            'Text', '%4d', ...
            'Location',  [1 1], ...
            'Color', [1 1 1], ...
            'FontSize', 12);

Create System objects to display the original video, motion vector video, the thresholded video and the results.

    sz = get(0,'ScreenSize');
    pos = [20 sz(4)-300 200 200];
    hVideo1 = vision.VideoPlayer('Name','Original Video','Position',pos);
    pos(1) = pos(1)+220; % move the next viewer to the right
    hVideo2 = vision.VideoPlayer('Name','Motion Vector','Position',pos);
    pos(1) = pos(1)+220;
    hVideo3 = vision.VideoPlayer('Name','Thresholded Video','Position',pos);
    pos(1) = pos(1)+220;
    hVideo4 = vision.VideoPlayer('Name','Results','Position',pos);

    % Initialize some variables used in plotting motion vectors.
    MotionVecGain = 20;
    line_row =  22;
    borderOffset   = 5;
    decimFactorRow = 5;
    decimFactorCol = 5;
    firstTime = true;

## Stream Processing Loop

Create the processing loop to track the cars in the input video. This loop uses the System objects previously instantiated.

The loop is stopped when you reach the end of the input file, which is detected by the VideoFileReader System object.

    while ~isDone(hvfr)
        imrgb = step(hvfr);      % Read input video frame
        I = step(hcsc, imrgb);   % Convert color image to intensity
        of = step(hof, I);       % Estimate optical flow

        % Thresholding and Region Filtering.
        y1 = of .* conj(of);
        % Compute the velocity threshold from the matrix of complex velocities.
        vel_th = 0.5 * step(hmean2, step(hmean1, y1));

        % Threshold the image and then filter it to remove fine speckle noise.
        filteredout = step(hmedianfilt, y1 >= vel_th);

        % Perform erosion operation to thin-out the parts of the road followed
        % by the closing operation to remove gaps in the blobs.
        th_image = step(hclose, step(herode, filteredout));

        % Regional Filtering.

        % Estimate the area and bounding box of the blobs in the thresholded
        % image.
        [area, bbox] = step(hblob, th_image);
        % Select those boxes which are in our ROI.
        Idx = bbox(:,1) > line_row;

        % The next lines of code exclude other objects (like parts of the road)
        % which are also segmented as blobs and select only cars. When the
        % ratio between the area of the blob and the area of the bounding box
        % is above 0.4 (40%), it is considered as a car and hence the bounding
        % box for that object is used. Otherwise the bounding box is removed.
        ratio = zeros(length(Idx), 1);
        ratio(Idx) = single(area(Idx,1))./single(bbox(Idx,3).*bbox(Idx,4));
        ratiob = ratio > 0.4;
        count = int32(sum(ratiob));    % Number of cars
        bbox(~ratiob, :) = int32(-1);

        % Draw bounding rectangles around the tracked cars.
        y2 = step(hshapeins1, imrgb, bbox);

        % Display the number of cars tracked and a white line showing ROI.
        y2(22:23,:,:) = 1;             % The white line.
        y2(1:15,1:30,:) = 0;           % Background for displaying count
        image_out = step(htextins, y2, count);

        % Generate the coordinate points for plotting motion vectors.
        if firstTime
          [R C] = size(of);            % Height and width in pixels
          RV = borderOffset:decimFactorRow:(R-borderOffset);
          CV = borderOffset:decimFactorCol:(C-borderOffset);
          [Y X] = meshgrid(CV,RV);
          firstTime = false;
        end

        % Calculate and draw the motion vectors.
        tmp = of(RV,CV) .* MotionVecGain;
        lines = [Y(:), X(:), Y(:) + real(tmp(:)), X(:) + imag(tmp(:))];
        mv_video = step(hshapeins2, imrgb, lines);

        step(hVideo1, imrgb);          % Display Original Video
        step(hVideo2, mv_video);       % Display video with motion vectors
        step(hVideo3, th_image);       % Display Thresholded Video
        step(hVideo4, image_out);      % Display video with bounding boxes
    end

## Release

Here you call the release method on the System objects to close any open files and devices.

    release(hvfr);

## Summary

The output video shows the cars which were tracked by drawing boxes around them. The video also displays the number of cars being tracked.