Open videoabandonedobj.m in the Editor
Run in the Command Window

# Abandoned Object Detection

This example tracks objects at a train station and determines which ones remain stationary. Abandoned objects in public areas concern authorities since they might pose a security risk. Algorithms, such as the one used in this example, can be used to assist security officers monitoring live surveillance video by directing their attention to a potential area of interest.

This example illustrates how to use the BlobAnalysis System object to identify objects and track them. The example implements this algorithm using the following steps:

- Extract a region of interest (ROI), thus eliminating video areas that are unlikely to contain abandoned objects.
- Perform video segmentation using background subtraction.
- Calculate object statistics using the blob analysis System object.
- Track objects based on their area and centroid statistics.
- Visualize the results.

## Contents

- Initialize Required Variables and System Objects
- Video Processing Loop

## Initialize Required Variables and System Objects

Use these next sections of code to initialize the required variables and System objects.

Rectangular ROI [x y width height], where [x y] is the uppef left corner of the ROI

    roi = [100 80 360 240];
    % Maximum number of objects to track
    maxNumObj = 200;
    % Number of frames that an object must remain stationary before an alarm is
    % raised
    alarmCount = 45;
    % Maximum number of frames that an abandoned object can be hidden before it
    % is no longer tracked
    maxConsecutiveMiss = 4;
    areaChangeFraction = 13;     % Maximum allowable change in object area in percent
    centroidChangeFraction = 18; % Maximum allowable change in object centroid in percent
    % Minimum ratio between the number of frames in which an object is detected
    % and the total number of frames, for that object to be tracked.
    minPersistenceRatio = 0.7;
    % Offsets for drawing bounding boxes in original input video
    PtsOffset = int32(repmat([roi(1), roi(2), 0, 0],[maxNumObj 1]));

Create a VideoFileReader System object to read video from a file.

    hVideoSrc = vision.VideoFileReader;
    hVideoSrc.Filename = 'viptrain.avi';
    hVideoSrc.VideoOutputDataType = 'single';

Create a ColorSpaceConverter System object to convert the RGB image to Y'CbCr format.

    hColorConv = vision.ColorSpaceConverter('Conversion', 'RGB to YCbCr');

Create an Autothresholder System object to convert an intensity image to a binary image.

    hAutothreshold = vision.Autothresholder('ThresholdScaleFactor', 1.3);

Create a MorphologicalClose System object to fill in small gaps in the detected objects.

    hClosing = vision.MorphologicalClose('Neighborhood', strel('square',5));

Create a BlobAnalysis System object to find the area, centroid, and bounding box of the objects in the video.

    hBlob = vision.BlobAnalysis('MaximumCount', maxNumObj, 'ExcludeBorderBlobs', true);
    hBlob.MinimumBlobAreaSource = 'Property';
    hBlob.MinimumBlobArea = 100;
    hBlob.MaximumBlobAreaSource = 'Property';
    hBlob.MaximumBlobArea = 2500;

Create a ShapeInserter System object to draw rectangles around the abandoned objects.

    hDrawRectangles1 = vision.ShapeInserter('Fill',true, 'FillColor', 'Custom', ...
        'CustomFillColor', [1 0 0], 'Opacity', 0.5);

Create a TextInserter System object to display the number of objects in the video.

    hDisplayCount = vision.TextInserter('Text', '%4d', 'Color', [1 1 1]);

Create a ShapeInserter System object to draw rectangles around all the detected objects in the video.

    hDrawRectangles2 = vision.ShapeInserter('BorderColor', 'Custom', ...
        'CustomBorderColor', [0 1 0]);

Create a ShapeInserter System object to draw a rectangle around the region of interest.

    hDrawBBox = vision.ShapeInserter('BorderColor', 'Custom', ...
        'CustomBorderColor', [1 1 0]);

Create a ShapeInserter System object to draw rectangles around all the identified objects in the segmented video.

    hDrawRectangles3 = vision.ShapeInserter('BorderColor', 'Custom', ...
        'CustomBorderColor', [0 1 0]);

Create System objects to display results.

    pos = [10 300 roi(3)+25 roi(4)+25];
    hAbandonedObjects = vision.VideoPlayer('Name', 'Abandoned Objects', 'Position', pos);
    pos(1) = 46+roi(3); % move the next viewer to the right
    hAllObjects = vision.VideoPlayer('Name', 'All Objects', 'Position', pos);
    pos = [80+2*roi(3) 300 roi(3)-roi(1)+25 roi(4)-roi(2)+25];
    hThresholdDisplay = vision.VideoPlayer('Name', 'Threshold', 'Position', pos);

## Video Processing Loop

Create a processing loop to perform abandoned object detection on the input video. This loop uses the System objects you instantiated above.

    firsttime = true;
    while ~isDone(hVideoSrc)
        Im = step(hVideoSrc);

        % Select the region of interest from the original video
        OutIm = Im(roi(2):end, roi(1):end, :);

        YCbCr = step(hColorConv, OutIm);
        CbCr  = complex(YCbCr(:,:,2), YCbCr(:,:,3));

        % Store the first video frame as the background
        if firsttime
            firsttime = false;
            BkgY      = YCbCr(:,:,1);
            BkgCbCr   = CbCr;
        end
        SegY    = step(hAutothreshold, abs(YCbCr(:,:,1)-BkgY));
        SegCbCr = abs(CbCr-BkgCbCr) > 0.05;

        % Fill in small gaps in the detected objects
        Segmented = step(hClosing, SegY | SegCbCr);

        % Perform blob analysis
        [Area, Centroid, BBox] = step(hBlob, Segmented);

        % Call the helper function that tracks the identified objects and
        % returns the bounding boxes and the number of the abandoned objects.
        [OutCount, OutBBox] = videoobjtracker(Area, Centroid, BBox, maxNumObj, ...
           areaChangeFraction, centroidChangeFraction, maxConsecutiveMiss, ...
           minPersistenceRatio, alarmCount);

        % Display the abandoned object detection results
        Imr = step(hDrawRectangles1, Im, OutBBox+PtsOffset);
        Imr(1:15,1:30,:) = 0;
        Imr = step(hDisplayCount, Imr, OutCount);
        step(hAbandonedObjects, Imr);

        BlobCount = size(BBox,1);

        BBoxOffset = BBox + int32(repmat([roi(1) roi(2) 0  0],[BlobCount 1]));
        Imr = step(hDrawRectangles2, Im, BBoxOffset);

        % Display all the detected objects
        Imr(1:15,1:30,:) = 0;
        Imr = step(hDisplayCount, Imr, OutCount);
        Imr = step(hDrawBBox, Imr, roi);
        step(hAllObjects, Imr);

        % Display the segmented video
        SegBBox = PtsOffset;
        SegBBox(1:BlobCount,:) = BBox;
        SegIm = step(hDrawRectangles3, repmat(Segmented,[1 1 3]), SegBBox);
        step(hThresholdDisplay, SegIm);
    end

    release(hVideoSrc);

The `Abandoned Objects` window highlights the abandoned objects with a red box. The `All Objects` window marks the region of interest (ROI) with a yellow box and all detected objects with green boxes. The `Threshold` window shows the result of the background subtraction in the ROI.