Image Recognition and Tracking

You can extract features of an image object and recognize it from specific images. You can also track the image object in your application.

The main image recognition and tracking features include the following:

Recognizing images

You can recognize image objects in specific images, and extract image object features.
Tracking images

You can track image objects using the camera preview images and a specific image tracking model.

Prerequisites

To enable your application to use the image recognition and tracking functionality, proceed as follows:

Install the NuGet packages for media vision.

To use the methods and properties of the image recognition and tracking classes and to handle camera preview, include the Tizen.Multimedia and Tizen.Multimedia.Vision namespaces in your application:


                C#
                
                    Copy
                    
                
            
using Tizen.Multimedia;
using Tizen.Multimedia.Vision;

Define the configuration settings:

For configuring image object and feature extraction, create an instance of the Tizen.Multimedia.Vision.ImageFillConfiguration class and set its attributes accordingly:


                C#
                
                    Copy
                    
                
            
static ImageFillConfiguration configFill = new ImageFillConfiguration();

/// Set the scale factor of the image being recognized
configFill.ObjectScaleFactor = 1.2;

/// Set the maximum amount of image key points to be detected
configFill.ObjectMaxKeyPoints = 1000;

For image recognition, create an instance of the Tizen.Multimedia.Vision.ImageRecognitionConfiguration class and set its attributes accordingly:


                C#
                
                    Copy
                    
                
            
static ImageRecognitionConfiguration configRecog = new ImageRecognitionConfiguration();

/// Set the scene scale factor
configRecog.SceneScaleFactor = 1.2;

/// Set the maximum amount of key points to be detected in a scene
configRecog.SceneMaxKeyPoints = 3000;

For image tracking, create an instance of the Tizen.Multimedia.Vision.ImageTrackingConfiguration class and set its attributes accordingly:


                C#
                
                    Copy
                    
                
            
static ImageTrackingConfiguration configTrack = new ImageTrackingConfiguration();

/// Set the history amount
configTrack.HistoryAmount = 5;

/// Set the expected offset
configTrack.ExpectedOffset = 0.5;

Recognize images

To recognize an image (the target) in another (the scene), proceed as follows:

To prepare the target image being recognized, create an instance of the Tizen.Multimedia.Vision.MediaVisionSource class with raw image buffer data and its corresponding width, height, and color space parameters:


                C#
                
                    Copy
                    
                
            
/// Assume that there is a decoded raw data buffer of the byte[] type, and
/// it has 640x480 resolution with an RGB888 color space

MediaVisionSource sourceTarget = new MediaVisionSource(bytes, width, height, ColorSpace.Rgb888);

Create an instance of the Tizen.Multimedia.Vision.ImageObject class and use its Fill() method to fill it with the Tizen.Multimedia.Vision.MediaVisionSource instance:


                C#
                
                    Copy
                    
                
            
static ImageObject obj = new ImageObject();

obj.Fill(sourceTarget);

/// If you want to apply configuration options to the fill operation:
/// obj.Fill(sourceTarget, configFill);

/// If you want a specific label for the ImageObject instance, set it manually
/// Otherwise the label automatically increments with each fill operation
obj.SetLabel(1);

To prepare the scene where the target image is to be recognized, create a Tizen.Multimedia.Vision.MediaVisionSource instance which stores the scene:


                C#
                
                    Copy
                    
                
            
/// Assume that there is a decoded raw data buffer of the byte[] type, and
/// it has 640x480 resolution with an RGB888 color space

MediaVisionSource sourceScene  = new MediaVisionSource(bytes, width, height, ColorSpace.Rgb888);

To recognize the target inside the scene, use the RecognizeAsync() method of the Tizen.Multimedia.Vision.ImageRecognizer class:


                C#
                
                    Copy
                    
                
            
/// You can recognize multiple targets
ImageObject[] targets = new ImageObject[1] (obj);

var results = await ImageRecognizer.RecognizeAsync(sourceScene, targets);

foreach (ImageRecognitionResult imageResult in results)
{
    if (imageResult.Success)
        Log.info(LogUtils.TAG, imageResult, Region.ToString();
    else
        Log.info(LogUtils.Tag, "ImageRecognition: " + imageResult.Success.ToString());
}

Track images

To track images, proceed as follows:

To prepare the camera and create an image tracking model:

Define a camera preview event handler for the Preview event of the Tizen.Multimedia.Camera class and create an instance of that class:


                C#
                
                    Copy
                    
                
            
/// Define a camera preview event handler
static void PreviewCallback(object sender, PreviewEventArgs e)
{
    PreviewData preview = e.Preview;

    SinglePlane singlePlane = (SinglePlane)preview.Plane;
    if (preview.Format == CameraPixelFormat.Rgb888)
    {
        MediaVisionSource source = new MediaVisionSource(singlePlane.Data, preview.width, preview.height, ColorSpace.Rgb888);
    }
}

/// Create the Tizen.Multimedia.Camera instance
static Camera camera = null;
try
{
    camera = new Camera(CameraDevice.Rear);
}
catch (NotSupportedException)
{
    Log.Info("Image Tracking Sample", "NotSupported");
}

Set the camera display, register the camera preview event handler, and start the camera preview with the StartPreview() method:


                C#
                
                    Copy
                    
                
            
/// Set the camera display
camera.Display = new Display(new Window("Preview"));

/// Register the camera preview event handler
camera.Preview += PreviewCallback;

IList previewFormats = camera.Feature.SupportedPreviewPixelFormats.ToList();
foreach (CameraPixelFormat previewFormat in previewFormats)
{
    camera.Setting.PreviewPixelFormat = previewFormat;
    break;
}

/// Start the camera preview
camera.StartPreview();

Create the image tracking model as an instance of the Tizen.Multimedia.Vision.ImageTrackingModel class:


                C#
                
                    Copy
                    
                
            
static ImageTrackingModel model = new ImageTrackingModel();

Create a target image as an instance of the Tizen.Multimedia.Vision.MediaVisionSource class.

Create an instance of the Tizen.Multimedia.Vision.ImageObject class and use its Fill() method to fill it with the target image:


                C#
                
                    Copy
                    
                
            
static MediaVisionSource sourceTarget = new MediaVisionSource(bytes, width, height, ColorSpace.Rgb888);

static ImageObject obj = new ImageObject();
obj.Fill(sourceTarget);

Set the target of the image tracking model with the SetTarget() method of the Tizen.Multimedia.Vision.ImageTrackingModel class, which takes the Tizen.Multimedia.Vision.ImageObject instance as its parameter:


                C#
                
                    Copy
                    
                
            
model.SetTarget(obj)

To track the target, use the TrackAsync() method of the Tizen.Multimedia.Vision.ImageTracker class:


                C#
                
                    Copy
                    
                
            
/// Assume that "frames" contains a sequence of decoded images as
/// Tizen.Multimedia.Vision.MediaVisionSource instances

foreach (MediaVisionSource frame in frames)
{
    var result = await ImageTracker.TrackAsync(frame, model);

    /// If you want to apply configuration options to the tracking operation:
    /// var result = await ImageTracker.TrackAsync(frame, model, configTrack);

    if (result == null)
    {
        continue;
    }
}

When image tracking is no longer needed, deregister the camera preview event handler, stop the camera preview, and destroy the camera instance:


                C#
                
                    Copy
                    
                
            
camera.Preview -= PreviewCallback;

camera.StopPreview();
camera.Dispose();

Dependencies
- Tizen 4.0 and Higher

Image Recognition and Tracking

Prerequisites

Recognize images

Track images

Related information