Tizen Native API  5.0
Media Vision Face

Face detection, recognition, and tracking.

Required Header

#include <mv_face.h>

Related Features

This API is related with the following features:

  • http://tizen.org/feature/vision.face_recognition

It is recommended to design feature related codes in your application for reliability.
You can check if a device supports the related features for this API by using System Information, thereby controlling the procedure of your application.
To ensure your application is only running on the device with specific features, please define the features in your manifest file using the manifest editor in the SDK.
More details on featuring your application can be found from Feature Element.

Overview

Media Vision Face contains mv_face_detect() function to detect faces on mv_source_h, and mv_face_detected_cb() callback to process detected faces. Also it contains mv_face_recognize() function which performs face recognition on mv_source_h for recognition model mv_face_recognition_model_h handle. Results of the recognition will be passed to the mv_face_recognized_cb(). Results of the recognition consist of unique face label and confidence of the recognition model that face was recognized correctly. Unique face label is the integer identifier should be introduced to the model before starting recognition, when learning examples are added (see mv_face_recognition_model_add() function documentation for details about unique face labels).
Recognition model should be created with mv_face_recognition_model_create() and destroyed with mv_face_recognition_model_destroy(). Model can be cloned with mv_face_recognition_model_clone(), saved to the file with mv_face_recognition_model_save(), loaded with mv_face_recognition_model_load(). Model learning can be provided with mv_face_recognition_model_add(), and mv_face_recognition_model_learn() functions. These two methods has to be called in the direct order: first, labeled face examples should be added to the model using mv_face_recognition_model_add(). It is expected that images of the same face will be added specifying the same face label. When examples were added, model has to be learned based on the collected set of labeled face images. mv_face_recognition_model_learn() function will perform learning. If it is required to get the list of unique face labels learned by the model, mv_face_recognition_model_query_labels() function can be used.
Module contains function mv_face_track() which performs tracking on mv_source_h for mv_face_tracking_model_h and mv_face_tracked_cb() which process tracked face. Tracking model should be created with mv_face_tracking_model_create() and destroyed with mv_face_tracking_model_destroy(). Tracking model should be prepared with mv_face_tracking_model_prepare() before each session of tracking. Model can be cloned with mv_face_tracking_model_clone(), saved to the file with mv_face_tracking_model_save(), loaded with mv_face_tracking_model_load().
Module provides function for detecting eye-blink status - mv_face_eye_condition_recognize(), which provides detection on mv_source_h and face location - mv_rectangle_s. Callback mv_face_eye_condition_recognized_cb() handles result.
Also module contains mv_face_facial_expression_recognize() function which detects face expression on mv_source_h, face location is determined by mv_rectangle_s. Result is handled with mv_face_facial_expression_recognized_cb().

Functions

int mv_face_detect (mv_source_h source, mv_engine_config_h engine_cfg, mv_face_detected_cb detected_cb, void *user_data)
 Performs face detection on the source for the engine_conf.
int mv_face_recognize (mv_source_h source, mv_face_recognition_model_h recognition_model, mv_engine_config_h engine_cfg, mv_rectangle_s *face_location, mv_face_recognized_cb recognized_cb, void *user_data)
 Performs face recognition on the source image.
int mv_face_track (mv_source_h source, mv_face_tracking_model_h tracking_model, mv_engine_config_h engine_cfg, mv_face_tracked_cb tracked_cb, bool do_learn, void *user_data)
 Performs face tracking on the source for the tracking_model.
int mv_face_eye_condition_recognize (mv_source_h source, mv_engine_config_h engine_cfg, mv_rectangle_s face_location, mv_face_eye_condition_recognized_cb eye_condition_recognized_cb, void *user_data)
 Determines eye-blink condition for face_location on media source.
int mv_face_facial_expression_recognize (mv_source_h source, mv_engine_config_h engine_cfg, mv_rectangle_s face_location, mv_face_facial_expression_recognized_cb expression_recognized_cb, void *user_data)
 Determines facial expression for face_location on media source.
int mv_face_recognition_model_create (mv_face_recognition_model_h *recognition_model)
 Creates a face recognition model handle.
int mv_face_recognition_model_destroy (mv_face_recognition_model_h recognition_model)
 Destroys the face recognition model handle and releases all its resources.
int mv_face_recognition_model_clone (mv_face_recognition_model_h src, mv_face_recognition_model_h *dst)
 Creates a copy of existed recognition model handle and clones all its resources.
int mv_face_recognition_model_save (const char *file_name, mv_face_recognition_model_h recognition_model)
 Saves recognition model to the file.
int mv_face_recognition_model_load (const char *file_name, mv_face_recognition_model_h *recognition_model)
 Loads recognition model from file.
int mv_face_recognition_model_add (const mv_source_h source, mv_face_recognition_model_h recognition_model, const mv_rectangle_s *example_location, int face_label)
 Adds face image example to be used for face recognition model learning with mv_face_recognition_model_learn().
int mv_face_recognition_model_reset (mv_face_recognition_model_h recognition_model, int *face_label)
 Removes from recognition_model all collected with mv_face_recognition_model_add() function face examples labeled with face_label.
int mv_face_recognition_model_learn (mv_engine_config_h engine_cfg, mv_face_recognition_model_h recognition_model)
 Learns face recognition model.
int mv_face_recognition_model_query_labels (mv_face_recognition_model_h recognition_model, int **labels, unsigned int *number_of_labels)
 Queries labels list and number of labels had been learned by the model.
int mv_face_tracking_model_create (mv_face_tracking_model_h *tracking_model)
 Calls this function to create a face tracking model handle.
int mv_face_tracking_model_destroy (mv_face_tracking_model_h tracking_model)
 Calls this function to destroy the face tracking model handle and release all its resources.
int mv_face_tracking_model_prepare (mv_face_tracking_model_h tracking_model, mv_engine_config_h engine_cfg, mv_source_h source, mv_quadrangle_s *location)
 Calls this function to initialize tracking model by the location of the face to be tracked.
int mv_face_tracking_model_clone (mv_face_tracking_model_h src, mv_face_tracking_model_h *dst)
 Calls this function to make a copy of existed tracking model handle and clone all its resources to the copy.
int mv_face_tracking_model_save (const char *file_name, mv_face_tracking_model_h tracking_model)
 Calls this method to save tracking model to the file.
int mv_face_tracking_model_load (const char *file_name, mv_face_tracking_model_h *tracking_model)
 Calls this method to load a tracking model from file.

Typedefs

typedef void(* mv_face_detected_cb )(mv_source_h source, mv_engine_config_h engine_cfg, mv_rectangle_s *faces_locations, int number_of_faces, void *user_data)
 Called when faces are detected for the source.
typedef void(* mv_face_recognized_cb )(mv_source_h source, mv_face_recognition_model_h recognition_model, mv_engine_config_h engine_cfg, mv_rectangle_s *face_location, const int *face_label, double confidence, void *user_data)
 Called each time when face is recognized by mv_face_recognize() function.
typedef void(* mv_face_tracked_cb )(mv_source_h source, mv_face_tracking_model_h tracking_model, mv_engine_config_h engine_cfg, mv_quadrangle_s *location, double confidence, void *user_data)
 Called when face determined by tracking_model is tracked.
typedef void(* mv_face_eye_condition_recognized_cb )(mv_source_h source, mv_engine_config_h engine_cfg, mv_rectangle_s face_location, mv_face_eye_condition_e eye_condition, void *user_data)
 Called when eye blink condition is recognized.
typedef void(* mv_face_facial_expression_recognized_cb )(mv_source_h source, mv_engine_config_h engine_cfg, mv_rectangle_s face_location, mv_face_facial_expression_e facial_expression, void *user_data)
 Called when facial expression is recognized.
typedef void * mv_face_recognition_model_h
 The handle to the model aggregating recognition face features.
typedef void * mv_face_tracking_model_h
 The handle to the model aggregating tracking face features.

Defines

#define MV_FACE_DETECTION_MODEL_FILE_PATH   "MV_FACE_DETECTION_MODEL_FILE_PATH"
 Defines MV_FACE_DETECTION_MODEL_FILE_PATH to set face detection haarcascade xml file attribute of the engine configuration.
#define MV_FACE_RECOGNITION_MODEL_TYPE   "MV_FACE_RECOGNITION_MODEL_TYPE"
 Defines MV_FACE_RECOGNITION_MODEL_TYPE to set the method used for face recognition model learning attribute of the engine configuration.
#define MV_FACE_DETECTION_ROI_X   "MV_FACE_DETECTION_ROI_X"
 Define MV_FACE_DETECTION_ROI_X to set X coordinate of face detection roi (Region of Interest) as attribute of the engine configuration.
#define MV_FACE_DETECTION_ROI_Y   "MV_FACE_DETECTION_ROI_Y"
 Define MV_FACE_DETECTION_ROI_Y to set Y coordinate of face detection roi (Region of Interest) as attribute of the engine configuration.
#define MV_FACE_DETECTION_ROI_WIDTH   "MV_FACE_DETECTION_ROI_WIDTH"
 Define MV_FACE_DETECTION_ROI_WIDTH to set width of face detection roi (Region of Interest) as attribute of the engine configuration.
#define MV_FACE_DETECTION_ROI_HEIGHT   "MV_FACE_DETECTION_ROI_HEIGHT"
 Define MV_FACE_DETECTION_ROI_HEIGHT to set height of face detection roi (Region of Interest) as attribute of the engine configuration.
#define MV_FACE_DETECTION_MIN_SIZE_WIDTH   "MV_FACE_DETECTION_MIN_SIZE_WIDTH"
 Define MV_FACE_DETECTION_MIN_SIZE_WIDTH to set minimum width of face which will be detected as attribute of the engine configuration.
#define MV_FACE_DETECTION_MIN_SIZE_HEIGHT   "MV_FACE_DETECTION_MIN_SIZE_HEIGHT"
 Define MV_FACE_DETECTION_MIN_SIZE_HEIGHT to set minimum height of face which will be detected as attribute of the engine configuration.

Define Documentation

#define MV_FACE_DETECTION_MIN_SIZE_HEIGHT   "MV_FACE_DETECTION_MIN_SIZE_HEIGHT"

Define MV_FACE_DETECTION_MIN_SIZE_HEIGHT to set minimum height of face which will be detected as attribute of the engine configuration.

Default value is -1 (all detected faces will be applied) can be changed to specify the minimum face height

Since :
3.0
See also:
mv_engine_config_set_int_attribute()
mv_engine_config_get_int_attribute()
#define MV_FACE_DETECTION_MIN_SIZE_WIDTH   "MV_FACE_DETECTION_MIN_SIZE_WIDTH"

Define MV_FACE_DETECTION_MIN_SIZE_WIDTH to set minimum width of face which will be detected as attribute of the engine configuration.

Default value is -1 (all detected faces will be applied) can be changed to specify the minimum face width

Since :
3.0
See also:
mv_engine_config_set_int_attribute()
mv_engine_config_get_int_attribute()
#define MV_FACE_DETECTION_MODEL_FILE_PATH   "MV_FACE_DETECTION_MODEL_FILE_PATH"

Defines MV_FACE_DETECTION_MODEL_FILE_PATH to set face detection haarcascade xml file attribute of the engine configuration.

Face detection haarcascade model can be changed to specify the path to the file

Since :
3.0
See also:
mv_engine_config_set_string_attribute()
mv_engine_config_get_string_attribute()
#define MV_FACE_DETECTION_ROI_HEIGHT   "MV_FACE_DETECTION_ROI_HEIGHT"

Define MV_FACE_DETECTION_ROI_HEIGHT to set height of face detection roi (Region of Interest) as attribute of the engine configuration.

Default value is -1 (the roi will be a full image) can be changed to specify the roi for face detection

Since :
3.0
See also:
mv_engine_config_set_int_attribute()
mv_engine_config_get_int_attribute()
#define MV_FACE_DETECTION_ROI_WIDTH   "MV_FACE_DETECTION_ROI_WIDTH"

Define MV_FACE_DETECTION_ROI_WIDTH to set width of face detection roi (Region of Interest) as attribute of the engine configuration.

Default value is -1 (the roi will be a full image) can be changed to specify the roi for face detection

Since :
3.0
See also:
mv_engine_config_set_int_attribute()
mv_engine_config_get_int_attribute()
#define MV_FACE_DETECTION_ROI_X   "MV_FACE_DETECTION_ROI_X"

Define MV_FACE_DETECTION_ROI_X to set X coordinate of face detection roi (Region of Interest) as attribute of the engine configuration.

Default value is -1 (the roi will be a full image) can be changed to specify the roi for face detection

Since :
3.0
See also:
mv_engine_config_set_int_attribute()
mv_engine_config_get_int_attribute()
#define MV_FACE_DETECTION_ROI_Y   "MV_FACE_DETECTION_ROI_Y"

Define MV_FACE_DETECTION_ROI_Y to set Y coordinate of face detection roi (Region of Interest) as attribute of the engine configuration.

Default value is -1 (the roi will be a full image) can be changed to specify the roi for face detection

Since :
3.0
See also:
mv_engine_config_set_int_attribute()
mv_engine_config_get_int_attribute()
#define MV_FACE_RECOGNITION_MODEL_TYPE   "MV_FACE_RECOGNITION_MODEL_TYPE"

Defines MV_FACE_RECOGNITION_MODEL_TYPE to set the method used for face recognition model learning attribute of the engine configuration.

Switches between three types of methods used for face recognition model learning. Possible values of the attribute are: 1 - Eigenfaces, 2 - Fisherfaces, 3 - Local Binary Patterns Histograms (LBPH) Default type is LBPH

Since :
3.0
See also:
mv_engine_config_set_int_attribute()
mv_engine_config_get_int_attribute()

Typedef Documentation

typedef void(* mv_face_detected_cb)(mv_source_h source, mv_engine_config_h engine_cfg, mv_rectangle_s *faces_locations, int number_of_faces, void *user_data)

Called when faces are detected for the source.

This type callback can be invoked each time when mv_face_detect() is called to process the results of face detecting.

Since :
3.0
Remarks:
If no face is detected then the callback will be invoked, but faces_locations array will be NULL, and number_of_faces will be equal to 0.
Parameters:
[in]sourceThe handle to the source of the media where faces were detected
[in]engine_cfgThe handle to the configuration of engine was used for face detecting, or NULL if default settings were applied
[in]faces_locationsRectangular locations of detected faces
[in]number_of_facesNumber of detected faces
[in]user_dataThe user data passed from callback invoking code
Precondition:
Call mv_face_detect() function to perform detection of the face for the face image and invoke this callback as a result
See also:
mv_face_detect()
typedef void(* mv_face_eye_condition_recognized_cb)(mv_source_h source, mv_engine_config_h engine_cfg, mv_rectangle_s face_location, mv_face_eye_condition_e eye_condition, void *user_data)

Called when eye blink condition is recognized.

This type callback can be invoked each time when mv_face_eye_condition_recognize() is called for face_location to recognize eye-blink condition for the face at the source.
Usage example for this callback can be found in mv_face_eye_condition_recognize() documentation.

Since :
3.0
Parameters:
[in]sourceThe handle to the source of the media for which eye-blink condition was recognized
[in]engine_cfgThe handle to the configuration of engine was used for eye-blink condition recognition, or NULL if default settings were applied.
[in]face_locationThe location bounding the face at the source
[in]eye_conditionThe type of eye-blink condition recognized for face bounded by face_location
[in]user_dataThe user data passed from callback invoking code
Precondition:
Call mv_face_eye_condition_recognize() function to perform eye-blink condition recognition and invoke this callback as a result
See also:
mv_face_eye_condition_recognize()
typedef void(* mv_face_facial_expression_recognized_cb)(mv_source_h source, mv_engine_config_h engine_cfg, mv_rectangle_s face_location, mv_face_facial_expression_e facial_expression, void *user_data)

Called when facial expression is recognized.

This type callback can be invoked each time when mv_face_facial_expression_recognize() is called for face_location to recognize facial expression for the face at the source.
Usage example for this callback can be found in mv_face_facial_expression_recognize() documentation.

Since :
3.0
Parameters:
[in]sourceThe handle to the source of the media for which facial expression was recognized
[in]engine_cfgThe handle to the configuration of engine was used for expression recognition
[in]face_locationThe location bounding the face at the source
[in]facial_expressionThe type of facial expression recognized for face bounded by face_location
[in]user_dataThe user data passed from callback invoking code
Precondition:
Create a source handle by calling mv_create_source()
Create a face engine configuration handle by calling mv_create_engine_config()
See also:
mv_face_facial_expression_recognize()

The handle to the model aggregating recognition face features.

This handle can be used for faces recognizing with mv_face_recognize() function. Handle has to be created by mv_face_recognition_model_create() and destroyed by mv_face_recognition_model_destroy() functions. To use recognition models effectively learning process has to be performed before recognition. In other words, appropriate set of the face image examples has to be collected with mv_face_recognition_model_add() function before mv_face_recognition_model_learn() function call, then recognition can be performed with mv_face_recognize().

Since :
3.0
See also:
mv_face_recognition_model_create()
mv_face_recognition_model_destroy()
mv_face_recognition_model_learn()
typedef void(* mv_face_recognized_cb)(mv_source_h source, mv_face_recognition_model_h recognition_model, mv_engine_config_h engine_cfg, mv_rectangle_s *face_location, const int *face_label, double confidence, void *user_data)

Called each time when face is recognized by mv_face_recognize() function.

Since :
3.0
Parameters:
[in]sourceThe handle to the image source for which face has been recognized/not recognized
[in]recognition_modelThe handle to the recognition model has been used for recognition
[in]engine_cfgThe handle to the configuration of engine was used for recognition, or NULL if default settings were applied
[in]face_locationThe pointer to the location of the face recognized on source. If face wasn't recognized, then pointer is NULL
[in]face_labelThe label that identifies face which was recognized in the source. NULL if recognition was performed, but no faces were recognized in the source
[in]confidenceThe confidence of the recognition_model that face has been recognized correctly (value from 0.0 to 1.0). No faces were recognized if confidence was 0.0. When model has been learned on large amount of examples, threshold for this value can be high (0.85-0.95). If model was learned for small amount of examples, then threshold can be reduced (0.5-0.85)
[in]user_dataThe user data passed from callback invoking code
Precondition:
Call mv_face_recognize() function to perform recognition of the face for the face image and invoke this callback as a result
See also:
mv_face_recognize()
typedef void(* mv_face_tracked_cb)(mv_source_h source, mv_face_tracking_model_h tracking_model, mv_engine_config_h engine_cfg, mv_quadrangle_s *location, double confidence, void *user_data)

Called when face determined by tracking_model is tracked.

This type callback can be invoked each time when mv_face_track() is called to process the results of face tracking.

Since :
3.0
Parameters:
[in]sourceThe handle to the video frame or image from sequence for which face was tracked
[in]tracking_modelThe handle to the model that was used for tracking
[in]engine_cfgThe handle to the configuration of engine was used for tracking, or NULL if default settings were applied.
[in]locationThe pointer to the quadrangle-shaped location which determines new position of the tracked face on the source. If NULL, then face was lost by tracking algorithm during last iteration
[in]confidenceThe confidence of the tracking_model that new location of the face was determined correctly (value from 0.0 to 1.0). If no location was determined during last track iteration, then value is 0.0
[in]user_dataThe user data passed from callback invoking code
Precondition:
Call mv_face_track() function to perform track iteration for the video frame or the image from sequence and invoke this callback as a result
See also:
mv_face_track()
typedef void* mv_face_tracking_model_h

The handle to the model aggregating tracking face features.

This model can be used for face tracking with mv_face_track() function. Handle has to be created by mv_face_tracking_model_create() and destroyed by mv_face_tracking_model_destroy() function. Tracking model can be improved during tracking task and allows to track face more accurately. So, you can create several tracking models independently by default, then apply tracking task for each of them. After some tracking these models will be different and each of them will be more efficient for tracking of face for which was created.

Since :
3.0
Remarks:
Create each tracking model for single face.
See also:
mv_face_tracking_model_create()
mv_face_tracking_model_destroy()

Enumeration Type Documentation

Enumeration for eyes state type.

Since :
3.0
See also:
mv_face_eye_condition_recognize()
Enumerator:
MV_FACE_EYES_OPEN 

Eyes are open

MV_FACE_EYES_CLOSED 

Eyes are closed

MV_FACE_EYES_NOT_FOUND 

The eyes condition wasn't determined

Enumeration for expression types can be determined for faces.

Since :
3.0
See also:
mv_face_facial_expression_recognize()
Enumerator:
MV_FACE_UNKNOWN 

Unknown face expression

MV_FACE_NEUTRAL 

Face expression is neutral

MV_FACE_SMILE 

Face expression is smiling

MV_FACE_SADNESS 

Face expression is sadness

MV_FACE_SURPRISE 

Face expression is surprise

MV_FACE_ANGER 

Face expression is anger

MV_FACE_FEAR 

Face expression is fear

MV_FACE_DISGUST 

Face expression is disgust


Function Documentation

int mv_face_detect ( mv_source_h  source,
mv_engine_config_h  engine_cfg,
mv_face_detected_cb  detected_cb,
void *  user_data 
)

Performs face detection on the source for the engine_conf.

Use this function to launch face detection algorithm configured by engine_conf configuration. Each time when mv_face_detect is called, detected_cb will receive a set of the detected faces at the media source.

Since :
3.0
Parameters:
[in]sourceThe handle to the source of the media where faces will be detected
[in]engine_cfgThe handle to the configuration of engine will be used for detecting. If NULL, then default settings will be used.
[in]detected_cbThe callback which will be called for all face locations detected on media source. This callback will receive detecting results
[in]user_dataThe user data passed from the code where mv_face_detect() is invoked. This data will be accessible from detected_cb callback.
Returns:
0 on success, otherwise a negative error value
Return values:
MEDIA_VISION_ERROR_NONESuccessful
MEDIA_VISION_ERROR_INVALID_PARAMETERInvalid parameter
MEDIA_VISION_ERROR_INVALID_OPERATIONInvalid operation
MEDIA_VISION_ERROR_NOT_SUPPORTED_FORMATSource colorspace isn't supported
MEDIA_VISION_ERROR_NOT_SUPPORTEDNot supported
Precondition:
Create a source handle by calling mv_create_source()
Postcondition:
detected_cb will be called to process detection results
See also:
mv_face_detected_cb
int mv_face_eye_condition_recognize ( mv_source_h  source,
mv_engine_config_h  engine_cfg,
mv_rectangle_s  face_location,
mv_face_eye_condition_recognized_cb  eye_condition_recognized_cb,
void *  user_data 
)

Determines eye-blink condition for face_location on media source.

Use this function to recognize eye-blink condition for the face bounded by face_location at source.

Since :
3.0
Parameters:
[in]sourceThe handle to the source of the media to recognize eye-blink condition for
[in]engine_cfgThe handle to the configuration of engine will be used for eye-blink condition recognition. If NULL, the default configuration will be used.
[in]face_locationThe location bounding the face at the source
[in]eye_condition_recognized_cbThe callback for processing result of eye-blink condition recognition
[in]user_dataThe user data passed from the code where mv_face_eye_condition_recognize() is invoked. This data will be accessible from eye_condition_recognized_cb callback.
Returns:
0 on success, otherwise a negative error value
Return values:
MEDIA_VISION_ERROR_NONESuccessful
MEDIA_VISION_ERROR_INVALID_PARAMETERInvalid parameter
MEDIA_VISION_ERROR_NOT_SUPPORTED_FORMATSource colorspace isn't supported
MEDIA_VISION_ERROR_NOT_SUPPORTEDNot supported
Precondition:
Create a source handle by calling mv_create_source()
See also:
mv_face_eye_condition_recognized_cb
int mv_face_facial_expression_recognize ( mv_source_h  source,
mv_engine_config_h  engine_cfg,
mv_rectangle_s  face_location,
mv_face_facial_expression_recognized_cb  expression_recognized_cb,
void *  user_data 
)

Determines facial expression for face_location on media source.

Use this function to determine facial expression for the face bounded by face_location at source.

Since :
3.0
Parameters:
[in]sourceThe handle to the source of the media to recognize facial expression for
[in]engine_cfgThe handle to the configuration of engine will be used for expression recognition
[in]face_locationThe location bounding the face at the source
[in]expression_recognized_cbThe callback for processing result of facial expression determining
[in]user_dataThe user data passed from the code where mv_face_facial_expression_recognize() is invoked. This data will be accessible from expression_recognized_cb callback.
Returns:
0 on success, otherwise a negative error value
Return values:
MEDIA_VISION_ERROR_NONESuccessful
MEDIA_VISION_ERROR_INVALID_PARAMETERInvalid parameter
MEDIA_VISION_ERROR_NOT_SUPPORTED_FORMATSource colorspace isn't supported
MEDIA_VISION_ERROR_NOT_SUPPORTEDNot supported
Precondition:
Create a source handle by calling mv_create_source()
Create a face engine configuration handle by calling mv_create_engine_config()
See also:
mv_face_facial_expression_recognized_cb
int mv_face_recognition_model_add ( const mv_source_h  source,
mv_face_recognition_model_h  recognition_model,
const mv_rectangle_s example_location,
int  face_label 
)

Adds face image example to be used for face recognition model learning with mv_face_recognition_model_learn().

Since :
3.0
Remarks:
It is possible to destroy source after calling this method. Source isn't used for learning directly.
Face image example_location location can be determined using mv_face_detect function.
Parameters:
[in]sourceThe handle to source that contains face image
[in]recognition_modelThe handle to the recognition model which could be learned based on example
[in]example_locationThe pointer to the rectangular location of the face image at the source image. If NULL, then full image will be analyzed as the face image
[in]face_labelThe label that identifies face for which example is adding. Specify the same labels for the face images of a single person when calling this method. Has to be unique for each face
Returns:
0 on success, otherwise a negative error value
Return values:
MEDIA_VISION_ERROR_NONESuccessful
MEDIA_VISION_ERROR_INVALID_PARAMETERInvalid parameter
MEDIA_VISION_ERROR_NOT_SUPPORTED_FORMATSource colorspace isn't supported
MEDIA_VISION_ERROR_NOT_SUPPORTEDNot supported
Precondition:
Create a face recognition handle by calling mv_face_recognition_model_create() function
Postcondition:
When appropriate amount of face image examples is added to the recognition_model, this model has to be learned by mv_face_recognition_model_learn() function call. Only after learning of the model it can be used for face recognition with mv_face_recognize() function
See also:
mv_face_recognition_model_reset()
mv_face_recognition_model_learn()

Creates a copy of existed recognition model handle and clones all its resources.

Since :
3.0
Remarks:
Cloning perform not only handle copy, but also copies all internal resources of the model. dst must be released using mv_face_recognition_model_destroy().
Parameters:
[in]srcThe handle to the recognition model to be copied
[out]dstThe handle to the copy of existed recognition model specified as src
Returns:
0 on success, otherwise a negative error value
Return values:
MEDIA_VISION_ERROR_NONESuccessful
MEDIA_VISION_ERROR_INVALID_PARAMETERInvalid parameter
MEDIA_VISION_ERROR_OUT_OF_MEMORYOut of memory
MEDIA_VISION_ERROR_NOT_SUPPORTEDNot supported
Precondition:
Create face recognition handles by calling mv_face_recognition_model_create()
See also:
mv_face_recognition_model_create()

Creates a face recognition model handle.

Use this function to create default face recognition model. Creating process is defined by concrete face engine library. After creation recognition model has to be learned with mv_face_recognition_model_learn() function to provide appropriate results of face recognition functionality. Or learned earlier model can be loaded by mv_face_recognition_model_load() function.

Since :
3.0
Remarks:
It can cause incompatibility issues when saved models (see mv_face_recognition_model_save(), mv_face_recognition_model_load() functions documentation) are used in applications for different platforms which use different computer vision libraries underlying this API.
You must release recognition_model by using mv_face_recognition_model_destroy() function.
Parameters:
[out]recognition_modelThe handle to the recognition model to be created
Returns:
0 on success, otherwise a negative error value
Return values:
MEDIA_VISION_ERROR_NONESuccessful
MEDIA_VISION_ERROR_INVALID_PARAMETERInvalid parameter
MEDIA_VISION_ERROR_OUT_OF_MEMORYOut of memory
MEDIA_VISION_ERROR_NOT_SUPPORTEDNot supported
Postcondition:
Model can be loaded from the file after creation. Use mv_face_recognition_model_load() function to load it from file
Release recognition_model by using mv_face_recognition_model_destroy() function when it is not needed anymore
See also:
mv_face_recognition_model_destroy()

Destroys the face recognition model handle and releases all its resources.

Since :
3.0
Parameters:
[in]recognition_modelThe handle to the face recognition model to be destroyed
Returns:
0 on success, otherwise a negative error value
Return values:
MEDIA_VISION_ERROR_NONESuccessful
MEDIA_VISION_ERROR_INVALID_PARAMETERInvalid parameter
MEDIA_VISION_ERROR_NOT_SUPPORTEDNot supported
Precondition:
Create recognition model by using mv_face_recognition_model_create()
See also:
mv_face_recognition_model_create()

Learns face recognition model.

Before you start learning process, face recognition models has to be filled with training data - face image examples. These examples has to be provided by mv_face_recognition_model_add() function. Usually, recognition accuracy is increased when number of not identical examples is large. But it depends on the used learning algorithm.

Since :
3.0
Remarks:
Common flow is to collect face examples as much as possible, add them to the recognition model with mv_face_recognition_model_add(), then call mv_face_recognition_model_learn() for this recognition model to learn it (or update the model if updating is supported by the used algorithm).
Selection of the learning algorithm can be performed by setting corresponding attributes for the engine_cfg. You can check supported by engine_cfg attributes using mv_engine_config_foreach_supported_attribute() function call. By default, Local Binary Patterns Histograms (LBPH) based recognition algorithm will be used.
Parameters:
[in]engine_cfgThe handle to the configuration of engine will be used for learning of the recognition models. If NULL, then default settings will be used
[in,out]recognition_modelThe model which will be learned. After learning process these model may be changed, so mv_face_recognize() results may differ before and after method call respectively to the face examples collected for the recognition_model
Returns:
0 on success, otherwise a negative error value
Return values:
MEDIA_VISION_ERROR_NONESuccessful
MEDIA_VISION_ERROR_INVALID_PARAMETERInvalid parameter
MEDIA_VISION_ERROR_NO_DATANo data
MEDIA_VISION_ERROR_NOT_SUPPORTEDNot supported
Precondition:
Create a face engine configuration handle by calling mv_create_engine_config() and set supported parameters if needed. Or just set engine_cfg as NULL to learn with default settings
Create a face recognition model handles by calling mv_face_recognition_model_create() function
Add face image examples to the recognition_model by calling mv_face_recognition_model_add() function
Postcondition:
If it is not planned to learn the model again, clear memory by mv_face_recognition_model_reset() function
When model has been learned, it can be used for face recognition with mv_face_recognize() function
See also:
mv_face_recognition_model_add()
mv_face_recognition_model_reset()
mv_face_recognize()
int mv_face_recognition_model_load ( const char *  file_name,
mv_face_recognition_model_h recognition_model 
)

Loads recognition model from file.

Since :
3.0
Remarks:
This function doesn't modify the set of face image examples added with mv_face_recognition_model_add() function. Model will be loaded from file without loss of collected examples. If you want to free memory from examples, use mv_face_recognition_model_reset() function. It is recommended to clear the memory if learning algorithm doesn't support reinforcement learning.
recognition_model is loaded from the absolute path directory. Use app_get_data_path for the private app storage path. recognition_model must be destroyed using mv_face_recognition_model_destroy().
Parameters:
[in]file_nameName of path/file to load the model
[out]recognition_modelThe handle to the recognition model to be loaded from the file
Returns:
0 on success, otherwise a negative error value
Return values:
MEDIA_VISION_ERROR_NONESuccessful
MEDIA_VISION_ERROR_INVALID_PARAMETERInvalid parameter
MEDIA_VISION_ERROR_INVALID_PATHInvalid path
MEDIA_VISION_ERROR_PERMISSION_DENIEDNot permitted
MEDIA_VISION_ERROR_OUT_OF_MEMORYOut of memory
MEDIA_VISION_ERROR_NOT_SUPPORTED_FORMATNot supported model format
MEDIA_VISION_ERROR_NOT_SUPPORTEDNot supported
Precondition:
Recognition model can be preliminary saved with mv_face_recognition_model_save() function
See also:
mv_face_recognition_model_save()
mv_face_recognition_model_destroy()
app_get_data_path()
int mv_face_recognition_model_query_labels ( mv_face_recognition_model_h  recognition_model,
int **  labels,
unsigned int *  number_of_labels 
)

Queries labels list and number of labels had been learned by the model.

Since :
3.0
Remarks:
labels array has to be released using free().
Parameters:
[in]recognition_modelThe handle to the recognition model for which set of the learned labels will be queried
[out]labelsThe array which will be filled with labels had been learned by the model
[out]number_of_labelsThe number of labels in labels array
Returns:
0 on success, otherwise a negative error value
Return values:
MEDIA_VISION_ERROR_NONESuccessful
MEDIA_VISION_ERROR_INVALID_PARAMETERInvalid parameter
MEDIA_VISION_ERROR_NOT_SUPPORTEDNot supported
Precondition:
Add face image examples with labels to the recognition_model by calling the mv_face_recognition_model_add() function
Learn the recognition_model by labeled examples using mv_face_recognition_model_learn() function
Postcondition:
labels array has to be freed in the function invoking code
See also:
mv_face_recognition_model_add()
mv_face_recognition_model_reset()
mv_face_recognition_model_learn()
int mv_face_recognition_model_reset ( mv_face_recognition_model_h  recognition_model,
int *  face_label 
)

Removes from recognition_model all collected with mv_face_recognition_model_add() function face examples labeled with face_label.

Since :
3.0
Remarks:
Be aware that if this function is called before mv_face_recognition_model_learn() function call, all or part of the required for learning data will be lost. It means that face image examples determined by the face_label label will be removed from the model and not taken into account when mv_face_recognition_model_learn() will be called next time.
Call of this function will free all the memory has been allocated during previous mv_face_recognition_model_add() calls for the corresponding face_label label.
Parameters:
[in]recognition_modelThe handle to the recognition model for which face image examples will be reset.
[in]face_labelThe label that identifies face for which examples will be removed from the recognition_model. If NULL, then all known by recognition_model face image examples will be removed
Returns:
0 on success, otherwise a negative error value
Return values:
MEDIA_VISION_ERROR_NONESuccessful
MEDIA_VISION_ERROR_INVALID_PARAMETERInvalid parameter
MEDIA_VISION_ERROR_KEY_NOT_AVAILABLEKey not available
MEDIA_VISION_ERROR_NOT_SUPPORTEDNot supported
See also:
mv_face_recognition_model_add()
mv_face_recognition_model_learn()
int mv_face_recognition_model_save ( const char *  file_name,
mv_face_recognition_model_h  recognition_model 
)

Saves recognition model to the file.

Since :
3.0
Remarks:
This function doesn't save face image examples (image itself) added by mv_face_recognition_model_add() function. This examples can be removed by mv_face_recognition_model_reset() function if it is needed to clear the memory.
recognition_model is saved to the absolute path directory. Use app_get_data_path for the private app storage path. After model is saved to the file, it can be loaded from this file by mv_face_recognition_model_load() function.
Parameters:
[in]file_nameName of the path/file to save the model
[in]recognition_modelThe handle to the recognition model to be saved to the file
Returns:
0 on success, otherwise a negative error value
Return values:
MEDIA_VISION_ERROR_NONESuccessful
MEDIA_VISION_ERROR_INVALID_PARAMETERInvalid parameter
MEDIA_VISION_ERROR_INVALID_OPERATIONInvalid operation
MEDIA_VISION_ERROR_INVALID_PATHInvalid path
MEDIA_VISION_ERROR_PERMISSION_DENIEDNot permitted
MEDIA_VISION_ERROR_NOT_SUPPORTED_FORMATNot supported model format
MEDIA_VISION_ERROR_NOT_SUPPORTEDNot supported
Precondition:
Create a face recognition handle by calling mv_face_recognition_model_create() function
Postcondition:
Saved model can be loaded later by calling mv_face_recognition_model_load() function
See also:
mv_face_recognition_model_load()
mv_face_recognition_model_create()
app_get_data_path()
int mv_face_recognize ( mv_source_h  source,
mv_face_recognition_model_h  recognition_model,
mv_engine_config_h  engine_cfg,
mv_rectangle_s face_location,
mv_face_recognized_cb  recognized_cb,
void *  user_data 
)

Performs face recognition on the source image.

Use this function to launch face recognition algorithm configured by engine_conf configuration using recognition_model recognition model. Each time when mv_face_recognize() is called, recognized_cb will receive recognition results:

  • Location in the source of the face has been recognized;
  • Label of the face has been recognized;
  • Confidence of the recognition_model that face has been recognized correctly (value from 0.0 to 1.0).
Since :
3.0
Remarks:
Using of untrained or weakly trained recognition models will cause not accurate results even if resulting confidence will be high. Use mv_face_recognition_model_learn() function before mv_face_recognize() call. Best results can be achieved when big set of face image examples were added by mv_face_recognition_model_add() before mv_face_recognition_model_learn() call.
Parameters:
[in]sourceThe handle to the source of the media to recognize face(s) for
[in]recognition_modelThe handle to the model will be used for recognition
[in]engine_cfgThe handle to the configuration of engine will be used for recognition. If NULL, then default settings will be used
[in]face_locationRectangular box bounding face image on the source. If NULL, then full source will be analyzed
[in]recognized_cbThe callback which will be called for the face recognition results on the source.
[in]user_dataThe user data passed from the code where mv_face_recognize() is invoked. This data will be accessible from recognized_cb callback
Returns:
0 on success, otherwise a negative error value
Return values:
MEDIA_VISION_ERROR_NONESuccessful
MEDIA_VISION_ERROR_INVALID_PARAMETERInvalid parameter
MEDIA_VISION_ERROR_INVALID_OPERATIONInvalid operation
MEDIA_VISION_ERROR_NOT_SUPPORTED_FORMATSource colorspace isn't supported
MEDIA_VISION_ERROR_NOT_SUPPORTEDNot supported
Precondition:
Create a source handle by calling mv_create_source()
Create a face recognition model handle by calling mv_face_recognition_model_create()
Postcondition:
recognized_cb will be called to process recognition results
See also:
mv_face_recognized_cb
int mv_face_track ( mv_source_h  source,
mv_face_tracking_model_h  tracking_model,
mv_engine_config_h  engine_cfg,
mv_face_tracked_cb  tracked_cb,
bool  do_learn,
void *  user_data 
)

Performs face tracking on the source for the tracking_model.

Use this function to launch face tracking algorithm configured by engine_conf configuration using tracking_model tracking model. Each time when this function is called, tracked_cb will receive updated tracking_model, new location determined for the tracked face and model confidence that location is determined correctly.

Since :
3.0
Remarks:
To allow correct tracking tracking_model has to be already used in previous tracking process(es) or prepared with mv_face_tracking_model_prepare(). Preparation requires specifying the face location for the source on which tracking was started. I.e. mv_face_tracking_model_prepare() function has to be called at least once before this method call.
Parameters:
[in]sourceThe handle to the source of the media to recognize face for
[in]tracking_modelThe handle to the model will be used for tracking
[in]engine_cfgThe handle to the configuration of engine will be used for tracking. If NULL, the default configuration will be used.
[in]tracked_cbThe callback which will be called for tracking event on the source where face would be tracked. This callback will receive tracking results
[in]do_learnThe model learning flag. If it is set true then model will try to learn (if it supports learning feature), otherwise model will be not learned during the invoking tracking iteration. Learning process improves tracking correctness, but can decrease tracking performance
[in]user_dataThe user data passed from the code where mv_face_track() is invoked. This data will be accessible from tracked_cb callback
Returns:
0 on success, otherwise a negative error value
Return values:
MEDIA_VISION_ERROR_NONESuccessful
MEDIA_VISION_ERROR_INVALID_PARAMETERInvalid parameter
MEDIA_VISION_ERROR_INVALID_OPERATIONInvalid operation
MEDIA_VISION_ERROR_NOT_SUPPORTED_FORMATSource colorspace isn't supported
MEDIA_VISION_ERROR_NOT_SUPPORTEDNot supported
Precondition:
Create a source handle by calling mv_create_source()
Create a face tracking model handle by calling mv_face_tracking_model_create()
Postcondition:
tracked_cb will be called to process tracking results
See also:
mv_face_tracked_cb

Calls this function to make a copy of existed tracking model handle and clone all its resources to the copy.

Since :
3.0
Remarks:
Cloning performs not only handle copy, but also copies all internal resources of the model. dst must be released using mv_face_tracking_model_destroy().
Parameters:
[in]srcThe handle to the tracking model to be copied
[out]dstThe handle to the copy of existed tracking model specified as src
Returns:
0 on success, otherwise a negative error value
Return values:
MEDIA_VISION_ERROR_NONESuccessful
MEDIA_VISION_ERROR_INVALID_PARAMETERInvalid parameter
MEDIA_VISION_ERROR_OUT_OF_MEMORYOut of memory
MEDIA_VISION_ERROR_NOT_SUPPORTEDNot supported
Precondition:
Create face tracking src handle by calling mv_face_tracking_model_create()
See also:
mv_face_tracking_model_create()

Calls this function to create a face tracking model handle.

Use this function to create default face tracking model handle. After creation this handle has to be initialized with mv_face_tracking_model_prepare() function to provide appropriate results of face tracking functionality. When handle is prepared, it is possible to use it for tracking on continuous sequence of the sources. Call mv_face_tracking_model_prepare() function each time before starting tracking on the new sequence. The exception is situation when the new sequence is continuation of the previous sequence for which model has been tracked.

Since :
3.0
Parameters:
[out]tracking_modelThe pointer to the handle to the tracking model that will be created
Returns:
0 on success, otherwise a negative error value
Return values:
MEDIA_VISION_ERROR_NONESuccessful
MEDIA_VISION_ERROR_INVALID_PARAMETERInvalid parameter
MEDIA_VISION_ERROR_OUT_OF_MEMORYOut of memory
MEDIA_VISION_ERROR_NOT_SUPPORTEDNot supported
Postcondition:
Model can be loaded from the file after creation. Use mv_face_tracking_model_load() function to load it from file
Use mv_face_tracking_model_prepare() function before tracking on the new video or continuous images sequence
Release tracking_model by using mv_face_tracking_model_destroy() function when it is not needed anymore
See also:
mv_face_tracking_model_destroy()
mv_face_tracking_model_prepare()
mv_face_tracking_model_load()

Calls this function to destroy the face tracking model handle and release all its resources.

Since :
3.0
Parameters:
[in]tracking_modelThe handle to the face tracking model that will be destroyed
Returns:
0 on success, otherwise a negative error value
Return values:
MEDIA_VISION_ERROR_NONESuccessful
MEDIA_VISION_ERROR_INVALID_PARAMETERInvalid parameter
MEDIA_VISION_ERROR_NOT_SUPPORTEDNot supported
Precondition:
Create tracking model by using mv_face_tracking_model_create()
See also:
mv_face_tracking_model_create()
int mv_face_tracking_model_load ( const char *  file_name,
mv_face_tracking_model_h tracking_model 
)

Calls this method to load a tracking model from file.

Since :
3.0
Remarks:
tracking_model is loaded from the absolute path directory. Use app_get_data_path for the private app storage path. tracking_model must be destroyed using mv_face_tracking_model_destroy.
Parameters:
[in]file_nameName of path/file to load the model
[out]tracking_modelThe handle to the tracking model to be loaded from file
Returns:
0 on success, otherwise a negative error value
Return values:
MEDIA_VISION_ERROR_NONESuccessful
MEDIA_VISION_ERROR_INVALID_PARAMETERInvalid parameter
MEDIA_VISION_ERROR_INVALID_OPERATIONInvalid operation
MEDIA_VISION_ERROR_INVALID_PATHInvalid path
MEDIA_VISION_ERROR_PERMISSION_DENIEDNot permitted
MEDIA_VISION_ERROR_OUT_OF_MEMORYOut of memory
MEDIA_VISION_ERROR_NOT_SUPPORTED_FORMATNot supported model format
MEDIA_VISION_ERROR_NOT_SUPPORTEDNot supported
Precondition:
Models has been saved by mv_face_tracking_model_save() function can be loaded with this function
Postcondition:
After model has been loaded and if further tracking will be performed on the video which is not continuation of the last tracking performed for the model, it is recommended to call mv_face_tracking_model_prepare() function
See also:
mv_face_tracking_model_save()
mv_face_tracking_model_destroy()
app_get_data_path()
int mv_face_tracking_model_prepare ( mv_face_tracking_model_h  tracking_model,
mv_engine_config_h  engine_cfg,
mv_source_h  source,
mv_quadrangle_s location 
)

Calls this function to initialize tracking model by the location of the face to be tracked.

This function is usually called once after tracking model is created and each time before tracking is started for the new sequence of sources which is not the direct continuation of the sequence for which tracking has been performed before. But it is allowed to call it between tracking sessions to allow Media Vision start to track more accurately.

Since :
3.0
Parameters:
[in]tracking_modelThe handle to the tracking model that will be prepared for tracking on new video or image sequence
[in]engine_cfgThe handle to the configuration of engine will be used for model preparing. If NULL, then default settings will be used.
[in]sourceThe handle to the source where face location is specified. Usually it is the first frame of the video or the first image in the continuous image sequence planned to be used for tracking
[in]locationThe quadrangle-shaped location (actually, rectangle can be used) determining position of the face to be tracked on the source. If NULL, then tracking model will try to find previously tracked face by itself. Don't set NULL when called first time for the tracking model.
Returns:
0 on success, otherwise a negative error value
Return values:
MEDIA_VISION_ERROR_NONESuccessful
MEDIA_VISION_ERROR_INVALID_PARAMETERInvalid parameter
MEDIA_VISION_ERROR_NOT_SUPPORTEDNot supported
Precondition:
Create a face tracking model handle by calling mv_face_tracking_model_create() function
Create a source handle by calling mv_create_source() function
Postcondition:
When model is prepared, mv_face_track() function can be used to track on the video or continuous image sequence
See also:
mv_face_tracking_model_create()
mv_face_track()
int mv_face_tracking_model_save ( const char *  file_name,
mv_face_tracking_model_h  tracking_model 
)

Calls this method to save tracking model to the file.

Since :
3.0
Remarks:
tracking_model is saved to the absolute path directory. Use app_get_data_path for the private app storage path. After model is saved to the file, it can be loaded from this file with mv_face_tracking_model_load() function.
Parameters:
[in]file_nameName of the path/file to save the model
[in]tracking_modelThe handle to the tracking model to be saved to the file
Returns:
0 on success, otherwise a negative error value
Return values:
MEDIA_VISION_ERROR_NONESuccessful
MEDIA_VISION_ERROR_INVALID_PARAMETERInvalid parameter
MEDIA_VISION_ERROR_INVALID_OPERATIONInvalid operation
MEDIA_VISION_ERROR_INVALID_PATHInvalid path
MEDIA_VISION_ERROR_PERMISSION_DENIEDNot permitted
MEDIA_VISION_ERROR_NOT_SUPPORTED_FORMATNot supported model format
MEDIA_VISION_ERROR_NOT_SUPPORTEDNot supported
Precondition:
Create a face tracking handle by calling mv_face_tracking_model_create()
Postcondition:
Saved model can be loaded from file using mv_face_tracking_model_load() function
See also:
mv_face_tracking_model_load()
mv_face_tracking_model_create()
app_get_data_path()