Facial landmark detection.

Required Header

#include <mv_facial_landmark.h>

Overview

Media Vision Facial Landmark Detection contains mv_facial_landmark_h handle to perform facial landmark detection. Facial landmark handle should be created with mv_facial_landmark_create() and destroyed with mv_facial_landmark_destroy(). mv_facial_landmark_h should be configured by calling mv_facial_landmark_configure(). After configuration, mv_facial_landmark_h should be prepared by calling mv_facial_landmark_prepare() which loads models and set required parameters. After preparation, mv_facial_landmark_inference() or mv_facial_landmark_inference_async() can be called to detect facial landmarks in images on mv_source_h. Use mv_facial_landmark_get_result_count() to get the number of results and mv_facial_landmark_get_position() to get the position of each landmark.

Related Features

This API is related with the following features:

http://tizen.org/feature/vision.inference.image
http://tizen.org/feature/vision.inference.face

It is recommended to design feature related codes in your application for reliability.
You can check if a device supports the related features for this API by using System Information, thereby controlling the procedure of your application.
To ensure your application is only running on the device with specific features, please define the features in your manifest file using the manifest editor in the SDK.
More details on featuring your application can be found from Feature Element.

Functions
int	mv_facial_landmark_create (mv_facial_landmark_h *handle)
	Creates a inference handle for facial landmark object.
int	mv_facial_landmark_destroy (mv_facial_landmark_h handle)
	Destroys inference handle and releases all its resources.
int	mv_facial_landmark_configure (mv_facial_landmark_h handle)
	Configures the backend for the facial landmark inference.
int	mv_facial_landmark_prepare (mv_facial_landmark_h handle)
	Prepares the facial landmark inference.
int	mv_facial_landmark_inference (mv_facial_landmark_h handle, mv_source_h source)
	Performs the facial landmark inference on the source.
int	mv_facial_landmark_inference_async (mv_facial_landmark_h handle, mv_source_h source)
	Performs asynchronously the facial landmark inference on the source.
int	mv_facial_landmark_get_result_count (mv_facial_landmark_h handle, unsigned long frame_number, unsigned int result_cnt)
	Gets the result count to objects.
int	mv_facial_landmark_get_position (mv_facial_landmark_h handle, unsigned int index, unsigned int pos_x, unsigned int pos_y)
	Gets the facial landmark position values to a given index.
Typedefs
typedef void *	mv_facial_landmark_h
	The facial landmark object handle.

Typedef Documentation

typedef void* mv_facial_landmark_h

The facial landmark object handle.

Since :: 9.0

Function Documentation

int mv_facial_landmark_configure ( mv_facial_landmark_h handle )

Configures the backend for the facial landmark inference.

Since :: 9.0

Parameters:

[in] handle The handle to the inference

Returns:: 0 on success, otherwise a negative error value

Return values:

MEDIA_VISION_ERROR_NONE	Successful
MEDIA_VISION_ERROR_NOT_SUPPORTED	Not supported
MEDIA_VISION_ERROR_INVALID_PARAMETER	Invalid parameter
MEDIA_VISION_ERROR_INVALID_OPERATION	Invalid operation
MEDIA_VISION_ERROR_OUT_OF_MEMORY	Out of memory

int mv_facial_landmark_create ( mv_facial_landmark_h * handle )

Creates a inference handle for facial landmark object.

Use this function to create a inference handle. After the creation the facial landmark task has to be prepared with mv_facial_landmark_prepare() function to prepare a network for the inference.

Since :: 9.0

Remarks:: The handle should be released using mv_facial_landmark_destroy().

Parameters:

[out] handle The handle to the inference to be created.

Returns:: 0 on success, otherwise a negative error value

Return values:

MEDIA_VISION_ERROR_NONE	Successful
MEDIA_VISION_ERROR_NOT_SUPPORTED	Not supported
MEDIA_VISION_ERROR_INVALID_PARAMETER	Invalid parameter
MEDIA_VISION_ERROR_INTERNAL	Internal Error

 #include <mv_facial_landmark.h>
 ...
 mv_facial_landmark_h handle = NULL;
 mv_facial_landmark_create(&handle);
 ...
 mv_facial_landmark_destroy(handle);

See also:: mv_facial_landmark_destroy(); mv_facial_landmark_prepare()

int mv_facial_landmark_destroy ( mv_facial_landmark_h handle )

Destroys inference handle and releases all its resources.

Since :: 9.0

Parameters:

[in] handle The handle to the inference to be destroyed.

Returns:: 0 on success, otherwise a negative error value

Return values:

MEDIA_VISION_ERROR_NONE	Successful
MEDIA_VISION_ERROR_NOT_SUPPORTED	Not supported
MEDIA_VISION_ERROR_INVALID_PARAMETER	Invalid parameter

Precondition:: Create inference handle by using mv_facial_landmark_create()

See also:: mv_facial_landmark_create()

int mv_facial_landmark_get_position	(	mv_facial_landmark_h	handle,
		unsigned int	index,
		unsigned int *	pos_x,
		unsigned int *	pos_y
	)

Gets the facial landmark position values to a given index.

Since :: 9.0

Remarks:: pos_x and pos_y arrays are allocated internally by the framework and will remain valid until the handle is released. Please do not deallocate them directly, and if you want to use them after the handle is released, please copy them to user memory and use the copy.

This function operates differently depending on the inference request method.

After mv_facial_landmark_inference() calls, this function returns facial landmark positions immediately.
After mv_facial_landmark_inference_async() calls, this function can be blocked until the asynchronous inference request is completed or the timeout occurs if no result within 3 seconds.

Additionally, after calling the mv_facial_landmark_inference_async() function, the function operates in asynchronous mode until the handle is released.

Parameters:

[in]	handle	The handle to the inference
[in]	index	A result index.
[out]	pos_x	An array containing x-coordinate values.
[out]	pos_y	An array containing y-coordinate values.

Returns:: 0 on success, otherwise a negative error value

Return values:

MEDIA_VISION_ERROR_NONE	Successful
MEDIA_VISION_ERROR_NOT_SUPPORTED	Not supported
MEDIA_VISION_ERROR_INVALID_PARAMETER	Invalid parameter

Precondition:: Create a source handle by calling mv_create_source(); Create an inference handle by calling mv_facial_landmark_create(); Prepare an inference by calling mv_facial_landmark_configure(); Prepare an inference by calling mv_facial_landmark_prepare(); Prepare an inference by calling mv_facial_landmark_inference(); Get result count by calling mv_facial_landmark_get_result_count()

int mv_facial_landmark_get_result_count	(	mv_facial_landmark_h	handle,
		unsigned long *	frame_number,
		unsigned int *	result_cnt
	)

Gets the result count to objects.

Since :: 9.0

Parameters:

[in]	handle	The handle to the inference
[out]	frame_number	A frame number inferenced.
[out]	result_cnt	A number of results.

Returns:: 0 on success, otherwise a negative error value

Return values:

MEDIA_VISION_ERROR_NONE	Successful
MEDIA_VISION_ERROR_NOT_SUPPORTED	Not supported
MEDIA_VISION_ERROR_INVALID_PARAMETER	Invalid parameter

Precondition:: Create a source handle by calling mv_create_source(); Create an inference handle by calling mv_facial_landmark_create(); Prepare an inference by calling mv_facial_landmark_configure(); Prepare an inference by calling mv_facial_landmark_prepare(); Request an inference by calling mv_facial_landmark_inference()

int mv_facial_landmark_inference	(	mv_facial_landmark_h	handle,
		mv_source_h	source
	)

Performs the facial landmark inference on the source.

Since :: 9.0

Remarks:: This function is synchronous and may take considerable time to run.

Parameters:

[in]	handle	The handle to the inference
[in]	source	The handle to the source of the media

Returns:: 0 on success, otherwise a negative error value

Return values:

MEDIA_VISION_ERROR_NONE	Successful
MEDIA_VISION_ERROR_NOT_SUPPORTED	Not supported
MEDIA_VISION_ERROR_INVALID_PARAMETER	Invalid parameter
MEDIA_VISION_ERROR_NOT_SUPPORTED_FORMAT	Source colorspace isn't supported

Precondition:: Create a source handle by calling mv_create_source(); Create an inference handle by calling mv_facial_landmark_create(); Prepare an inference by calling mv_facial_landmark_configure(); Prepare an inference by calling mv_facial_landmark_prepare()

Inference Example

int main()
{
    int error_code = 0;
    mv_facial_landmark_h handle;
    mv_source_h mv_source = NULL;

    error_code = mv_create_source(&mv_source);
    if (error_code != MEDIA_VISION_ERROR_NONE)
        dlog_print(DLOG_ERROR, LOG_TAG, "error code = %d", error_code);

    load_image_to_source("/path/to/image", mv_source);

    error_code = mv_facial_landmark_create(&handle);
    if (error_code != MEDIA_VISION_ERROR_NONE)
        dlog_print(DLOG_ERROR, LOG_TAG, "error code = %d", error_code);

    error_code = mv_facial_landmark_configure(handle);
    if (error_code != MEDIA_VISION_ERROR_NONE)
        dlog_print(DLOG_ERROR, LOG_TAG, "error code = %d", error_code);

    error_code = mv_facial_landmark_prepare(handle);
    if (error_code != MEDIA_VISION_ERROR_NONE)
        dlog_print(DLOG_ERROR, LOG_TAG, "error code = %d", error_code);

    error_code = mv_facial_landmark_inference(handle, mv_source);
    if (error_code != MEDIA_VISION_ERROR_NONE)
        dlog_print(DLOG_ERROR, LOG_TAG, "error code = %d", error_code);

    unsigned long frame_number;
    unsigned int cnt;

    int error_code = mv_facial_landmark_get_result_count(handle, &frame_number, &cnt);
    if (error_code == MEDIA_VISION_ERROR_INVALID_OPERATION)
        break;

    for (unsigned int idx = 0; idx < cnt; ++idx) {
        unsigned int pos_x, pos_y;

        error_code = mv_facial_landmark_get_position(handle, idx, &pos_x, &pos_y);
        if (error_code != MEDIA_VISION_ERROR_NONE)
            dlog_print(DLOG_ERROR, LOG_TAG, "error code = %d", error_code);

        dlog_print(DLOG_INFO, LOG_TAG, "frame = %lu, idx = %u, x = %u, y = %u", frame_number, idx, pos_x, pos_y);
    }

    error_code = mv_destroy_source(mv_source);
    if (error_code != MEDIA_VISION_ERROR_NONE)
        dlog_print(DLOG_ERROR, LOG_TAG, "error code = %d", error_code);

    error_code = mv_facial_landmark_destroy(handle);
    if (error_code != MEDIA_VISION_ERROR_NONE)
        dlog_print(DLOG_ERROR, LOG_TAG, "error code = %d", error_code);
}

int mv_facial_landmark_inference_async	(	mv_facial_landmark_h	handle,
		mv_source_h	source
	)

Performs asynchronously the facial landmark inference on the source.

Since :: 9.0

Remarks:: This function operates asynchronously, so it returns immediately upon invocation. The inference results are inserted into the outgoing queue within the framework in the order of processing, and the results can be obtained through mv_facial_landmark_get_position().

Parameters:

[in]	handle	The handle to the inference
[in]	source	The handle to the source of the media

Returns:: 0 on success, otherwise a negative error value

Return values:

MEDIA_VISION_ERROR_NONE	Successful
MEDIA_VISION_ERROR_NOT_SUPPORTED	Not supported
MEDIA_VISION_ERROR_INVALID_PARAMETER	Invalid parameter
MEDIA_VISION_ERROR_NOT_SUPPORTED_FORMAT	Source colorspace isn't supported

Precondition:: Create a source handle by calling mv_create_source(); Create an inference handle by calling mv_facial_landmark_create(); Prepare an inference by calling mv_facial_landmark_configure(); Prepare an inference by calling mv_facial_landmark_prepare()

Async Inference Example

void facial_landmark_callback(void *user_data)
{
    mv_facial_landmark_h handle = (mv_facial_landmark_h) user_data;
    bool is_loop_exit = false;

    while (!is_loop_exit) {
        unsigned long frame_number;
        unsigned int cnt;

        int error_code = mv_facial_landmark_get_result_count(handle, &frame_number, &cnt);
        if (error_code == MEDIA_VISION_ERROR_INVALID_OPERATION)
            break;

        for (unsigned int idx = 0; idx < cnt; ++idx) {
            unsigned int pos_x, pos_y;

            error_code = mv_facial_landmark_get_position(handle, idx, &pos_x, &pos_y);
            if (error_code != MEDIA_VISION_ERROR_NONE)
                dlog_print(DLOG_ERROR, LOG_TAG, "error code = %d", error_code);

            dlog_print(DLOG_INFO, LOG_TAG, "frame = %lu, idx = %u, x = %u, y = %u", frame_number, idx, pos_x, pos_y);

            if (frame_number > MAX_INFERENCE_ITERATION - 10)
                is_loop_exit = true;
        }
    }
}

int main()
{
    int error_code = 0;
    mv_facial_landmark_h handle;

    error_code = mv_facial_landmark_create(&handle);
    if (error_code != MEDIA_VISION_ERROR_NONE)
        dlog_print(DLOG_ERROR, LOG_TAG, "error code = %d", error_code);

    error_code = mv_facial_landmark_configure(handle);
    if (error_code != MEDIA_VISION_ERROR_NONE)
        dlog_print(DLOG_ERROR, LOG_TAG, "error code = %d", error_code);

    error_code = mv_facial_landmark_prepare(handle);
    if (error_code != MEDIA_VISION_ERROR_NONE)
        dlog_print(DLOG_ERROR, LOG_TAG, "error code = %d", error_code);

    pthread_t thread_id;

    for (unsigned int iter = 0; iter < MAX_INFERENCE_ITERATION; ++iter) {
        mv_source_h mv_source = NULL;

        error_code = mv_create_source(&mv_source);
        if (error_code != MEDIA_VISION_ERROR_NONE)
            dlog_print(DLOG_ERROR, LOG_TAG, "error code = %d", error_code);

        load_image_to_source("/path/to/image", mv_source);

        error_code = mv_facial_landmark_inference_async(handle, mv_source);
        if (error_code != MEDIA_VISION_ERROR_NONE)
            dlog_print(DLOG_ERROR, LOG_TAG, "error code = %d", error_code);

        if (iter == 0)
            pthread_create(&thread_id, NULL, facial_landmark_callback, handle);

        error_code = mv_destroy_source(mv_source);
        if (error_code != MEDIA_VISION_ERROR_NONE)
            dlog_print(DLOG_ERROR, LOG_TAG, "error code = %d", error_code);
    }

    /* This thread will be exited after inference is performed MAX_INFERENCE_ITERATION times. */
    pthread_join(thread_id, NULL);

    error_code = mv_facial_landmark_destroy(handle);
    if (error_code != MEDIA_VISION_ERROR_NONE)
        dlog_print(DLOG_ERROR, LOG_TAG, "error code = %d", error_code);
}

int mv_facial_landmark_prepare ( mv_facial_landmark_h handle )

Prepares the facial landmark inference.

Use this function to prepare the facial landmark inference based on the configured network.

Since :: 9.0

Parameters:

[in] handle The handle to the inference.

Returns:: 0 on success, otherwise a negative error value

Return values:

MEDIA_VISION_ERROR_NONE	Successful
MEDIA_VISION_ERROR_NOT_SUPPORTED	Not supported
MEDIA_VISION_ERROR_INVALID_PARAMETER	Invalid parameter
MEDIA_VISION_ERROR_INVALID_DATA	Invalid model data
MEDIA_VISION_ERROR_OUT_OF_MEMORY	Out of memory
MEDIA_VISION_ERROR_INVALID_OPERATION	Invalid operation
MEDIA_VISION_ERROR_NOT_SUPPORTED_FORMAT	Not supported format

Required Header

Overview

Related Features

Functions

Typedefs

Typedef Documentation

Function Documentation