Pipeline API
The following functionalities are provided:
- Creation of NNStreamer pipelines.
- Starting and stopping processing in NNStreamer pipeline.
- Manipulation of pipeline node properties.
- Feeding the data into pipeline.
- Getting the output from pipeline.
- Development of custom pipeline filters.
- Switching direction of the data flow inside pipeline.
- Starting and stopping data flow in pipeline branches.
NNStreamer is a set of plugins extending the capabilities of the GStreamer framework. See the GStreamer documentation to learn more about the multimedia pipelines. To get descriptions of pipeline nodes you can also use gst-inspect-1.0 command line tool.
Remark: In order to access files, a proper privilege has to be defined additionally:
- for accessing only internal storage using this API, a privilege http://tizen.org/privilege/mediastorage must be provided.
- for accessing only external storage using this API, a privilege http://tizen.org/privilege/externalstorage must be provided.
- for accessing internal and external storage using this API, privileges (http://tizen.org/privilege/mediastorage and http://tizen.org/privilege/externalstorage) must be provided.
Remark: In order to access device's camera and recorder, a proper privilege has to be defined additionally:
- for accessing only camera using this API, a privilege http://tizen.org/privilege/camera must be provided.
- for accessing only recorder using this API, a privilege http://tizen.org/privilege/recorder must be provided.
- for accessing camera and recorder using this API, privileges (http://tizen.org/privilege/camera and http://tizen.org/privilege/recorder) must be provided.
For more information on the Pipeline API features, see Pipeline Guide.
Since: 6.5
Table of Contents
- 1. Type Definitions
- 1.1. PipelineState
- 1.2. Property
- 1.3. PropertyType
- 1.4. SwitchType
- 2. Interfaces
- 2.1. MachineLearningPipeline
- 2.2. Pipeline
- 2.3. NodeInfo
- 2.4. Source
- 2.5. Switch
- 2.6. Valve
- 2.7. PipelineStateChangeListener
- 2.8. SinkListener
- 2.9. CustomFilter
- 3. Related Feature
- 4. Full WebIDL
Summary of Interfaces and Methods
Interface | Method |
---|---|
MachineLearningPipeline |
void registerCustomFilter (DOMString filterName, CustomFilter filter, TensorsInfo inputInfo, TensorsInfo outputInfo, optional ErrorCallback? errorCallback)
void unregisterCustomFilter (DOMString filterName)
|
Pipeline |
void start ()
void stop ()
void dispose ()
NodeInfo getNodeInfo (DOMString name)
void registerSinkListener (DOMString sinkName, SinkListener sinkListener)
void unregisterSinkListener (DOMString sinkName)
|
NodeInfo | |
Source | void inputData (TensorsData data) |
Switch |
DOMString[] getPadList ()
void select (DOMString padName)
|
Valve | void setOpen (boolean open) |
PipelineStateChangeListener | void onstatechange (PipelineState newState) |
SinkListener | void ondata (DOMString sinkName, TensorsData data) |
CustomFilter |
1. Type Definitions
1.1. PipelineState
enum PipelineState { "NULL", "PAUSED", "PLAYING", "READY", "UNKNOWN" };
Since: 6.5
- NULL - the state of the pipeline after calling Pipeline::dispose().
- READY - a transient state of the pipeline.
- PAUSED - a state of the pipeline ready to make an inference.
- PLAYING - a state of the pipeline that currently makes inference.
- UNKNOWN - unknown state.
Remark: The pipeline can remain in blue, steady states for longer, whereas white states are transient. For more information, see the GStreamer documentation.
1.2. Property
typedef (boolean or DOMString or long or long long or double or unsigned long or unsigned long long) Property;
Since: 6.5
1.3. PropertyType
enum PropertyType { "BOOLEAN", "DOUBLE", "ENUM", "INT32", "INT64", "UINT32", "UINT64", "STRING" };
Since: 6.5
- BOOLEAN - a boolean.
- DOUBLE - a double-precision floating point number.
- ENUM - an enum.
- INT32 - a 32-bit integer.
- INT64 - a 64-bit integer.
- UINT32 - a 32-bit unsigned integer.
- UINT64 - a 64-bit unsigned integer.
- STRING - a string.
2. Interfaces
2.1. MachineLearningPipeline
[NoInterfaceObject] interface MachineLearningPipeline { Pipeline createPipeline(DOMString description, optional PipelineStateChangeListener? listener) raises(WebAPIException); void registerCustomFilter(DOMString filterName, CustomFilter filter, TensorsInfo inputInfo, TensorsInfo outputInfo, optional ErrorCallback? errorCallback) raises(WebAPIException); void unregisterCustomFilter(DOMString filterName) raises(WebAPIException); };
Since: 6.5
Methods
-
createPipeline
-
Creates a machine learning pipeline.
Pipeline createPipeline(DOMString description, optional PipelineStateChangeListener? listener);
Since: 6.5
A pipeline is created from a description, which is a string containing all the nodes and links. For more information about pipeline description, see the Pipeline Description section of gst-launch-1.0 documentation.
Remark: Pipeline creation is an asynchronous process, during which the pipeline state transitions through READY to PAUSED. The pipeline is ready to use when it is in PAUSED state.
Parameters:
- description: A description of the pipeline.
- listener [optional] [nullable]: Listener function that will be called when pipeline state changes.
Return value:
-
Pipeline:
A newly created Pipeline object.
Exceptions:
- WebAPIException
with error type NotSupportedError, if the feature is not supported.
with error type SecurityError, if the application does not have the privilege to call this method.
with error type TypeMismatchError, if any input attribute is not compatible with the expected type for this attribute.
with error type AbortError, if the description is invalid or any other error occurs.
Code example:
var pipelineDescription = "videotestsrc ! tizenwlsink"; function pipelineStateChangeListener(newState) { console.log("New pipeline state: " + newState); } var pipeline = tizen.ml.pipeline.createPipeline(pipelineDescription, pipelineStateChangeListener); /* after use */ pipeline.dispose();
Output example:
New pipeline state: READY New pipeline state: PAUSED
-
registerCustomFilter
-
Registers a CustomFilter, which implements a custom transform to the data coming through the pipeline.
void registerCustomFilter(DOMString filterName, CustomFilter filter, TensorsInfo inputInfo, TensorsInfo outputInfo, optional ErrorCallback? errorCallback);
Since: 6.5
errorCallback is used to report errors that occur during processing of the data by the filter. When an error is caused by CustomFilter::filter or an internal error occurs during processing of the data, the effect is the same as if CustomFilter::filter returned -1 and the errorCallback method is called with these error types:
- InvalidValuesError - if the value returned from CustomFilter::filter is not compliant with legal values defined in CustomFilter::filter reference.
- AbortError - if an exception thrown from CustomFilter::filter or any other error occurs.
Remark: Performance of a computationally intensive custom filter implemented in JavaScript may be unsatisfactory due to the overhead of transferring the data between the native pipeline implementation and a web application.
Remark: errorCallback is not triggered when CustomFilter returns a negative status.
Parameters:
- filterName: Name of the filter.
- filter: The function used as the data filter.
- inputInfo: Information about tensors taken as input.
- outputInfo: Information about tensors returned from the filter.
- errorCallback [optional] [nullable]: The method to call when an error occurs during processing of the data by the filter.
Exceptions:
- WebAPIException
with error type InvalidValuesError, if filterName is invalid or a custom filter with this name is already registered.
with error type NotSupportedError, if the feature is not supported.
with error type TypeMismatchError, if any input attribute is not compatible with the expected type for this attribute.
with error type AbortError, if any other error occurs.
Code example:
var inputInfo = new tizen.ml.TensorsInfo(); inputInfo.addTensorInfo("3D", "UINT8", [4, 20, 15, 1]); var outputInfo = new tizen.ml.TensorsInfo(); outputInfo.addTensorInfo("flat", "UINT8", [1200]); var flattenFilter = function(inputData, outputData) { console.log("Custom filter called"); var rawInputData = inputData.getTensorRawData(0); outputData.setTensorRawData(0, rawInputData.data); return 0; }; function errorCallback(error) { console.error("flattenFilter error: " + error.name + ". Reason: " + error.message); } tizen.ml.pipeline.registerCustomFilter("flattenFilter", flattenFilter, inputInfo, outputInfo); var pipelineDescription = "videotestsrc num-buffers=3 " + "! video/x-raw,width=20,height=15,format=BGRA " + "! tensor_converter " + "! tensor_filter framework=custom-easy model=flattenFilter " + "! fakesink"; function pipelineStateChangeListener(newState) { console.log("New pipeline state: " + newState); } var pipeline = tizen.ml.pipeline.createPipeline(pipelineDescription, pipelineStateChangeListener); pipeline.start();
Output example:
New pipeline state: READY Custom filter called New pipeline state: PAUSED New pipeline state: PLAYING Custom filter called Custom filter called
-
unregisterCustomFilter
-
Unregisters a pipeline's CustomFilter.
void unregisterCustomFilter(DOMString filterName);
Since: 6.5
Remark: CustomFilter can only be unregistered when it is currently not processing data. Stop pipelines using the filter to ensure that condition. Otherwise an InvalidStateError may be thrown.
Parameters:
- filterName: The name of the filter.
Exceptions:
- WebAPIException
with error type InvalidStateError, if the filter is currently processing data.
with error type InvalidValuesError, if filterName is invalid.
with error type NotSupportedError, if the feature is not supported.
with error type TypeMismatchError, if any input attribute is not compatible with the expected type for this attribute.
with error type AbortError, if any other error occurs.
Code example:
/* Suppose that a CustomFilter named "flattenFilter" has been registered and is */ /* no longer needed */ tizen.ml.pipeline.unregisterCustomFilter("flattenFilter");
2.2. Pipeline
[NoInterfaceObject] interface Pipeline { readonly attribute PipelineState state raises(WebAPIException); void start() raises(WebAPIException); void stop() raises(WebAPIException); void dispose() raises(WebAPIException); NodeInfo getNodeInfo(DOMString name) raises(WebAPIException); Source getSource(DOMString name) raises(WebAPIException); Switch getSwitch(DOMString name) raises(WebAPIException); Valve getValve(DOMString name) raises(WebAPIException); void registerSinkListener(DOMString sinkName, SinkListener sinkListener) raises(WebAPIException); void unregisterSinkListener(DOMString sinkName) raises(WebAPIException); };
Since: 6.5
Attributes
-
readonly
PipelineState stateThe current state of the pipeline.
Since: 6.5
Exceptions:
- WebAPIException
with error type NotSupportedError, if the feature is not supported.
with error type AbortError, if any other error occurs.
- WebAPIException
Methods
-
start
-
Starts the pipeline.
void start();
Since: 6.5
The pipeline starts asynchronously and its state may not be set to PLAYING right after start() returns. Register a PipelineStateChangeListener with MachineLearningPipeline::createPipeline() to watch state changes.
Remark: Before calling start(), the Pipeline::state should be PAUSED.
Remark: Pipeline::state after successful start() will be set to PLAYING.
Exceptions:
- WebAPIException
with error type NotFoundError, if the pipeline was disposed.
with error type NotSupportedError, if the feature is not supported.
with error type AbortError, if any other error occurs.
Code example:
var pipelineDescription = "videotestsrc ! tizenwlsink"; function pipelineStateChangeListener(newState) { console.log("New pipeline state: " + newState); } var pipeline = tizen.ml.pipeline.createPipeline(pipelineDescription, pipelineStateChangeListener); pipeline.start();
Output example:
New pipeline state: READY New pipeline state: PAUSED New pipeline state: PLAYING
- WebAPIException
-
stop
-
Stops the pipeline.
void stop();
Since: 6.5
The pipeline stops asynchronously and its state may not be set to PAUSED right after stop() returns. Register a PipelineStateChangeListener with MachineLearningPipeline::createPipeline() to watch state changes.
Remark: Before calling stop(), state should be PLAYING.
Remark: The pipeline state after successful stop() will be set to PAUSED.
Exceptions:
- WebAPIException
with error type NotFoundError, if the pipeline was disposed.
with error type NotSupportedError, if the feature is not supported.
with error type AbortError, if any other error occurs.
Code example:
var pipelineDescription = "videotestsrc ! tizenwlsink"; function pipelineStateChangeListener(newState) { console.log("New pipeline state: " + newState); } var pipeline = tizen.ml.pipeline.createPipeline(pipelineDescription, pipelineStateChangeListener); pipeline.start(); /* Application needs to have background-support enabled */ /* A few moments later pipeline is stopped and disposed */ setTimeout(function() { pipeline.stop(); pipeline.dispose(); }, 5000);
Output example:
New pipeline state: READY New pipeline state: PAUSED New pipeline state: PLAYING New pipeline state: PAUSED
- WebAPIException
-
dispose
-
Releases the resources allocated by the pipeline.
void dispose();
Since: 6.5
A pipeline may be expensive in terms of allocated resources such as memory, and dispose() releases them.
Remark: Before calling dispose(), Pipeline::state should be PLAYING or PAUSED.
Remark: Pipeline::state after successful dispose() will be set to NULL.
Remark: Pipeline can only be disposed when no CustomFilter is currently processing data. Dispose pipelines when no custom filter is processing data to ensure that condition. Otherwise an InvalidStateError will be thrown.
Exceptions:
- WebAPIException
with error type InvalidStateError, if any CustomFilter is currently processing data.
with error type NotFoundError, if the pipeline has already been disposed.
with error type NotSupportedError, if the feature is not supported.
with error type AbortError, if any other error occurs.
Code example:
var pipelineDescription = "videotestsrc ! tizenwlsink"; function pipelineStateChangeListener(newState) { console.log("New pipeline state: " + newState); } var pipeline = tizen.ml.pipeline.createPipeline(pipelineDescription, pipelineStateChangeListener); pipeline.start(); /* Application needs to have background-support enabled */ /* A few moments later pipeline is stopped and disposed */ setTimeout(function() { pipeline.stop(); pipeline.dispose(); }, 5000);
Output example:
New pipeline state: READY New pipeline state: PAUSED New pipeline state: PLAYING New pipeline state: PAUSED
- WebAPIException
-
getNodeInfo
-
Gets a NodeInfo object allowing to get and set pipeline node's properties.
NodeInfo getNodeInfo(DOMString name);
Since: 6.5
Parameters:
- name: Name of the pipeline node.
Return value:
-
NodeInfo:
The retrieved NodeInfo object.
Exceptions:
- WebAPIException
with error type InvalidValuesError, if name is invalid.
with error type NotFoundError, if the pipeline was disposed.
with error type NotSupportedError, if the feature is not supported.
with error type TypeMismatchError, if any input attribute is not compatible with the expected type for this attribute.
with error type AbortError, if any other error occurs.
Code example:
var pipelineDescription = "videotestsrc name=vsrc is-live=true " + "! videoconvert " + "! videoscale name=vscale " + "! video/x-raw,format=RGBx,width=224,height=224,framerate=60/1 " + "! tensor_converter " + "! valve name=valvex " + "! input-selector name=is01 " + "! tensor_sink name=sinkx"; var pipeline = tizen.ml.pipeline.createPipeline(pipelineDescription); var inputSelectorInfo = pipeline.getNodeInfo("is01"); console.log("Retrieved NodeInfo: " + inputSelectorInfo.name);
Output example:
Retrieved NodeInfo: is01
-
getSource
-
Gets a Source object that allows input to the pipeline.
Source getSource(DOMString name);
Since: 6.5
Remark: This function can only retrieve sources meant to be used as application's data feeds, in particular appsrc. Use gst-inspect-1.0 tool to check, if a given source can be used to input data from an application.
Parameters:
- name: Name of the source.
Return value:
-
Source:
The retrieved Source object.
Exceptions:
- WebAPIException
with error type InvalidStateError, if the pipeline is not ready yet.
with error type InvalidValuesError, if name is invalid.
with error type NotFoundError, if the pipeline was disposed.
with error type NotSupportedError, if the feature is not supported.
with error type TypeMismatchError, if any input attribute is not compatible with the expected type for this attribute.
with error type AbortError, if any other error occurs.
Code example:
var pipelineDescription = "appsrc name=srcx " + "! other/tensor,dimension=(string)1:1:1:1,type=(string)int8,framerate=(fraction)0/1 " + "! fakesink"; var pipeline = tizen.ml.pipeline.createPipeline(pipelineDescription); var source = pipeline.getSource("srcx"); console.log("Retrieved source: " + source.name);
Output example:
Retrieved source: srcx
-
getSwitch
-
Gets a Switch object that allows to select a pipeline branch to be used as a source or sink.
Switch getSwitch(DOMString name);
Since: 6.5
Parameters:
- name: Name of the switch.
Return value:
-
Switch:
The retrieved Switch object.
Exceptions:
- WebAPIException
with error type InvalidStateError, if the pipeline is not ready yet.
with error type InvalidValuesError, if name is invalid.
with error type NotFoundError, if the pipeline was disposed.
with error type NotSupportedError, if the feature is not supported.
with error type TypeMismatchError, if any input attribute is not compatible with the expected type for this attribute.
with error type AbortError, if any other error occurs.
Code example:
var pipelineDescription = "videotestsrc is-live=true " + "! videoconvert " + "! tensor_converter " + "! output-selector name=outs outs.src_0 " + "! tensor_sink name=sink0 async=false outs.src_1 " + "! tensor_sink name=sink1 async=false "; var pipeline = tizen.ml.pipeline.createPipeline(pipelineDescription); var outsSwitch = pipeline.getSwitch("outs"); console.log("Retrieved switch: " + outsSwitch.name + " (type: " + outsSwitch.type + ")");
Output example:
Retrieved switch: outs (type: OUTPUT_SELECTOR)
-
getValve
-
Gets a Valve object that allows to start and stop streaming data to a branch of a pipeline.
Valve getValve(DOMString name);
Since: 6.5
Parameters:
- name: Name of the valve.
Return value:
-
Valve:
The retrieved Valve object.
Exceptions:
- WebAPIException
with error type InvalidValuesError, if name is invalid.
with error type NotFoundError, if the pipeline was disposed.
with error type NotSupportedError, if the feature is not supported.
with error type TypeMismatchError, if any input attribute is not compatible with the expected type for this attribute.
with error type AbortError, if any other error occurs.
Code example:
var pipelineDescription = "videotestsrc is-live=true " + "! videoconvert " + "! videoscale " + "! video/x-raw,format=RGBx,width=16,height=16,framerate=10/1 " + "! tensor_converter " + "! valve name=valve1 " + "! fakesink"; var pipeline = tizen.ml.pipeline.createPipeline(pipelineDescription); var valve = pipeline.getValve("valve1"); console.log( "Retrieved valve: " + valve.name + " (state: " + (valve.isOpen ? "open" : "closed") + ")");
Output example:
Retrieved valve: valve1 (state: open)
-
registerSinkListener
-
Registers a SinkListener for a given sink. The listener is used to get output data from a pipeline.
void registerSinkListener(DOMString sinkName, SinkListener sinkListener);
Since: 6.5
Remark: This function can only register listeners for sinks meant to provide a direct output to an application, in particular appsink. Use gst-inspect-1.0 tool to check, if a given sink can be used as an application output.
Remark: Before calling registerSinkListener(), the pipeline state should be PAUSED.
Parameters:
- sinkName: Name of the sink.
- sinkListener: The listener to be registered and used as pipeline's output.
Exceptions:
- WebAPIException
with error type InvalidValuesError, if sinkName is invalid.
with error type NotFoundError, if the pipeline was disposed.
with error type NotSupportedError, if the feature is not supported.
with error type TypeMismatchError, if any input attribute is not compatible with the expected type for this attribute.
with error type AbortError, if any other error occurs.
Code example:
var pipelineDescription = "videotestsrc num-buffers=3 " + "! videoconvert " + "! tensor_converter " + "! tensor_sink name=sinkx"; var pipeline = tizen.ml.pipeline.createPipeline(pipelineDescription); pipeline.registerSinkListener("sinkx", function(sinkName, data) { console.log("SinkListener for " + sinkName + " sink called. Data dimensions: " + data.tensorsInfo.getDimensions(0)); }); pipeline.start();
Output example:
SinkListener for sinkx sink called. Data dimensions: 4,320,240,1 SinkListener for sinkx sink called. Data dimensions: 4,320,240,1 SinkListener for sinkx sink called. Data dimensions: 4,320,240,1
-
unregisterSinkListener
-
Unregisters a sink's SinkListener.
void unregisterSinkListener(DOMString sinkName);
Since: 6.5
Remark: Before calling unregisterSinkListener(), the Pipeline::state should be PAUSED.
Parameters:
- sinkName: Name of the sink.
Exceptions:
- WebAPIException
with error type InvalidValuesError, if sinkName is invalid.
with error type NotFoundError, if the pipeline was disposed.
with error type NotSupportedError, if the feature is not supported.
with error type TypeMismatchError, if any input attribute is not compatible with the expected type for this attribute.
with error type AbortError, if any other error occurs.
Code example:
var pipelineDescription = "videotestsrc num-buffers=3 " + "! videoconvert " + "! tensor_converter " + "! tensor_sink name=sinkx"; var unregisterSinkListener = false; function pipelineStateChangeListener(newState) { /* pipelineStateChangeListener will be called with newState === "PAUSED" */ /* just after pipeline creation, but we don't want the SinkListener to be */ /* unregistered right then. */ /* Thus we need the unregisterSinkListener flag to signal when we want to */ /* unregister the SinkListener. */ if (newState === "PAUSED" && unregisterSinkListener) { pipeline.unregisterSinkListener("sinkx"); console.log("sinkx unregistered"); } } var pipeline = tizen.ml.pipeline.createPipeline(pipelineDescription, pipelineStateChangeListener); pipeline.registerSinkListener("sinkx", function(sinkName, data) { console.log("SinkListener for " + sinkName + " sink called. Data dimensions: " + data.tensorsInfo.getDimensions(0)); }); pipeline.start(); /* Application needs to have background-support enabled */ /* A few moments later pipeline is stopped and disposed */ setTimeout(function() { unregisterSinkListener = true; pipeline.stop(); pipeline.dispose(); }, 5000);
Output example:
SinkListener for sinkx sink called. Data dimensions: 4,320,240,1 SinkListener for sinkx sink called. Data dimensions: 4,320,240,1 SinkListener for sinkx sink called. Data dimensions: 4,320,240,1 sinkx unregistered
2.3. NodeInfo
[NoInterfaceObject] interface NodeInfo { readonly attribute DOMString name; Property getProperty(DOMString name, PropertyType type) raises(WebAPIException); void setProperty(DOMString name, PropertyType type, Property value) raises(WebAPIException); };
Since: 6.5
Attributes
-
readonly
DOMString nameName of the node.
Since: 6.5
Methods
-
getProperty
-
Retrieves the value of node's property.
Property getProperty(DOMString name, PropertyType type);
Since: 6.5
Parameters:
- name: Name of the node.
- type: Data type of the property.
Return value:
-
Property:
Value of the retrieved property.
Exceptions:
- WebAPIException
with error type InvalidValuesError, if name is invalid.
with error type NotFoundError, if the node belongs to a disposed pipeline.
with error type NotSupportedError, if the feature is not supported.
with error type TypeMismatchError, if any input attribute is not compatible with the expected type for this attribute.
with error type AbortError, if any other error occurs.
Code example:
var pipelineDescription = "videotestsrc name=vsrc is-live=true " + "! videoconvert " + "! videoscale name=vscale " + "! video/x-raw,format=RGBx,width=224,height=224,framerate=60/1 " + "! tensor_converter " + "! valve name=valvex " + "! input-selector name=is01 " + "! tensor_sink name=sinkx"; var pipeline = tizen.ml.pipeline.createPipeline(pipelineDescription); var inputSelectorInfo = pipeline.getNodeInfo("is01"); inputSelectorInfo.setProperty("sync-streams", "BOOLEAN", true); var syncStreams = inputSelectorInfo.getProperty("sync-streams", "BOOLEAN"); console.log("Retrieved sync-streams: " + syncStreams);
Output example:
Retrieved sync-streams: true
-
setProperty
-
Sets the value of node's property.
void setProperty(DOMString name, PropertyType type, Property value);
Since: 6.5
Parameters:
- name: Name of the property.
- type: The type of the property.
- value: The new value of the property.
Exceptions:
- WebAPIException
with error type InvalidValuesError, if any of the input parameters contain an invalid value.
with error type NotFoundError, if the node belongs to a disposed pipeline.
with error type NotSupportedError, if the feature is not supported.
with error type TypeMismatchError, if any input attribute is not compatible with the expected type for this attribute.
with error type AbortError, if any other error occurs.
Code example:
var pipelineDescription = "videotestsrc name=vsrc is-live=true " + "! videoconvert " + "! videoscale name=vscale " + "! video/x-raw,format=RGBx,width=224,height=224,framerate=60/1 " + "! tensor_converter " + "! valve name=valvex " + "! input-selector name=is01 " + "! tensor_sink name=sinkx"; var pipeline = tizen.ml.pipeline.createPipeline(pipelineDescription); var inputSelectorInfo = pipeline.getNodeInfo("is01"); inputSelectorInfo.setProperty("sync-streams", "BOOLEAN", true); var syncStreams = inputSelectorInfo.getProperty("sync-streams", "BOOLEAN"); console.log("Retrieved sync-streams: " + syncStreams);
Output example:
Retrieved sync-streams: true
2.4. Source
[NoInterfaceObject] interface Source { readonly attribute TensorsInfo inputTensorsInfo raises(WebAPIException); readonly attribute DOMString name; void inputData(TensorsData data) raises(WebAPIException); };
Since: 6.5
Attributes
-
readonly
TensorsInfo inputTensorsInfoThe information about the format of tensor input expected by the source.
Since: 6.5
Exceptions:
- WebAPIException
with error type NotFoundError, if the source belongs to a disposed pipeline.
with error type AbortError, if any other error occurs.
- WebAPIException
-
readonly
DOMString nameName of the source.
Since: 6.5
Methods
-
inputData
-
Feeds the source with input data.
void inputData(TensorsData data);
Since: 6.5
Parameters:
- data: The input data for the source.
Exceptions:
- WebAPIException
with error type InvalidStateError, if the pipeline is not ready yet.
with error type NotFoundError, if the source belongs to a disposed pipeline.
with error type NotSupportedError, if the feature is not supported.
with error type TypeMismatchError, if any input attribute is not compatible with the expected type for this attribute.
with error type AbortError, if any other error occurs.
Code example:
var pipelineDescription = "appsrc name=srcx " + "! other/tensor,dimension=(string)1:1:1:1,type=(string)int8,framerate=(fraction)0/1 " + "! fakesink"; var pipeline = tizen.ml.pipeline.createPipeline(pipelineDescription); pipeline.start(); var source = pipeline.getSource("srcx"); var inputInfo = new tizen.ml.TensorsInfo(); inputInfo.addTensorInfo("tensor1", "UINT8", [1, 1]); var inputData = inputInfo.getTensorsData(); for (var i = 0; i < 5; i++) { var data = [i]; console.log("Input: " + JSON.stringify(data)); inputData.setTensorRawData(0, data); source.inputData(inputData); } pipeline.stop(); pipeline.dispose();
Output example:
Input: [0] Input: [1] Input: [2] Input: [3] Input: [4]
2.5. Switch
[NoInterfaceObject] interface Switch { readonly attribute SwitchType type; readonly attribute DOMString name; DOMString[] getPadList() raises(WebAPIException); void select(DOMString padName) raises(WebAPIException); };
Since: 6.5
Attributes
-
readonly
SwitchType typeDetermines the switch type.
Since: 6.5
-
readonly
DOMString nameName of the switch.
Since: 6.5
Methods
-
getPadList
-
Retrieves the list of pad names of the switch.
DOMString[] getPadList();
Since: 6.5
Return value:
-
DOMString[]:
The list of switch's pads.
Exceptions:
- WebAPIException
with error type NotSupportedError, if the feature is not supported.
with error type NotFoundError, if the switch belongs to a disposed pipeline.
with error type InvalidStateError, if the pipeline is not ready yet.
with error type AbortError, if any other error occurs.
Code example:
var pipelineDescription = "videotestsrc is-live=true " + "! videoconvert " + "! tensor_converter " + "! output-selector name=outs outs.src_0 " + "! tensor_sink name=sink0 async=false outs.src_1 " + "! tensor_sink name=sink1 async=false"; var pipeline = tizen.ml.pipeline.createPipeline(pipelineDescription); var outsSwitch = pipeline.getSwitch("outs"); var pads = outsSwitch.getPadList(); console.log("Pads:"); for (var i = 0; i < pads.length; i++) { console.log(" " + pads[i]); }
Output example:
Pads: src_0 src_1
- WebAPIException
-
select
-
Selects a pad to be used as a source or sink of the switch node.
void select(DOMString padName);
Since: 6.5
The switch selects a sink if its Switch::type is OUTPUT_SELECTOR or it selects a source if its Switch::type is INPUT_SELECTOR.
Parameters:
- padName: Name of the pad to be used as a sink or source.
Exceptions:
- WebAPIException
with error type InvalidValuesError, if padName is invalid.
with error type NotFoundError, if the switch belongs to a disposed pipeline.
with error type NotSupportedError, if the feature is not supported.
with error type TypeMismatchError, if any input attribute is not compatible with the expected type for this attribute.
with error type AbortError, if any other error occurs.
Code example:
var pipelineDescription = "videotestsrc is-live=true " + "! videoconvert " + "! tensor_converter " + "! output-selector name=outs outs.src_0 " + "! tensor_sink name=sink0 async=false outs.src_1 " + "! tensor_sink name=sink1 async=false"; var pipeline = tizen.ml.pipeline.createPipeline(pipelineDescription); var outsSwitch = pipeline.getSwitch("outs"); outsSwitch.select("src_1");
2.6. Valve
[NoInterfaceObject] interface Valve { readonly attribute DOMString name; readonly attribute boolean isOpen raises(WebAPIException); void setOpen(boolean open) raises(WebAPIException); };
Since: 6.5
Valves are open by default.
See the the gstreamer documentation for more details.
Attributes
-
readonly
DOMString nameName of the valve.
Since: 6.5
-
readonly
boolean isOpenState of the valve.
Since: 6.5
Remark: isOpen is an alias for negated value of the Valve element's drop property in the GStreamer framework.
Exceptions:
- WebAPIException
with error type NotSupportedError, if the feature is not supported.
with error type NotFoundError, if the valve belongs to a disposed pipeline.
with error type AbortError, if any other error occurs.
- WebAPIException
Methods
-
setOpen
-
Enables or disables the flow of the data through the valve by setting it to open or closed, respectively.
void setOpen(boolean open);
Since: 6.5
Parameters:
- open: When it is true, the valve will be open and closed otherwise.
Exceptions:
- WebAPIException
with error type TypeMismatchError, if open argument is missing.
with error type NotSupportedError, if the feature is not supported.
with error type NotFoundError, if the valve belongs to a disposed pipeline.
with error type AbortError, if any other error occurs.
Code example:
var pipelineDescription = "videotestsrc is-live=true " + "! videoconvert " + "! videoscale " + "! video/x-raw,format=RGBx,width=16,height=16,framerate=10/1 " + "! tensor_converter " + "! valve name=valve1 " + "! fakesink"; var pipeline = tizen.ml.pipeline.createPipeline(pipelineDescription); var valve = pipeline.getValve("valve1"); console.log( "Retrieved valve: " + valve.name + " (state: " + (valve.isOpen ? "open" : "closed") + ")"); valve.setOpen(false); console.log("State: " + (valve.isOpen ? "open" : "closed"));
Output example:
Retrieved valve: valve1 (state: open) State: closed
2.7. PipelineStateChangeListener
[Callback=FunctionOnly, NoInterfaceObject] interface PipelineStateChangeListener { void onstatechange(PipelineState newState); };
Since: 6.5
Methods
-
onstatechange
-
Called when pipeline state changes.
void onstatechange(PipelineState newState);
Since: 6.5
Remark: Usage example can be found in MachineLearningPipeline::createPipeline()'s code example.
Parameters:
- newState: The new state of the pipeline.
2.8. SinkListener
[Callback=FunctionOnly, NoInterfaceObject] interface SinkListener { void ondata(DOMString sinkName, TensorsData data); };
Since: 6.5
Methods
-
ondata
-
Called when new data arrives to the sink.
void ondata(DOMString sinkName, TensorsData data);
Since: 6.5
Parameters:
- sinkName: Name of the sink.
- data: The data that arrived to the sink.
2.9. CustomFilter
[Callback=FunctionOnly, NoInterfaceObject] interface CustomFilter { long filter(TensorsData input, TensorsData output); };
Since: 6.5
Methods
-
filter
-
Called when data to be processed arrives to the filter.
long filter(TensorsData input, TensorsData output);
Since: 6.5
This user-defined callback transforms the input and writes the transformation result to output. output contents will be passed to the further stages of the pipeline.
Remark: The input and output are only valid within the callback. They are automatically disposed after callback's return. To use their contents after callback return, copy them to objects valid outside the callback.
Remark: The input and output cannot be disposed. Calling the TensorsData::dispose()has no effect.
Remark: The input cannot be modified. Calling TensorsData::setTensorRawData() on input has no effect.
Remark: The output passed to the callback contains random values. Use TensorsData::setTensorRawData() to modify it.
Remark: Returning a negative status value stops the pipeline and changes its state to UNKNOWN.
Remark: The only legal positive return value is 1. On an attempt of returning another positive status value, the errorCallback registered with MachineLearningPipeline::registerCustomFilter() will report an error and the pipeline will stop as if the callback would return -1.
Remark: Usage example can be found in MachineLearningPipeline::registerCustomFilter()'s code example.
Parameters:
- input: The data passed to filter to be processed.
- output: Filter's output.
Return value:
-
long:
The status of data processing.
This should be 0 when processing is successful, 1 when current data should be ignored
or a negative value to report an error.
3. Related Feature
To guarantee the running of the application on a device with Machine Learning feature, declare the following feature requirement in the config file:
To guarantee the running of the application on a device with Machine Learning Inference feature, declare the following feature requirement in the config file:
4. Full WebIDL
module Pipeline { typedef (boolean or DOMString or long or long long or double or unsigned long or unsigned long long) Property; enum PipelineState { "NULL", "PAUSED", "PLAYING", "READY", "UNKNOWN" }; enum PropertyType { "BOOLEAN", "DOUBLE", "ENUM", "INT32", "INT64", "UINT32", "UINT64", "STRING" }; enum SwitchType { "INPUT_SELECTOR", "OUTPUT_SELECTOR" }; [NoInterfaceObject] interface MachineLearningPipeline { Pipeline createPipeline(DOMString description, optional PipelineStateChangeListener? listener) raises(WebAPIException); void registerCustomFilter(DOMString filterName, CustomFilter filter, TensorsInfo inputInfo, TensorsInfo outputInfo, optional ErrorCallback? errorCallback) raises(WebAPIException); void unregisterCustomFilter(DOMString filterName) raises(WebAPIException); }; [NoInterfaceObject] interface Pipeline { readonly attribute PipelineState state raises(WebAPIException); void start() raises(WebAPIException); void stop() raises(WebAPIException); void dispose() raises(WebAPIException); NodeInfo getNodeInfo(DOMString name) raises(WebAPIException); Source getSource(DOMString name) raises(WebAPIException); Switch getSwitch(DOMString name) raises(WebAPIException); Valve getValve(DOMString name) raises(WebAPIException); void registerSinkListener(DOMString sinkName, SinkListener sinkListener) raises(WebAPIException); void unregisterSinkListener(DOMString sinkName) raises(WebAPIException); }; [NoInterfaceObject] interface NodeInfo { readonly attribute DOMString name; Property getProperty(DOMString name, PropertyType type) raises(WebAPIException); void setProperty(DOMString name, PropertyType type, Property value) raises(WebAPIException); }; [NoInterfaceObject] interface Source { readonly attribute TensorsInfo inputTensorsInfo raises(WebAPIException); readonly attribute DOMString name; void inputData(TensorsData data) raises(WebAPIException); }; [NoInterfaceObject] interface Switch { readonly attribute SwitchType type; readonly attribute DOMString name; DOMString[] getPadList() raises(WebAPIException); void select(DOMString padName) raises(WebAPIException); }; [NoInterfaceObject] interface Valve { readonly attribute DOMString name; readonly attribute boolean isOpen raises(WebAPIException); void setOpen(boolean open) raises(WebAPIException); }; [Callback=FunctionOnly, NoInterfaceObject] interface PipelineStateChangeListener { void onstatechange(PipelineState newState); }; [Callback=FunctionOnly, NoInterfaceObject] interface SinkListener { void ondata(DOMString sinkName, TensorsData data); }; [Callback=FunctionOnly, NoInterfaceObject] interface CustomFilter { long filter(TensorsData input, TensorsData output); }; };