VisionWorks Toolkit Reference

December 18, 2015 | 1.2 Release

 All Data Structures Namespaces Files Functions Variables Typedefs Enumerations Enumerator Macros Groups Pages

Detailed Description

The Harris Track Primitive.

This primitive detects and tracks Harris corners for the input image.

The primitive uses the following identifiers:

Corners classification

For each pixel the primitive calculates a gradient covariance matrix over a \( blockSize \times blockSize \) neighborhood:

\begin{eqnarray} G &=& \frac{blockSize}{2} \\ A(y, x) &=& \sum\limits_{v=-G}^G \sum\limits_{u=-G}^G (f_x(y + v, x + u))^2 \cdot w(v, u) \\ B(y, x) &=& \sum\limits_{v=-G}^G \sum\limits_{u=-G}^G (f_y(y + v, x + u))^2 \cdot w(v, u) \\ C(y, x) &=& \sum\limits_{v=-G}^G \sum\limits_{u=-G}^G f_x(y + v, x + u) \cdot f_y(y + v, x + u) \cdot w(v, u) \end{eqnarray}

Then, it computes the following characteristic:

\begin{eqnarray} trace(M(y, x)) &=& A(y, x) + B(y, x) \\ det(M(y, x)) &=& A(y, x) \cdot B(y, x) - C(y, x)^2 \\ H(y, x) &=& det(M(y, x)) - k \cdot trace(M(y, x))^2 \end{eqnarray}

A candidate pixel is classified as a corner, if its \( H(y, x) \) characteristic is greater than the specified threshold.

Optionally, a binary mask can be provided to filter out corners from specified regions. The primitive only leaves points with a non-zero mask value.

Cell-based Non-max Suppression

This primitive applies a cell-based, non-max suppression step on all detected corners to remove multiple or spurious responses.

The input image is split into cells (cell size is a primitive's parameter), and for each cell the corner with the highest strength is selected. If there are several corners with the same strength the bottom-most right corner is chosen.

Tracking

Additionally, this primitive can be used in the tracking scenario. It accepts an optional tracked features list from optical flow. The primitive extracts corners only for those cells that do not contain any tracked points.

The primitive applies cell-based, non-max suppressions for input tracked points, too, so only non-lost points with the highest strength are left per cell.

Example Code

Detection scenario
vx_float32 k = 0.04f;
vx_float32 threshold = 110;
vx_uint32 cell_size = 18;
vx_array corners = vxCreateArray(context, VX_TYPE_KEYPOINT, 3000);
nvxuHarrisTrack(context, src, corners, NULL, NULL, k, threshold, cell_size, NULL);
Tracking scenario
vx_float32 k = 0.04f;
vx_float32 threshold = 110;
vx_uint32 cell_size = 18;
vx_pyramid pyr_exemplar = vxCreatePyramid(context, pyr_levels, VX_SCALE_PYRAMID_HALF, width, height, VX_DF_IMAGE_U8);
vx_delay pyr_delay = vxCreateDelay(context, (vx_reference)pyr_exemplar, 2);
vxReleasePyramid(&pyr_exemplar);
vx_array tracked_points = vxCreateArray(context, NVX_TYPE_POINT2F, 3000);
vx_delay pts_delay = vxCreateDelay(context, (vx_reference)tracked_points, 2);
// Build pyramid for current image
vx_node pyr_node = vxGaussianPyramidNode(graph,
src,
// Calculate optical flow between current and previous images
vx_node opt_flow_node = vxOpticalFlowPyrLKNode(graph,
tracked_points, VX_TERM_CRITERIA_BOTH, lk_epsilon, lk_num_iters, lk_use_init_est, lk_win_size);
// Update lost points
vx_node harris_track_node = nvxHarrisTrackNode(graph, src,
mask, tracked_points,
k, threshold, cell_size,
NULL);
vxReleaseNode(&pyr_node);
vxReleaseNode(&opt_flow_node);
vxReleaseNode(&harris_track_node);
vxAgeDelay(pyr_delay);
vxAgeDelay(pts_delay);

Functions

vx_node nvxHarrisTrackNode (vx_graph graph, vx_image input, vx_array output, vx_image mask, vx_array tracked_points, vx_float32 k, vx_float32 threshold, vx_uint32 cell_size, vx_scalar num_corners)
 [Graph] Detects and tracks Harris corners for the input image. More...
 
vx_status nvxuHarrisTrack (vx_context context, vx_image input, vx_array output, vx_image mask, vx_array tracked_points, vx_float32 k, vx_float32 threshold, vx_uint32 cell_size, vx_scalar num_corners)
 [Immediate] Detects and tracks Harris corners for the input image. More...
 

Function Documentation

vx_node nvxHarrisTrackNode ( vx_graph  graph,
vx_image  input,
vx_array  output,
vx_image  mask,
vx_array  tracked_points,
vx_float32  k,
vx_float32  threshold,
vx_uint32  cell_size,
vx_scalar  num_corners 
)

[Graph] Detects and tracks Harris corners for the input image.

Parameters
[in]graphSpecifies the graph.
[in]inputSpecifies the input image. Only VX_DF_IMAGE_U8 format is supported. The image size must be not greater than \( 2^{32} \) pixels.
[out]outputSpecifies the output list of corners. Only VX_TYPE_KEYPOINT, NVX_TYPE_KEYPOINTF and NVX_TYPE_POINT2F item types are supported. The array capacity must be explicitly provided, even for virtual arrays.
[in]maskSpecifies the optional mask. Only VX_DF_IMAGE_U8 format is supported. The mask should have the same size as input image.
[in]tracked_pointsSpecifies the optional tracked features list. If specified, it should have the same item type as output array.
[in]kSpecifies the Harris K parameter. The value of k has to be determined empirically, and in the literature values in the range \( [0.04, 0.15] \) have been reported as feasible.
[in]thresholdSpecifies the Harris threshold.
[in]cell_sizeSpecifies the size of cells for cell-based non-max suppression. The cell_size should be less than input image dimensions.
[out]num_cornersSpecifies the total number of detected corners in image (optional). It should be VX_TYPE_SIZE scalar.
Returns
A valid node reference or an error object (use vxGetStatus).
vx_status nvxuHarrisTrack ( vx_context  context,
vx_image  input,
vx_array  output,
vx_image  mask,
vx_array  tracked_points,
vx_float32  k,
vx_float32  threshold,
vx_uint32  cell_size,
vx_scalar  num_corners 
)

[Immediate] Detects and tracks Harris corners for the input image.

Parameters
[in]contextSpecifies the context.
[in]inputSpecifies the input image. Only VX_DF_IMAGE_U8 format is supported. The image size must be not greater than \( 2^{32} \) pixels.
[out]outputSpecifies the output list of corners. Only VX_TYPE_KEYPOINT, NVX_TYPE_KEYPOINTF and NVX_TYPE_POINT2F item types are supported. The array capacity must be explicitly provided, even for virtual arrays.
[in]maskSpecifies the optional mask. Only VX_DF_IMAGE_U8 format is supported. The mask should have the same size as input image.
[in]tracked_pointsSpecifies the optional tracked features list. If specified, it should have the same item type as output array.
[in]kSpecifies the Harris K parameter. The value of k has to be determined empirically, and in the literature values in the range \( [0.04, 0.15] \) have been reported as feasible.
[in]thresholdSpecifies the Harris threshold.
[in]cell_sizeSpecifies the size of cells for cell-based non-max suppression. The cell_size should be less than input image dimensions.
[out]num_cornersSpecifies the total number of detected corners in image (optional). It should be VX_TYPE_SIZE scalar.
Returns
A vx_status enumerator.
Return values
VX_SUCCESSNo errors.
VX_ERROR_INVALID_REFERENCESupplied parameters are not a valid references.
VX_ERROR_INVALID_PARAMETERSSupplied parameters are not valid.
VX_ERROR_INVALID_SCOPESupplied parameters are virtual objects, which can't be used in immediate mode.
VX_FAILUREInternal error in primitive implementation, check log for detailed information (Framework: Log).