Covariant feature detectors

Author:

Karel Lenc

Andrea Vedaldi

Michal Perdoch

covdet.h implements a number of covariant feature detectors, based on three cornerness measures (determinant of the Hessian, trace of the Hessian (aka Difference of Gaussians, and Harris). It supprots affine adaptation, orientation estimation, as well as Laplacian scale detection.

• Covariant detectors fundamentals
• Principles of covariant detection
• Differential and integral image operations
• Cornerness measures

Getting started

The VlCovDet object implements a number of covariant feature detectors: Difference of Gaussian, Harris, determinant of Hessian. Variant of the basic detectors support scale selection by maximizing the Laplacian measure as well as affine normalization.

// create a detector object
VlCovDet * covdet = vl_covdet_new(method) ;

// set various parameters (optional)
vl_covdet_set_first_octave(covdet, -1) ; // start by doubling the image resolution
vl_covdet_set_octave_resolution(covdet, octaveResolution) ;
vl_covdet_set_peak_threshold(covdet, peakThreshold) ;
vl_covdet_set_edge_threshold(covdet, edgeThreshold) ;

// process the image and run the detector
vl_covdet_put_image(covdet, image, numRows, numCols) ;
vl_covdet_detect(covdet) ;

// drop features on the margin (optional)
vl_covdet_drop_features_outside (covdet, boundaryMargin) ;

// compute the affine shape of the features (optional)
vl_covdet_extract_affine_shape(covdet) ;

// compute the orientation of the features (optional)
vl_covdet_extract_orientations(covdet) ;

// get feature frames back
vl_size numFeatures = vl_covdet_get_num_features(covdet) ;
VlCovDetFeature const * feature = vl_covdet_get_features(covdet) ;

// get normalized feature appearance patches (optional)
vl_size w = 2*patchResolution + 1 ;
for (i = 0 ; i < numFeatures ; ++i) {
  float * patch = malloc(w*w*sizeof(*desc)) ;
  vl_covdet_extract_patch_for_frame(covdet,
                                    patch,
                                    patchResolution,
                                    patchRelativeExtent,
                                    patchRelativeSmoothing,
                                    feature[i].frame) ;
  // do something with patch
}

This example code:

• Calls vl_covdet_new constructs a new detector object. covdet.h supports a variety of different detectors (see VlCovDetMethod).
• Optionally calls various functions to set the detector parameters if needed (e.g. vl_covdet_set_peak_threshold).
• Calls vl_covdet_put_image to start processing a new image. It causes the detector to compute the scale space representation of the image, but does not compute the features yet.
• Calls vl_covdet_detect runs the detector. At this point features are ready to be extracted. However, one or all of the following steps may be executed in order to process the features further.
• Optionally calls vl_covdet_drop_features_outside to drop features outside the image boundary.
• Optionally calls vl_covdet_extract_affine_shape to compute the affine shape of features using affine adaptation.
• Optionally calls vl_covdet_extract_orientations to compute the dominant orientation of features looking for the dominant gradient orientation in patches.
• Optionally calls vl_covdet_extract_patch_for_frame to extract a normalized feature patch, for example to compute an invariant feature descriptor.