Fisher Vector encoding (FV)

Author:

David Novotny

Andrea Vedaldi

fisher.h implements the Fisher Vectors (FV) image representation [perronnin06fisher]} [perronnin10improving]}. A FV is a statistics capturing the distribution of a set of vectors, usually a set of local image descriptors.

Getting started demonstrates how to use the C API to compute the FV representation of an image. For further details refer to:

• Fisher vector fundamentals - Fisher Vector definition.
• Fisher vector derivation - Deriving the Fisher Vectors as a Fisher Kernel.
• Fisher kernel - The Fisher Kernel in general.

Getting started

The Fisher Vector encoding of a set of features is obtained by using the function vl_fisher_encode. Note that the function requires a Gaussian Mixture Model (GMM) of the encoded feature distribution. In the following code, the result of the coding process is stored in the enc array and the improved fisher vector normalization is used.

float * means ;
float * covariances ;
float * priors ;
float * posteriors ;
float * enc;

// create a GMM object and cluster input data to get means, covariances
// and priors of the estimated mixture
gmm = vl_gmm_new (VL_TYPE_FLOAT) ;
vl_gmm_cluster (gmm, data, dimension, numData, numClusters);

// allocate space for the encoding
enc = vl_malloc(sizeof(float) * 2 * dimension * numClusters);

// run fisher encoding
vl_fisher_encode
    (enc, VL_F_TYPE,
     vl_gmm_get_means(gmm), dimension, numClusters,
     vl_gmm_get_covariances(gmm),
     vl_gmm_get_priors(gmm),
     dataToEncode, numDataToEncode,
     VL_FISHER_FLAG_IMPROVED
     ) ;

The performance of the standard Fisher Vector can be significantly improved [perronnin10improving]} by using appropriate Normalization and improved Fisher vectors normalizations. These are controlled by the flag parameter of vl_fisher_encode.