Integrating Multiple Local Binary Descriptors via Learnt Weights

Yongqiang Gao, Weilin Huang,Yu Qiao

Shenzhen Key lab of Computer Vision and Pattern Recognition

Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, China

Shenzhen Colledge of Advanced Technology, University of Chinese Academy of Sciences

Chinese University of Hong Kong, Hong Kong SAR


Binary descriptors have received extensive research interests due to their low memory storage and computational efficiency. However, the discriminative ability of the binary descriptors is often limited by comparing to general float-point ones. In this paper, we present a learning framework to effectively integrate multiple binary descriptors, which is referred as learning-based multiple binary descriptors (LMBD). We observe that previous successful binary descriptors like Receptive Fields Descriptor (RFD) which includes rectangular pooling area (RFDR) and Gaussian pooling area (RFDR)), BinBoost, and Boosted Gradient Maps (BGM) , are highly complementary to each other. We show that the proposed LMBD can improve the discriminative ability of individual binary descriptors significantly. The learning process of fusing multiple groups of the binary descriptors was formulated as a pair-wise ranking problem, which can be solved effectively in a rankSVM framework. Extensive experiments were conducted to evaluate the efficiency of LMBD. The proposed LMBD obtains the error rate of 12.44% on the challenging local patch datasets[10], which is about 2% lower than the state-of-the-art results[9] (obtained by a learning based float-point descriptor). Furthermore, the proposed binary descriptor is also outperforms other binary descriptors on image matching task.


Coming soon...


Download: coming soon, code(matlab)

Experimental Results
  • Evaluation Results.
  • Comparison experiments: combination by same weights (SW) (e.g., w m = 1) and learnt weights (LW) which are bolded along the four group candidates with different dimensions for each group. Testing datasets are 100k and false positive rate at 95% recall are reported. Note that All the models are trained on "Liberty" dataset, and "All" denotes the reported dimension in each papers, i.e. 256(b) for BinBoost [6] and BGM[5] 293(b) for RFD R and 406(b) for RFD G [7].

  • Comparison Results with State-of-the-art.
  • Brown et al.'s Patch Dataset

    Comparison of combined-binary descriptor to the state-of-the-art descriptors, including binary descriptors and float-value descriptors. Testing datasets are 100k and false positive rate at 95% recall are reported across all the splits of training and testing configurations.

    Train Liberty
    Test Notre Dame Yosemite
    Binary Descriptors
    BRIEF[1] 48.64 (512b) 52.69 (512b)
    BRISK[2] 74.88 (1024b) 73.21 (1024b)
    D-BRIEF[3] 43.10 (32b) 47.29 (32b)
    ITQ-SFIT[4] 31.07 (64b) 34.43 (64b)
    BGM[5] 15.99 (256b) 21.11 (256b)
    BinBoost[6] 16.90 (64b) 22.88 (64b)
    RFDG[7] 12.49 (406b) 17.62 (406b)
    RFDR[7] 13.23 (293b) 16.99 (293b)
    Combined-binary 9.52 12.44
    Float Descriptors
    SIFT[8] 26.44 (1024b) 30.84 (1024b)
    L-BGM[5] 14.15 (512b) 19.63 (512b)
    Simonyan et al.[9] 9.07 14.32
  • Application: Image Matching.
  • Oxford Dataset


    BRIEF[1]: M. Calonder, V. Lepetit, M. Ozuysal, T. Trzcinski, C. Strecha and P. Fua, "BRIEF: Computing a Local Binary Descriptor Very Fast", IEEE TPAMI, 2012

    BRISK[2]: S. Leutenegger, M. Chli and R. Siegwart, "BRISK: Binary Robust invariant scalable keypoints", ICCV, 2011

    D-BRIEF[3]: T. Trzcinski and V. Lepetit, "Efficient Discriminative Projections for Compact Binary Descriptors", ECCV, 2012

    ITK-SFIT[4]: Yunchao Gong, S. Lazebnik, A. Gordo, and F. Perronnin, "Iterative Quantization: A Procrustean Approach to Learning Binary Codes for Large-scale Image Retrieval", IEEE TPAMI, 2012

    BGM[5]: T. Trzcinski and M. Christoudias and P. Fua and V. Lepetit, "Learning image descriptors with the boosting-trick", NIPS, 2012

    BinBoost[6]: STrzcinski T. , Christoudias M. , Lepetit V. and Pascal Fua, "Boosting Binary Keypoint Descriptors", CVPR, 2013

    RFD[7]: Fan, B. and Kong, Q.Q. and Trzcinski, T. and Wang, Z. H. and Pan, C.H. and Fua, P., "Receptive Fields selectioni for binary Feature Description", TIP, 2014

    SFIT[8]: Lowe, David G., "Distinctive Image Features from Scale-Invariant Keypoints", IJCV, 2004

    Simonyan et al.[9]: H. G. Brown M. and W.S., "Discriminative learning of local image descriptors", IEEE TPAMI, 2010

    Brown et al.[10]: Simonyan, K. and Vedaldi, A. and Zisserman, A., "Learning Local Feature Descriptors Using Convex Optimisation", IEEE TPAMI, 2013


    Yongqiang Gao,