Unconstrained Ear Recognition Challenge

Welcome to the Unconstrained Ear Recognition Challenge!

The 2nd Unconstrained Ear Recognition Challenge (UERC) is still in the planning phase. Stay tuned for details

The 1st Unconstrained Ear Recognition Challenge (UERC) that was organized in the scope of the IEEE/IAPR International Joint Conference on Biometrics 2017. The goal of the challenge was to advance the state-of- technology in the field of automatic ear recognition, to provide participants with a challenging research problem and introduce a benchmark dataset and protocol for assessing the latest techniques, models, and algorithms related to automatic ear recognition.

The results of UERC were published in an IJCB conference paper co-authored jointly by all challenge participants, available here and here.


Despite the numerous application possibilities in security, surveillance applications, forensics, criminal investigations or border control, the existing research in ear recognition has seldom gone beyond laboratory settings. This can mostly be attributed to the enormous appearance variability of ear images when captured in unconstrained settings. However, due to recent advances in computer vision, machine learning and artificial intelligence (e.g. with deep learning), many recognition problems are now solvable (to at least some extent) in unconstrained settings and many modalities that were too complex to use in real-life situations are now becoming a viable source of data for person recognition.

The Unconstrained Ear Recognition Challenge (UERC) built on the advances outlined above and address the problem of ear recognition “in the wild”. While many competitions and challenges were organized in the scope of past biometrics-oriented conferences (ICB, BTAS, FG, etc.) for various biometric modalities and numerous problems, ear recognition has not yet been studied within group evaluations making the UERC a unique opportunity with high potential impact.

The Dataset, Protocol and Performance Metrics

The challenge was held on an extended version of the Annotated Web Ears (AWE)dataset, containing a total of 9,500 ear images. The images were collected with a semi-automatic procedure that involved web-crawlers and a subsequent manual inspection. Because the AWE images were not gathered in controlled laboratory-like conditions, they better represent the variability in ear appearance than existing datasets of ear images. However, the problem of automatic ear recognition is also significantly harder. A few example images from the extended AWE dataset are shown below.

A more in depth description of the images, acquisition procedure, dataset characteristics and other information on the AWE dataset is available in the Neurocomputing paper.

UERC used three image datasets:

  • part A: the main datasets of 3,300 ear images belonging to 330 distinct identities (with 10 images per subject) that was used for the recognition experiments (training and testing),
  • part B: a set of 804 ear images of 16 subjects (with a variable number of images per subject) that was used for the recognition experiments (training),
  • part C: an additional set of 7,700 ear images of around 3,360 identities that was used to test the scalability of the submitted algorithms.

The 3,300 images of the main dataset contain various annotations, such as the level of occlusion, rotation (yaw, roll and pitch angles), presence of accessories, gender and side. This information was also made available during training and can be exploited to build specialized recognition techniques.

The 3,300 images of the main part of the dataset was split into a training set of 1,500 images (belonging to 150 subjects) and a test set of 1,800 images (belonging to 180 subjects). The identities in the training and test set were disjoint. The purpose of the training set is to train recognition models and set any open hyper-parameters, while the test set is reserved for the final evaluation. The test set MUST NOT have been used to learn or fine-tune any parameters of the recognition model. The organizers reserved the right to exclude a team from the competition (and consequently the jointly authored IJCB conference paper) if the final result analysis suggested that the test images were also used for training.

The train and test sets were split:

  • Train (2,304 images of 166 subjects): 1,500 images (150 subjects) from part A and all images from part C (804 - 16 subjects).
  • Test (9,500 images of 3,540 subjects): 1,800 images (180 subjects) from part A and all impostor images (7,700 - 3,360 subjects).

UERC tested the recognition performance of all submitted algorithms through identification experiments. The participants had to generate a similarity score matrix with comparisons of each image in the probe.txt file to each image in the gallery.txt file and returned the resulting similarity matrix to the organizers for scoring. Thus, each participant was required to generate a 7442x9500 matrix for each submitted system.

The number of approaches that each participant was allowed to submit was not limited. However, only approaches with a least a short description (written by the participants) were included in the IJCB summary paper of UERC 2017. Submissions were possible via a simple web interface.

The submitted similarity matrices were scored by the organizers. Rank-1 recognition rates, complete CMC curves and the Area-under- the CMC (AUC) curve were computed and reported for each submitted algorithm. The AUC was used to rank the participating algorithms. Based on the annotations available with the extended AWE dataset, we also computed results for sub-sets of the data, e.g. focusing only on certain ranges of head rotations, presence/absence of accessories, etc.

Starter Kit

We provided all participants with a Matlab starter kit that generated a matrix of comparisons for each sample in the dataset and computed all relevant performance metrics. The starter kit helped participants to quickly start with the research work and generate score matrices compliant with our scoring procedures. For the baseline approaches we made a number of descriptor-based approaches (using SIFTs,POEMs, LBPs, etc.) available to the participants as well as a deep learning approach.

Provisional Timeline

  • 2017/02/01: UERC 2017 kick-off: starter kit and datasets made available
  • 2017/04/17: Initial results and a two-sentence description of the approach (for better planning of the paper structure)
  • 2017/04/24: Final results submission
  • 2017/05/01: Final submissions of the results and algorithm descriptions
  • 2017/05/20: Submission of UERC summary paper to IJCB 2017

The Call for Participants PDF is available here.

The IET Call for Papers PDF is available here.

The final PDF of the paper available on IEEE Xplore.

The final PDF of the paper available on ArXiv.

If you have any questions, suggestions or would like to participate in the competition, feel free to contact