Datasets

Three datasets are used for all experiments in this chapter, of which two were collated purposely for this thesis and future research. These to the best of our knowledge are the only existing video-based birds dataset, giving the opportunity to perform classification using motion features. The first which is known as "dataset #1", is a set of videos covering

4.1. DATASETS ANDMETHODS 81

seven classes, made up of seven different bird species in flight. This has a total of 170 videos, representing 23,932 image sequences. The second referred to as "dataset #2", is an extended set of videos covering thirteen classes made up of eleven bird species, one (Budgerigar (Melopsittacus undulatus)) with three colour forms, also in flight. This has a total of 957 videos, representing 161,907 image sequences. These datasets were collated because there is currently no bird species datasets which incorporate motion. The last, "Caltech-UCSD Birds-200-2011" dataset, has been extensively applied to fine-grained classification using computer vision. This is a still image data set of birds compiled by Wah et al. (2011b) from the internet, which contains 200 species, with a total of 11,788 images.

Figure 4.1: Segmented birds from the dataset #1 using the method in (Zivkovic and van der Heijden, 2006). From left to right: Common Wood Pigeon, Superb starling,

Nanday Parakeet, Budgerigar (wild-type) and Cockatiel.

Dataset #1 and #2 were recorded using a Casio Exilim ZR100, mounted on a tripod and recording at 240 frames per second. The videos were recorded over different days over two years and from various sites, which includes the National Parrot Zoo, and my backyard garden in Lincoln. Most of the species recorded consist of more than 15 individ- uals, apart from Superb Starlings, which had only three. Dataset #1 was the first collated and consist of seven bird species, which is made up of 170 short videos (representing 23,932 image frames) of approximately two to five seconds. Details of this has been provided in Table 4.1 and samples in Figure 4.1. This dataset has been used to perform experiments in Chapters 4 - 7. The majority class (class with the most samples) for these samples is the Common House Martin (Delichon urbica), which is made up of 30 videos and the minority (class with the least samples) is the Common Wood Pigeon (Columba palumbus), which is made up of 17. The dataset #2 consists of thirteen classes (eleven

bird species, one with three colour forms), and is an extension to dataset #1 with seven new classes but without the superb starling class. The superb starling class was omitted from dataset #2 as there were limited number of samples (videos) for this species (21 videos) as opposed to the minimum of 35 videos used for each class of this dataset. This consists of 957 short videos with 161,907 images. This has more classes and samples than dataset #1 and was collated to help understand the effects of increasing the classes in the datasets on classification rate and to make the dataset more challenging for future studies (more related species).

Again, this has been used to perform experiments in Chapters 4 - 7. Details and samples of this dataset have been provided in Table 4.2 and Figure 4.2 respectively. Most of the classes in this dataset have more than 37 videos. The minority class has 37 videos, whiles the majority has 147.

There are three species that have very closely related appearances in dataset #2, these are referred to as "species with fine-grained appearances". They are the Alexandrine Parakeet (Psittacula eupatria), Nanday Parakeet (Aratinga nenday) and Blue-crowned Parakeet (Thectocercus acuticaudatus). One particular species, the Budgerigar, has three different colour forms, which were used to form three different classes (this is to make the motion part of the system more challenging). Table 4.3 show all the thirteen classes with their distinguishing features. All the datasets were slightly imbalanced, which means that there is a class with fewer samples, known as the minority and another with the majority samples, known as the majority class. The imbalanced ratio for dataset #1 is approximately 1.76 and that of dataset #2 is 3.97. The imbalanced ratio is defined as the ratio of the number of samples in the majority class to that of the minority (García et al., 2012; Orriols-Puig and Bernadó-Mansilla, 2009). The two dataset are highly challenging and can be used in research involving motion, species with fine-grained appearances and imbalanced datasets. However, it’s more suitable for classification of species in flight from videos and for trajectories categorisation.

4.1. DATASETS ANDMETHODS 83

Table 4.1: Table showing the number of videos and images in the seven species dataset taken at three different sites, with majority from The National Parrot Sanctuary, Lin-

colnshire, UK

Species # of videos # of images

Common House Martin 30 4,378

Common Wood Pigeon 17 4,347

Superb Starling 21 1,920 Nanday Parakeet 27 4,155 Cockatiel 25 3,942 Common Starling 23 2,914 Budgerigar (wild-type) 27 2,276 Total 170 23,932

The last dataset, Caltech-UCSD Birds-200-2011 is an extended version of the CUB- 200 dataset, with roughly double the number of images per class. All images have been annotated with bounding boxes, part locations, and attribute labels. The images and an- notations have been filtered by multiple users of Mechanical Turk. There are 11,788 still images in this dataset covering 200 categories (species). Each category contains 60 or more images with 15 part locations per image, and a bounding box. Figure 4.3 shows im- ages of some species taken from the Caltech-UCSD Birds-200-2011 dataset. This dataset was chosen because it has been well validated and is one of the few bird species datasets which has been well used in computer vision for still image classification; and it has also been used by Marini et al. (2013), the work which is used in the evaluation of the proposed appearance features.

Table 4.2: Table showing the number of videos and images in thirteen classes (eleven bird species, one with three colour forms) dataset taken at three different sites, with

majority from The National Parrot Sanctuary, Lincolnshire, UK

Species # of videos # of images

Alexandrine Parakeet 79 12,801

Nanday Parakeet 60 10,025

Blue-crowned Parakeet 60 9,076

Common House Martin 139 25,517

Eastern Rosella 44 5,929

Budgerigar (yellow) 54 7,667

House Sparrow 78 10,191

Budgerigar (wild-type) 48 6,283

Common Wood Pigeon 37 4,301

Black-headed Gull 147 38,764

Cockatiel 59 9,398

Budgerigar (blue) 81 12,090

Common Starling 71 9,865

Total 957 161,907