Image segmentation is the automatic process of identifying common areas within images. In order to satisfy two major tasks of segmenting images and categorize morphologically similar galaxies I deviate from the standard method of analysing square images of whole galaxies and instead employ an overlapping-patch based model. By analysing small overlapping patches which are typically much smaller than the sizes of resolved galaxies, the model is able to identify common sub areas and components within galaxies. The segmentation process consists of the following steps:
• Densely extract overlapping fixed size squares of pixel values (image patches) from unla- belled survey images.
• Perform feature extraction by calculating feature descriptors on the overlapping image patches.
• Apply unsupervised machine learning algorithms to create a dictionary of patches using the centres of clusters.
• Create a segmentation image of a survey image using the dictionary of patches.
4.2.1 Patch and Feature Extraction
A fixed sized, sliding window is moved over every pixel in a FITS file. A window size must be chosen by performing tests to identify an optimal size (see Chapter 5). The window contains the intensity values for one ‘patch’. A patch is extracted for every pixel. The pixel intensity values within a patch are then converted into features by applying a feature descriptor for the patch. In the previous Chapter I presented several options of feature descriptor such as RIFT, Spin and the power spectrum.
The patch extraction and feature extraction is calculated for each image file. The choice of image files depends on the selected wavelengths to be included in the analysis. The feature
descriptions for each pixel (one from each image file) are concatenated together to form a single vector for the pixel. The concatenation ensures that each vector encodes the information from multiple colours at a single position in the sky.
The vectors for all the pixels form a matrixn×mwhennis the number of patches, andmis the number of pixels in a patch. Each of themcolumns (each feature) in the matrix is normalised to have a mean of zero and unit standard deviation.
4.2.2 Application of unsupervised algorithms
Converting all the overlapping patches into features results in a very large data set proportional to the size of the input survey image files. The model uses unsupervised machine learning algorithms to identify the structure of this data, thereby creating a model of this structure. This occurs in two steps. The first is to use GNG to toplogically map the structure of the data with the extracted feature descriptors. A graph or vertices and edges is created which models the latent structure of the data. If the input data is too large, then pixels can be extracted from the images at random.
The graph created by GNG typically has many thousands of vertices. Each vertice can be viewed as a position in Cartesian space and models any data point (image patch) in close proximity, where proximity is measured using the Euclidean distance between the vertex and the patch. Another way of perceiving the graph produced by GNG is as a method to sort the number of patches into groups, where each vertex represents a bin of similar patches. In this way I create a dictionary of patches, where each vertex represents a visual item in the dictionary.
The graph typically contains thousands of vertices. In order to identify areas of interest the graph is cut using agglomerative hierarchical clustering.
Both the graph and groups identified by the agglomerative clustering can be thought of as a dictionary of patches. The graph represents a more detailed dictionary containing thousands of patch types, and the agglomerative groups a coarser grained dictionary containing hundreds or up to a thousand patch types.
Chapter 4.The Proposed Model 67
4.2.3 Segmentation of survey images
The first steps are to select a survey image and then follow the overlapping patch and feature extraction process: overlapping patches are extracted using the sliding window technique, fea- ture descriptors are calculated for each patch and the data normalised as shown in Section 4.2.1. However, instead of then applying the unsupervised machine learning algorithms, I use the dic- tionary of patch types instead. If the GNG/HC graph successfully maps the latent structure of the data then the graph can be used as a model by identifying which vertex in the graph is most similar to a feature descriptor. Each vertex represents a patch type. So by identifying the ver- tex most similar to a feature descriptor I identify the patch type used as an identifier to tag the corresponding pixel. This process is repeated by tagging all the pixels in an image with a patch type identifier.
Normally I would consider this identifier as a pixel’s classification. However, the word ‘classifi- cation’ in the machine learning community strongly implies a supervised process, which we do not use and so I use the term ‘identifier’ instead.
The result is a new image of the same size as the original with an identifier placed at each pixel position. A segmentation image is then produced by selecting a colour for each identifier and producing an image with all identifiers replaced with their corresponding colours.