CHAPTER 3: Multiple Instance Learning for Heterogeneous Images with an SVM
3.1 Related Work
}
maximum voting noisy-AND generalized mean rank-based Lp norm median quantile}
}
}
Bag paradigm:pool features and make prediction
Instance paradigm:
pool instance predictions
determined by MI assumption
Standard assumption:
positive if and only if at least one instance is positive
Majority vote:
instances vote for bag class Hand-crafted features: cell-based morphology color histograms texture SIFT LBP Capture intra-class heterogeneity with representation learning: instance prototypes dictionary learning CNN Class 1 Class 2
{
{
feature learning applied to one class only feature learning applied to both classes, but still usingstandard assumption
feature learning applied to both classes with majority vote assumption
(proposed method A)
Quantile aggregation:
classifier learns how to combine instance predictions
feature learning applied to both classes with quantile function to
learn aggregation
(proposed method B)
Figure 3.2: Instance paradigm methods first classify individual instances and then pool the predictions, while bag paradigm methods pool the instance features and make a single pre- diction. Instance paradigm methods also differ in whether they use the standard assumption positive if and only if at least one instance is positive), the majority vote, or learn how instance predictions should be aggregated, such as with the proposed method of quantile aggregation. If classes are homogeneous, a simple mean of hand-crafted cell features or patch descriptors is sufficient to characterize each image. When classes are heterogeneous, learning a representa- tion becomes important. Proposed method A uses the majority vote instead of the standard assumption, along with feature learning and a iterative method to learn the latent instance labels. Proposed method B uses a quantile function of instance predictions to predict the bag class, thus capturing a wider range of possible MI assumptions.
Reduction to Single Instance Learning. Many image classification solutions for histology and other data types turn the problem into a fully supervised one by representing each bag as a single feature vector [Chen and Wang, 2004; Chen et al., 2006] or applying a specialized kernel [Zhou et al., 2009]. This class of methods can only make predictions at the bag level, not the instance level, so is not suitable for characterizing tissue heterogeneity or interpreting results. The methods that I present make use of an instance classifier.
Instance-level Methods. Rather than making decisions at the bag level, other MI ap- proaches design classifiers to operate on individual instances and then aggregate their output scores or decisions. Andrews et al. developed mi-SVM, in which they apply an SVM to MI learning by iteratively learning the latent instance labels while enforcing the standard MI as- sumption [Andrews et al., 2002]. This class of score- or decision-level fusion methods is able to make use of a larger number of samples drawn from the set of instances when training the classifier. However, mi-SVM still follows the standard assumption: treating classes asymmet- rically. I use the power of mi-SVM in learning latent instance labels but adapt it for a wider range of possible MI assumptions (Section 3.4).
MI Assumptions. For the standard MI assumption, all instances in a negative bag are negative and at least one instance in each positive bag is positive. This asymmetric definition treats positive bags differently than negative. MI techniques have been applied to histology for distinguishing images containing cancer from those that are cancer-free, using the standard assumption [Kandemir and Hamprecht, 2014; Xu et al., 2014a,b]. This asymmetric definition is not appropriate for classifying tumors by subtype. I propose a more general MI assumption in which a given percentage of instances must be positive and a method to learn the latent instance labels. Further, I propose the quantile function as a method for learning to aggregate instance predictions and for use when a suitable MI assumption for a particular task is unknown.
Bag- vs. Instance-level Predictions. Cheplygina et al. compare the stability of MI meth- ods for three biomedical applications and find that the best bag-level classifier is not always the best instance-level classifier [Cheplygina et al., 2015]. mi-SVM is the most stable of the methods tested. Vanwinckelen et al. also compare instance-level and bag-level classification,
showing that the correlation varies widely by data set domain, learner assumptions, and perfor- mance measures [Vanwinckelen et al., 2016]. They also compared MI methods with their single instance (SI) counterpart, finding that often the SI method outperforms the MI algorithm. In some cases this may be due to a high witness rate - the proportion of positive instances in positive bags [Carbonneau et al., 2017]. To address this, Wang et al. optimize both the bag- level and instance-level loss by including them both in the cost function [Wang et al., 2015b]. My work addresses the challenge of producing an accurate instance- and bag-level classifier by better bridging the gap between them with a more powerful pooling function (Section 3.5).
Image Heterogeneity. Intra-class heterogeneity can be accounted for by forming multiple prototypes for each class; however, initial research in this direction focused on the standard assumption and only applied heterogeneity to the pathological case [Xu et al., 2014b; Li et al., 2015; Wang et al., 2013; Varol et al., 2015]. When each class represents a different subtype of the disease, heterogeneity should be accounted for in all classes. While dictionary learning applied to all classes could address this deficiency, existing methods still remain focused on the standard assumption [Shrivastava et al., 2015; Song et al., 2013; Jiao and Zare, 2015].
CNNs produce another powerful feature set for capturing heterogeneity. Hou et al. use a CNN with an iterative MI method for predicting cancer subtypes from whole slide histology images [Hou et al., 2016]. The standard MI assumption does not apply, so they must learn which image patches belong to the labeled class of the slide. Different methods of aggregating patch predictions were tested: maximum, voting, and a histogram of predictions with a logistic regression classifier. Their iterative MI method uses EM to maximize the data likelihood and does not take into consideration the MI assumption chosen; the iterative method that I propose learns the latent instance labels given the MI assumption as a constraint. My quantile aggregation method is also more suitable than a histogram of predictions because it easily accommodates a non-uniform distribution of predictions without needing to specify bin sizes. I experiment with pre-trained CNN features in this chapter and end-to-end training of a CNN in the following chapter.