Experimental Setup

Data Set: We randomly selected 46 de-identified video files captured during routine screening colonoscopy at the image resolution of 720x480 pixels at 29.97 frames per second (fps). Each file contains only one colonoscopy procedure in its entirety. Twenty-eight of these videos were captured using FUJINON endoscopes, and the remaining eighteen video files were captured using OLYMPUS endoscopes. The colons seen in these videos had little stool inside them. We extracted from these video files 69 smaller clips: 50 clips with polyps and 19 clips without any polyps. The 50 clips represent 50 different polyps (one polyp per clip). Each polyp clip has consecutive frames showing the same polyp appearing at various viewing angles, light conditions, and scales. Pit patterns [56] of these polyps are not clearly seen since magnifying endoscopes were not used.

Our non-polyp clips contain images with most types of objects often seen in colonscopy, including blood vessels, colon wall, colon folds, stool, retroflexion, appendiceal orifices and

diverticula [1]. For each clip, we extracted the images in JPEG format with a resolution of 720x480

pixels at the frame rate of 1 fps. We removed blurry images using blurry frame removal software [70], leaving the remaining clear (in-focus) images for the experiments. The blurry frame removal algorithm in [70] considers a frame as blurry if it does not have sufficient number of connected edge pixels detected using Canny edge detector. Totally, we obtained 1025 images each with a polyp in it and 488 images without any polyps on them. Two domain experts marked and agreed on the ground truth of all polyp regions in the images. Fig. 10 shows example ground truth polyp regions.

Table 6.2: Description of Data Sets Used for Polyp Image Detection

Category # Polyp clips

(# images, # edges)

# Non-polyp clips

(# images, # edges) Total

Training 8 (160, 844) 6 (87, 1305) 14 (247, 2149)

Testing 42 (865, 4983) 13(401, 1813) 55 (1266, 6796)

Total 50 (1025, 5827) 19 (488, 3118) 69 (1513, 8945)

Table 6.3: Number of Polyp Images of Each Video Clip in the Training Set

Video clips captured using FUFJINON endoscope

Clip ID 1 2 3

# Polyp images 19 10 36

Video clips captured using OLYMPUS endoscope

Clip ID 4 5 6 7 8

# Polyp images 20 43 8 18 6

Table 6.4: Number of Polyp Images of Each Video Clip in the Testing Set

Video clips captured using FUFJINON endoscope

Clip ID 1 2 3 4 5 6 7 8 9 10

# Polyp images 9 21 4 17 8 17 7 26 11 8

Clip ID 11 12 13 14 15 16 17 18 19 20

# Polyp images 18 13 43 58 21 2 19 16 6 8

Video clips captured using OLYMPUS endoscope

Clip ID 21 22 23 24 25 26 27 28 29 30 # Polyp images 19 23 30 45 8 7 53 17 8 10 Clip ID 31 32 33 34 35 36 37 38 39 40 # Polyp images 37 20 37 11 50 41 11 26 28 11 Clip ID 41 42 # Polyp images 12 30

Table 6.5: Polyp Categories in the Testing Set

Category On colon

wall

On colon

folds Near flat With stool All

# Images 437 428 130 196 865

We split the polyp clips into non-overlapping training and testing sets. Table 6.2 describes the data sets including the number of clips, images and extracted edges after the image preprocessing step in each category. Table 6.3 and Table 6.4 describe the numbers of polyp images in each video clip in the training data and the testing data, respectively. We included eight polyp clips in the training set, which covered all categories of polyp appearance in Table 6.5 (discussed in more details in a later paragraph).

Parameter value settings: All parameter values were determined based on experiments with the training set. The parameter values may vary for different image resolutions in other data sets.

Image pre-processing and ECSP calculation: We extracted edges using Canny edge detector with the low threshold, high threshold, and standard deviation of Gaussian filter of 0.05, 0.1, and 3.0, respectively. Next, we removed small branches on these edges. We discarded an edge if 1) it had less than 20 edge pixels (smaller than the number of pixels in the window [_) or 2) it was an approximately linear edge. An edge is considered linear if the least squares fitting error of its pixel locations fitted to a linear line is less than ε (ε = 8 pixels in our case) using a linear regression as in Chapter 4. We cut an S-like shape edge that typically occurred when a polyp was connected with a colon fold into several C-like shape edges by tracking edge pixels and examining the change of the edge tangent directions. Because our data set has clear images obtained based on the quality of their Canny edges, we successfully extracted real polyp edges for the vast majority of polyp images (1018 of 1025) with these parameters. For ECSP calculation, the sliding window [_ of sizes between 10x10 and 25x25 pixels gave good performance. The valid range of  was from 0.5 to 1.0. In subsequent experiments, we set the size of [_ to 15x15 pixels and  to 0.5, and limit /E to a maximum value of 80 pixels. The one-dimensional window size centering at r=0 for finding the minimum intensity point r0 in the window was 1x21.

Feature normalization: The values of most of our features, Vtexture1, Vtexture2, Vshape, Vsmoothness, h2,

h3, h4 are in the range of [0, 1]. The feature h1 - h2 is in the range of [-1, 1]. As the values of features

h2/h1 and h4/h3 are in large scale ranges, we transformed the data of these two features using log

log (h4/h3) by dividing it by its corresponding C (d)

, where C(d) is the absolute difference of maximum and minimum values for the feature d in the training data.

Performance metrics: We consider the minimum bounding rectangle of each edge obtained from Canny edge detector as one region. Let DET_RLPLP_IMG, RLPLP_IMG, FL_REGION, and

IMG be the number of correctly detected polyp images, the number of real polyp images, the number

of falsely detected regions (regions not overlapping with any real polyps), and the number of tested images, respectively. A polyp image is considered correctly detected if it has at least one correctly detected polyp region. A detected region is counted as a true positive if it overlaps with a ground truth polyp region. The number of wrongly detected polyp regions was counted as falsely detected regions (FLR). Regions that are falsely detected in true polyp images are also counted in FLR. We used the following performance metrics that were also used in [30].

}
R q_5`5 1`R }  <Ì¿f_ŒŒ_ëIì_{ŒŒ_ëIì}

¥
Q3R  1_R Rí5_ qR 5Q1íR -î  <ï_¿ìëðŽ _ëIì

To provide visual feedback of a detected polyp region, a high TPR is desirable. On the other hand, a low FLR is desirable since many false positives can distract the endoscopist. We used these metrics in the free-response receiver operating characteristic curves (FROC). Our TPR is not based on regions since our focus is not on detecting an accurate polyp contour.

Classifier selection: We investigated the performance of several classifiers: two-class Support Vector Machine (SVM) with radial basis function (RBF) kernel using Torch SVM library [81] and Generalized Linear Models (GLM) [82] with Binomial, Gaussian and Poisson distributions as kernels. We used feature vectors obtained from all images in the training and testing sets for a 10-fold cross

validation to obtain performance of these classifiers. We varied the standard deviation of Gaussian

kernel for the SVM classifier, and the threshold values of probability for GLM classifiers to obtain FROCs. There are two more parameters for the SVM classifier [82], but they had little effect on FROCs. These different classifiers showed similar FROCs (plots omitted). SVM with RBF kernel and GLM with Gaussian kernel showed the best performance with the largest area under the FROC.

We used a two-class SVM classifier with RBF kernel as the classifier for our technique for the rest of the experiments. The two classes were polyp images and non-polyp images. In these experiments, we varied the standard deviation of the Gaussian kernel for the SVM classifier to train the classifier using all images in the training set and applied the trained classifier on the entire testing set to get TPR and FLR for each point on FROC.