Registration Method

The flow chart of the detection and localization algorithm is shown in Fig. 3.5. The whole process could be separated into two major steps: Single image processing and inter-image processing. In the step of single image processing, after being read in, every interested z-frame MR image in the set gets reconstructed, then the centers of seven fiducial points are calculated. Finally they are numbered and matched to the fiducial pattern. After all of the images are processed by this procedure, the

Figure 3.4: Z-frame fiducial modules mounted on the robots: A pneumatic robot developed by Fischer et al.(left) [43]; Needle insertion robot developed by Su and Shang(right).

inter-image processing takes place to fit the spatial lines of the seven fiducial tubes and finally calculates the Z-frame’s position and orientation as a four by four trans- formation matrix T_ZRAS.

Single Image Processing Inter‐Image Processing

Figure 3.5: Fiducial detection and localization flow chart

Image reconstruction is done right after the images are read in. To avoid the irrel- evant objects as much as possible, user is firstly asked to choose the region of interest (Fig. 3.6(left)). The selected region of image is then filtered with threshold, con-

verted to binary image and denoised. The threshold is automatically calculated from the selected region of interest. These steps give us a clearer image (Fig. 3.6(right)). Object size check is performed to make sure the anything left on the image are the actual seven fiducial points. Until this point, the first part of morphological method is done.

1

2

3

4

5

6

7

Figure 3.6: Original Z-frame MR image with the crop box shown (left); Binary image with fiducial point order number (right).

The second part of morphological method is getting the centers of seven fiducial points. There are several challenges when dealing with fiducial MR images. The images are not always as sharp as what is shown in Fig. 3.6. Irregular shape could happen when the off-the-shelf product (MRSpots, Beekley Corp., Bristol, CT) is used because there is an air bubble inside the fiducial tube. Fig. 3.7(upper left) shows a Beekley fiducial spot with air bubble in the upper left corner (red circle). Besides irregular shape, sometimes one of the fiducial points may look much dimmer than

able to get its complete shape, like Fig. 3.7(lower right). In some of prior works, the center of each fiducial point was found directly by finding its centroid [79]. This may be effective on high quality images with no bubbles or other artifacts, but is not robust. Therefore, in our work, unlike simply getting the centroid, we use a flipping method with ellipse model to reconstruct every fiducial point on the image, then the centers of reconstructed points were calculated to be used in localization later.

The flipping method could be described as following steps: 1. Get the binary image of a single fiducial point out; 2. Find the weighted centroid (xc0, yc0); 3. Use

the current centroid as the origin to draw x and y axis in parallel to original image x and y axis; 3. Flip the fiducial image with respect to the new x and y axis. Take logical ’OR’ operation between a pair of symmetrical pixels in both x and y directions. In result of this process, part of the absence caused by air bubble and some black pixels in the middle will be filled with the other half of the image which has the good shape. 4. Calculate the new centroid, if the difference between the old and new centroid is larger than a threshold, repeat step 3. If it does not change larger than the threshold, it means the irregular shape is compensated with good image and this last centroid is the final center point we want. The actual flipping calculation could be expressed as following equations:

x :          B(xc0 − i, yj) = B(xc0 − i, yj)||B(xc0 + i, yj) B(xc0 + i, yj) = B(xc0 − i, yj)||B(xc0 + i, yj) i = 1, 2, 3..., j = 1, 2, 3... (3.2) y :          B(xj, yc0 − i) = B(xj, yc0 − i)||B(xj, yc0+ i) B(xj, yc0 + i) = B(xj, yc0 − i)||B(xj, yc0 + i) i = 1, 2, 3..., j = 1, 2, 3... (3.3)

where B(x, y) denotes the binary value at the coordinate location of (x, y). It is either 0 or 1 since this is a binary image. The (xc0, yc0) is the current weighted

centroid coordinates. i is the distance of the pixel currently being operated to the flipping axis. It starts from the closest pixel whose i = 1. || denotes the logical OR operator. After the current flipping is done to all the pixels for both x and y directions, the new centroid is calculated again as (xc1, yc1) to replace the (xc0, yc0) in

these equations.

As a result, in Fig. 3.7, the red stars represent the boundaries of the fiducial point after the flipping method. Blue and red circle represent the center of original image and the image after the flipping method. From the upper right figure in Fig. we can see that the updated boundaries successfully cover the air bubble. If we manually fit an ellipse to the image, it is obvious that the updated center (red circle) is closer to

the ideal center than the original center (blue).

Example 2: Fussy image

Example 1:Bubble in the tube

Figure 3.7: In all figures, red circles denote the center gotten from flipping method, and the blue circles denote the center gotten from traditional weighted centroid method. A fiducial point with an air bubble on its upper left corner(red dashed circle)(upper left), the same fiducial point with boundaries got from the flipping method, manually fitted ellipse and the center of ellipse(upper right). Another fidu- cial point with low contrast to the background(lower left) and the boundaries got from flipping method and manully fitted ellipse with center(lower right). From both upper right and lower right figure, it is obvious that the center got after flipping method(red circle) is closer to the ideal center than the center got from traditional weighted centroid method(blue circle).

After all seven fiducial points are done with the flipping method, by using the geometric character of Z-frame, they are assigned with number(Fig. 3.6(right)) and

put in the correct order for the spatial fitting later. This step is not as trivial as it looks like. There are several steps to number all the points in the correct order.

Before starting to talk about the procedures, it should be noticed that principal component analysis (PCA) is used instead of the ordinary least square (OLS) fitting. As shown in Fig. 3.8, the OLS minimizes the error between the dependent (y) and the model. PCA minimizes the error orthogonal (perpendicular) to the model line. In most of the cases, the dependent y is a function of x so it is natural to just consider the error between y and the model. But in our case, the y is not dependent to x. They should be equally treated and the error should be both considered. Thus, theoretically, it makes PCA a better approach than OLS.

a

b

Figure 3.8: Ordinary least square (OLS) fitting method (a); principal component analysis (PCA) fitting method (b).

The actual steps for numbering all the points in the correct order are as follows: 1) Get the distances between every two arbitrary points by the following equation:

D(i, j) = q

(xi− xj)2+ (yi− yj)2, i = 1, 2...7, j = 1, 2...7 (3.4)

This is saved for later use.

2) Fit any three arbitrary points to a straight line with the normalized vector of 3.5 by PCA method. Getting the coefficients and scores (residual errors) for each fitted line.

v = [vx, vy] (3.5)

As shown in Fig.3.6(right), there are only three perfectly fitted lines with residual error small enough. Following show the three vectors for one example image:

                   v1 = [−0.0035, 1] v2 = [1, −0.0046] v3 = [0.0037, 1] (3.6)

3) Next is to distinguish which line is which. This is done by comparing the perpendicularity between each two of the three lines by the following equation:

vi· vj = vixvjx+ viyvjy, (i 6= j) (3.7)

fectly perpendicular to each other. An estimated small threshold is used to determine the perpendicularity. After this step, we could find out the line which is perpendic- ular to both of two other lines. Combining the position of the other two lines in x direction, all three lines could be distinguished.

4) Finally, by using the geometric properties of the fiducial points forming each line, all the points could be numbered as shown in 3.6(right). The distances in 3.4 are also used for numbering and double check of the result in order to make the whole method more rigid.

After getting the coordinate information for the points on every image we need, PCA fitting methods were used to fit the seven fiducial lines of the Z-frame in 3D space. Finally, the 6-DOF position and orientation information of the fiducial frame is acquired from these seven fitted lines as a four by four transformation matrix TRAS

Z .

The algorithm was implemented in Matlab. Fig. 3.9 shows a Matlab 3D plot of centers of fiducial points on each image and seven fitted fiducial lines in RAS patient coordinate.