Control Software and Bead Tracking

The purpose of the software is to control the electrical current in each coil. Hence, the electrical current from the DC power supply is controlled through serial communication by an Arduino microcontroller (ARDUINO DUE R3). Serial commands are sent from the software to the power supply to alter the magnitude and polarity of the current output. The produced heat is checked in real-time by evaluating the changes in the resistance of the coils. When the resistance is beyond the calibrated threshold, which indicates a high temperature, the coils are automatically disconnected from the power supply.

In order to track the position of the bead in 3D, a calibration microscope system was designed to visually observe the bead in the capillary and to track the position of the bead during bead movement (Figure 3.6). An objective lens (4× magnification) was mounted on a 3-axis micrometer manual translation stage (Thorlabs PT3/M, 25 mm motion range) on a vibrational isolation table. The vertical direction of the stage was actuated by a linear motor (The LTA-HS 50 mm High Speed Motorized Actuator). Two tube lenses were used, each connected to a camera (AVT Guppy F-503C) with a resolution of 2588 × 1940 pixels and a maximum frame rate of 13 fps. Each pixel corresponded to a distance of 0.5 µm, which was much smaller than the bead diameter. The light beams from the objective lens were added to the second camera by a beam splitter and a mirror. The capillary was held horizontally and moved under the microscope by another 3-axis micrometer manual translation stage. We managed to observe the same field of view with the same resolution from the two cameras by an off-focus distance of around 100 µm.

Fig. 3.6 Diagram of the layout of the calibration setup with optical imaging systems mounted. (A) Perspective view. (B) Side view. Images by courtesy of Radu Tanasa.

The imaging volume is defined as the interest space for embryo manipulation. The size of the volume is 300 µm × 300 µm × 300 µm around the centre of the poles. A Cartesian coordinate system is imposed on the imaging volume with the origin at the centre of the poles. The y-axis is along the line of the capillary and the x-axis is perpendicular to it. The z-axis is defined as positive vertically upwards. The coordinate system follows the right handed coordinate convention.

Bead detection was achieved using open source computer vision libraries (OpenCV). Images were cropped to an area of twice the bead diameter (hereinafter referred to as "cropped image"). The Sobel edge detection algorithm [102] was applied to the cropped images (hereinafter referred to as "filtered image"). The coordinates of the bead in the x-y plane are calculated by counting the pixels from the centre of the detected bead to the origin of the imaging volume (Figure 3.7 (A)). The bead position on the z-axis is determined by the focus parameters [95]. A focus parameter characterises how in focus the bead images is. It is defined as the ratio of the standard deviation of pixel values and their mean. If the initial image of the bead is sharp, the edges of the bead in the filtered image becomes thinner and brighter. In this case, the focus parameter is large because of a high standard deviation and a low mean. By taking a stack around the bead, a Lorentzian distribution of the focus parameter along the detection axis was revealed. We have obtained these distributions for both cameras and stored them in the control software (Figure 3.8 (A)). During bead tracking, focus parameters were calculated for both images and these values were stored in the profiles.

Cropped Filtered (A) Objective Lens Observed Bead Position _Actual Bead Position Silicone Oil Air z (B)

Fig. 3.7 Bead observation and bead tracking in 3D. (A) An example image of the bead sample with indication of the coordinate system in the x-y plane. Scale bar: 200 µm. (B) The observed bead position in the live image and the actual position of the bead, as a result of different refractions in the air and in the silicone oil.

0.2 0.1 0.0 0.1 0.2 0.3

z(mm)

Focus Parameter (a.u.)

Top Plane Bottom Plane (A) 0.04 0.03 0.02 0.01 0.00 0.01 0.02 0.03 0.04

z(mm)

f

/

f

Fig. 3.8 Bead tracking in the direction of the z-axis. (A) The calibrated data of the focus parameters of the two tube lenses. (B) The bead position on the z-axis is detected by calculating the ratio of the two focusing parameters and fitting to the calibrated data.

Initially the cameras are moved manually in the z-axis until the bead is between the two

planes and is visible in both camera images. The two focus parameters, f1for the cropped

image and f2for the filtered image, are calculated. The bead position in the z-axis can be

searched by fitting the ratio of f1/ f2to the calibrated data (Figure 3.8 (B)). When the bead

moves away from the central line between the two image planes by more than 20 µm, the mechanical actuator will reposition the microscope to relocate the bead to the centre of the two planes.

However, the refraction in silicone oil is different from that in air. The refractive index of silicone oil is 1.40. Hence, the movement of the objective lens by the actuator, with the image plane movement, is corrected. For instance, by moving the objective lens (image planes) by 100 µm, the actual movement of a bead in the silicone oil will be 140 µm (Figure 3.7 (B)).

Finally, control software2is implemented in Python to integrate all the features described

above, including controlling the electrical current, displaying and processing the image from the cameras, communicating with the electronic components in the setup, and operating the feedback loop.