• No results found

Luminance Free Color Detection for Quantification and Automatic Segmentation in Microscopy: A Methodological Approach

N/A
N/A
Protected

Academic year: 2020

Share "Luminance Free Color Detection for Quantification and Automatic Segmentation in Microscopy: A Methodological Approach"

Copied!
9
0
0

Loading.... (view fulltext now)

Full text

(1)

Luminance-Free Color Detection for Quantification and

Automatic Segmentation in Microscopy: A

Methodological Approach

Teresa Lettini

1,*

, Gabriella Serio

1

, Tiziana Valente

2

, Flavio Ceglie

2

, Alessandra Punzi

1

, Rosalia Ricco

1

,

Vittorio Pesce Delfino

2

1Dept. Emergency and Organ Transplantation (DETO), Sect. of Pathology, Medical School, University of Bari 11 G. Cesare Square, 70124,

Bari, Italy

2Digamma Research Centre, Bari

*Corresponding Author: gabriella.serio1@uniba.it

Copyright © 2014 Horizon Research Publishing All rights reserved.

Abstract

This procedure is aimed at solving the chromatic quantification problem in histological images, as well as other fields. On digital images, this is possible using colorimetric software applications. Our solution was to adopt a pre-processing step of the analog signal coming from a video-source (cabling a suitable, dedicated unit to the video-line before signal grabbing). The unit is a hardware device that processes the voltage values of the video signal image section, line by line in the raster image exploiting the vertical interval. The output is a luminance-free video signal compatible with real-time needs (1/25 second), that is in turn compatible with the normal exploration speed of a histological slide. Some experimental results are presented.

Keywords

Chromatic Quantification, Microscopic Images, Immunohistochemistry

1. Introduction

The classical evaluation of objects in an image is based on visual analysis with the aim of detecting and classifying the diagnostic features. In all the fields, the analyst needs to consider the various different categories of information, that can be listed as follows: dimensions, shape, optical density, color. Such kinds of information have independent dynamics as regards both range and variation.

The human eye is capable of recognizing one hundred levels for each primary color; therefore, it can see a million of these different linear combinations. In the perception of color by humans, relevant optical signal processing takes place at several anatomical levels (cone cells of the retina, lateral geniculate bodies in the brain). In the cerebral cortex the signals are converted into color signal attributes, and analyzed.

Computerized image analysis systems simulate to some

extent the function of the human brain. Characteristic features of the color signal-saturation (that corresponds to pigment purity), hue (that represents the principal wave-length) and intensity – sometimes called luminance (the sum of light intensities received from the light source, with different color temperatures, in each of its three red, green and blue channels) - captured by a camera, are sent to an image processor, and analyzed.

For visualization, true color systems use 256 levels for each primary color (24 bits), so more than 16 million colors can be displayed. Colors can be specified as a linear combination of three numbers. These combinations form a three-dimensional color space within which colors can be specified. The RGB color space is created by suitably mixing together three primary lights, red (R), green (G) and blue (B).

The RGB, the HLS (Hue, Lightness, Saturation), and human-oriented color space CIELuv [CIE 1976 (L*, u*, v*) color space] [1] , which differ in the labeling of their three axes, are the color spaces most commonly used for display purposes. The RGB color space is based on the common observation that most colors can be created by suitably mixing together three primary lights. In this space a color is defined by the relative proportions of the red, green and blue display primaries required to produce it. This space can be considered as a 3D space with the origin at zero, and whose x, y and z axes are labeled by the Red, Green and Blue primaries, respectively. The value for each axis will range between 0 and 1 and the entire color space is contained within a cube. Colors that are plotted at the origin will appear black. A grey scale is represented by a vector from the origin along which Red, Green and Blue values are equal. The RGB color space is often employed in computers because of the three-gun hardware of the commercial CRT (Cathode Ray Tube) display and its lower computation complexity.

(2)

mixing colors than the RGB color space. The HLS space is a deformed version of the RGB cube. Colors are specified by their hue angle, between 0 degrees and 360 degrees, i.e. 120 degrees for blue and 240 degrees for green, with a lightness value that goes from black to white, through grey located on the z axis, and with a saturation value which is specified by moving in from the hue plane towards the z axis. The resulting 3D space is a double cone. Intuitive terms used to describe colors, such as tint shade and tone are easily represented in this space. However, the same changes of saturation or lightness produce different results in different zones of the space.

From the red, green and blue components the one that undergoes maximum absorption by a colored reaction product should be chosen. A simpler method is color segmentation. This can be executed either in the RGB color space or in the HSI (hue, saturation and intensity-luminance) color representation, and will provide a broad scale of image interaction and corrections. For instance, saturation enhancement can generate the segmented image in pastel colors.

Many applicative fields have long been in need of effective methods for color quantification and color-based segmentation [2]: physics, chemistry, medical imagery, quality testing for industrial lines, vegetable foods and flowers. In medicine this need is particularly relevant in

immunohistochemistry [3], [4], [5], [6], as a means of replacing the traditional score evaluation by a cellular chromatic quantification which could be either in points-form or referred to the surface of relevant objects in the field of observation [7-14].

2. Methodology

The goal is to create machines which can be combined with observation systems and which have advantages in relation both to the image processing function for the best display of significant morphological patterns and to image analysis by quantitative evaluations, without slowing down the observer’s work or making it more difficult. This means that a pre-processing device should work in “real time” [7].

[image:2.595.79.530.405.735.2]

Our solution was to adopt a pre-processing step of the analogical signal coming from a video-source (cabling a suitable, dedicated unit to the video line before signal grabbing). The unit is a hardware device which works on the voltage values of the video signal image section, line by line in the raster image exploiting the vertical interval. The output is a luminance-free video signal compatible with real-time needs (1/25 second), that is in turn compatible with the normal exploration speed of a histological slide. Figure 1A depict the steps involved in the whole process.

(3)

We stress the influence of the chromatic components, in the “white” light provided by the lighting source (microscope lamp with a variable colour temperature) which prevents a correct measuring standardization.

Skipping out the luminance from the image signal simultaneously achieves the elimination of the “white background”, typical of the transmitted lighting, together with the effects of the corresponding chromatic components on coloured objects.

As regards the above categories of density and color information, it is possible to observe that density information is connected to encoding, in the image, the differential characteristics of light transmission or reflectance of the studied object in relation to the lighting source intensity, whereas color information is characterized by the introduction of differential wavelength filtering (for instance, in microscopy after the application of staining techniques to reveal morphological, structural or biochemical substrata). In particular, immunohistochemical staining techniques are widely utilized in diagnostics for determining immunocomplexes in order to detect reactive material with an antigenic significance. In this application there is clearly a great need for practical and fast quantitative determination. When we say “real time”, we mean that the image acquisition, carried out by the machine, its processing and redisplay, must be performed within a time limit no longer then 1/25 second. In this way images processing will be compatible with the video display rate and, therefore, suitable for use in a version offering immediate assistance to the routine operation of observations by close exploration of a specimen with frequent adjustments of magnification and focus. These requirements are very demanding, and image processing functions can help to extract information on density and color and to perform segmentation of objects.

This involves a shift from image observation, in “relative mode”, as we call it because the colors are seen in relation to the chromatic components of the “white light”, to image observation in “absolute mode”, when this relation is suppressed. The present procedure was conceived primarily as “stand-alone”, to solve only the chromatic quantification problem within a single laboratory fitted with a “chromax unit” (the term “chromax” stands for the unit we planned and built).

For telepathology, chromatic quantifications are important; it was therefore necessary to modify the configuration of digital images that are converted to analogical ones, dealt with by the “chromax unit” and subsequently digitized before finally being submitted to colorimetric determinations.

To obtain luminance-free images with software tools, the MATHLAB® 7.0.1 package was used, that makes it possible to produce 28 modules of integrated functions used as a plug in for ImageJ® software (ImageJ is a software package for image processing and analysis written in JAVA by Wayne Rasband from National Institutes of Health).
 It is an open source JAVA 1.1 and could be used as applet, servlet etc.; it

works using scripts and macros and requires MATLAB® 7.0.1 and ImageJ installed.

After acquiring an image complete with luminance data, the luminance subtraction program is run using the “luminancefree.m” module. At the end of the procedure the luminance-free images are saved.

The software modules created also allow binary imaging, to create binary masks for the three RGB channels, operating on the relative levels of gray by defining their threshold values and making statistical analyses on the basis of the mean, standard deviation and weighted mean.

Luminance subtraction remarkably increases the possibility of automatic segmentation of objects and improves the results by enhancing the bimodal distribution of the histogram associated to the image. Without luminance subtraction, the background is much less well differentiated from the object. The adoption of the pre-processing unit therefore seems to provide considerable advantages in the field of automatic segmentation of images stained by immmunohistochemical methods.

The subtraction of luminance achieved with hardware results much faster but is associated with a loss of spatial resolution of the image. The best results in terms of segmentation and maintenance of the original image resolution are obtained with software, provided that there is correct balancing of the white (in fact, specific controls of the color temperature of the microscope lamp are made).

From the practical point of view, adopting the software solution for automatic segmentation applications is compatible bearing in mind that for segmentation operations real time performance is not required, although this is very useful for the quantitative control of images during observational exploration.

The hardware solution was definitely found to be best suited for the quantification of color data, and the software for the purposes of segmentation.

The whole procedure ends with the calculation of the value of the equivalent algorithm.

The equivalent algorithm for numerical reading on the digital display is:

1

( )

N

i i

i

V S xmZ

=

=

V, for all colored objects (N), is the sum of values obtained by multiplying the areas (Si ) by the mZ of each colored object (i.e. nucleus) in the field under observation.

The machine does not perform a numerical calculation after digital sampling but an analogic electric measurement on the videosignal by means of hardware filter architecture that works at very high speed (real time). Thus, the mathematical formula in equation is the numerical equivalent of the electronic processing actually performed.

(4)

Figure 1B. Measuring space graph.

If we consider each field observed as an X-Y plane containing information about the number (N) and surface (S) of positive objects (i.e nuclei), the variation of fundamental color selected is expressed along the Z axis; its measured value (mZ) is considered only if greater than a fixed threshold value.

For these applications we found that 40x microscopic magnification was the best for handling immunohistochemical images because it provided higher values of the equivalent algorithm; the mean value shown to be more robust and reliable was the “weighted mean”.

Mathlab is a registered trademark The MathWorks, Inc. Other product or brand names are trademarks or registered trademarks of their respective holders.

3. Experiment and Results

3.1. Examples of procedure

To test the performances of “chromax units” we used nine series of duplicate images in both the “relative” and “absolute” (luminance-free) modes. To test the quantifications, a typical bar video signal was used together with a commercial colorimetric routine (Color-Cop) (Figs. 1C-5). To test the segmentation two series of duplicate images (partly withered leaves and cells) were used (Figs. 6-9).

[image:4.595.78.270.77.231.2]

Figure 1 shows RGB values for the white bar. The colorimeter (Color Cop with a reading range from 0 to 255 for each RGB channel) is displayed on the left side. For the relative mode, white bar, the reading values were Red=254, Green=252, Blue=253. For the absolute mode, white bar, the reading values were Red=104, Green=113, Blue=106.

[image:4.595.322.550.81.355.2]

Figure 2 shows RGB values for the blue bar. For the relative mode, blue bar, the reading values were Red=103, Green=122, Blue=255. For the absolute mode, blue bar, the reading values were Red=64, Green=84, Blue=254.

Figure 1C. Relative mode top; absolute mode bottom; white bar (dropper position)

[image:4.595.321.549.397.675.2]
(5)

Figure 3 shows RGB values for the black bar. For the relative mode black bar the reading values were Red=101, Green=107, Blue=111. For the absolute mode, black bar, the reading values were Red=109, Green=102, Blue=121.

Importantly, very similar RGB metering values were obtained for the black bar in both the relative and absolute mode, as shown in fig. 3, and corresponding values for the white bar in absolute mode, as shown in fig. 1

Figure 3. Relative mode top; absolute mode bottom; black bar (dropper position)

[image:5.595.324.535.137.399.2]

Figure 4. Relative mode top; absolute mode bottom; red bar (dropper position)

Figure 4 shows RGB values for the red bar. For the relative mode, red bar, the reading values were Red=254, Green=98, Blue=134. For the absolute mode, red bar, the

reading values were Red=255, Green=21, Blue=18.

Figure 5 shows RGB values for the green bar. For the relative mode, green bar, the reading values were Red=117, Green=254, Blue=66. For the absolute mode, green bar, the reading values were Red=0, Green=209, Blue=0.

[image:5.595.81.284.173.426.2]

Figure 5. Relative mode top; absolute mode bottom; green bar (dropper position)

Figure 6 shows the results of color-based segmentation on the image of leaves in the relative mode: the original color image and RGB splitting are in the left column and the corresponding edge detection in the right column, the absolute mode color image is shown at the top for comparison.

[image:5.595.80.279.457.696.2] [image:5.595.349.510.501.740.2]
(6)
[image:6.595.361.502.78.330.2]

Figure 7 shows the results of color based segmentation on the leaves image in the absolute mode: the original color image and RGB splitting is in the left column and corresponding edge detection in the right column, the relative mode color image is shown at the top for comparison.

Figure 7. Leaves, absolute mode. RGB splitting and edge detection

Figure 8 shows the results of color based segmentation on an image of cells in the relative mode: the original color image and RGB splitting are in the left column and corresponding edge detection in the right column, the absolute mode color image is shown at the top for comparison.

[image:6.595.107.250.149.372.2]

Figure 8. Cells, relative mode and RGB splitting and edge detection

Figure 9 illustrates the results of color-based segmentation on the cells image in the absolute mode: the original color image and RGB splitting are in the left column and the corresponding edge detection in the right column, the relative mode color image is shown at the top for comparison.

[image:6.595.236.376.470.726.2]
(7)

3.2. Examples of medical application (immunohistochemistry)

3.2.1. Software Procedures

A B

C D

Figure 10. Lymphoma non Hodgkin B. Positivity for CD20. (X 40, original magnification). A). Relative mode. B) Histogram of RGB color levels. C) Absolute mode. D) Histogram of RGB color levels.

A B

[image:7.595.98.515.119.401.2]

C D

[image:7.595.96.511.432.715.2]
(8)

A B

[image:8.595.98.519.74.342.2]

C D

Figure 12. Carcinoma of the breast. Estrogen positive nuclei (X 40, original magnification). A) Relative mode. B) Histogram of RGB color levels. C) Absolute mode. D) Histogram of RGB color levels.

3.2.2. Hardware Procedures

A B

Figure 13. Lymphoma non Hodgkin B. Positivity for CD20. (X 40, original magnification). A) Relative mode. B) Absolute mode.

A B

[image:8.595.104.512.406.545.2] [image:8.595.102.516.591.728.2]
(9)
[image:9.595.98.513.75.211.2]

A B

Figure 15. Breast carcinoma. Estrogen positive nuclei (X 40, original magnification). A) Relative mode. B) Absolute mode.

4. Conclusions

A high speed pre-processing unit allows the luminance component to be subtracted from an analog video signal coming either from a local video camera or a digital analog converter for digital images trained from websites for remote applications. The main advantages in our experience were a reliable color quantification that was not affected by the chromatic components of the lighting source, together with a more effective color-based object segmentation achieved by RGB channel splitting and ordinary edge detection methods.

The Authors suggest the efficiency of this method (that is fast and easy to use) to assist pathology in routine medical image analysis and diagnosis. So, this study could be recommended for the selection, classification and automatic quantification of microscopic objects in assisted histological diagnosis.

Acknowledgments

The Authors are grateful to Mary V. Pragnell, BA, for language assistance.

REFERENCES

[1] Schanda J, (2007). Colorimetry: Understanding the CIE System. Wiley Interscience. pp. 61–64.

[2] Botticelli AR, Martinelli AM and Botticelli L, Morphocromatic Analysis of Cell Nuclear Markers in Human Tumours. European Microscopy and Analysis, 40:13-14, 1996

[3] Coltelli P and Gualtieri G. Colour Techniques in Digital Microscopy. European Microscopy and Analysis, 31:25-27, 1994

[4] Juliš I and Mikeš J. True Colour Image Analysis and Histopathology. European Microscopy and Analysis,

18:11-13, 1992

[5] Monaghan P and Moss D. An Introduction to Immunocytochemistry. European Microscopy and Analysis, 37:9-11, 1995

[6] Dìaz G, Gonzàles FA, Romero E. A semi-automatic method for quantification and classification of erythrocytes infected with malaria parasites in microscopic images. J Biomed Inform 42 (2): 296-307, 2009

[7] Pesce Delfino V, Lettini T, Troia M, De Benedictis S, Ricco R. Processing Microscope Images in Real Time. European Microscopy and Analysis, 26:25-29, 1993

[8] Ricco R, Cimmino A, Renzulli G, Serio G, Lozupone A, lattini T, Pece A, Giardina C, Pesce Delfino V. Real-time quantification of the proliferative state in astrocytomas. Anaytical Quantitative Cytology and Histology 22(3); 213-217, 2000

[9] Van Oostveldt PMV: Simple image analysis techniques in histochemistry. European Microscopy Analysis, 10: 7-11, 1991

[10] Pesce Delfino V, Lettini T, Troia M, Ricco R. Analisi di immagini microscopiche in tempo reale. In: Workshop Image Analysis and Morphometry, Taormina (Italy), pp 108-118, 1992

[11] Hsu CY, Ho DM, Yang CF, Chiang H. Interobserver reproducibility of MIB-1 labelling index: high levels suggest limited prognostic usefulness for patients with primary brain tumors. Cancer, 92(10): 2720-2726, 2001

[12] Sucju C, Muresan A, Cornea R, Cucju O, Dema a, Raica M.Semi-automated evaluation of Ki67 index in invasive ductal carcinoma of the breast. Oncology Letter, 7(1):107-114, 2014

[13] Nielsen PS, Bentzer NK, Jensen V, Steiniche T, Jylling AM. Immunohistochemical Ki-67/KL1 double stains increase accuracy of Ki-67 Indices in breast cancer and simplify automated image analysis. Appl Immunohistochem Mol Morphol. 2014 Jun 3, online.

Figure

Figure 1A.  Flow chart of image processing
Figure 1B.  Measuring space graph.
Figure 4 shows RGB values for the red bar. For the
Figure 7 shows the results of color based segmentation on
+4

References

Related documents

Here, the present work describes a reliable 3D numerical model with material nonlinearility for the simulation of prestressed steel-concrete composite beams with internal bonded

Table 1 shows the results of descriptive statistics such as mean and standard deviation related to the variables of the present study: job stress (interpersonal

In this suggested system the color image bands compress independently using the traditional linear polynomial coding of variable block sizes that changes according to

Comparison of 1 17 O data from two independent sets of calibrations of reference waters and standards measured by laser spectroscopy on the L2140-i (CRDS, open squares) with

This differential path search strategy is natural when one will handle the non-linear parts in a classic way (i.e. computing only forward) during the collision search, but in Section

In this report we describe a case of intra ACL mucoid degeneration and ganglion cyst combined with intra- osseous cysts in a 20-year-old medical student and keen rugby player who had

Parlier-Cuau C, Wybier M, Nizard R, Champsaur P, Le Hir P, Laredo JD: Symptomatic lumbar facet joint synovial cysts: clinical assessment of facet joint steroid injection after 1 and