Medical Imaging, Acquisition and Processing

(1)

Medical Imaging, Acquisition and

Processing

J. Kybic

Center for Machine Perception, CTU Prague

(2)

Overview

Introduction and motivation

Modalities (acquisition devices)

Microscopy, X-rays, CT, Ultrasound, PET/SPECT, EEG/MEG, . . .

Data processing

Preprocessing, Reconstruction, Segmentation, Registration, Classification, . . .

(3)

Medical imaging pipeline

(4)

Medical imaging pipeline example

patient/subject Physical property Imaging device Raw data Preprocessing Improved data Reconstruction Images/volumes Interpretation

(5)

Medical imaging pipeline example

patient/subject Physical property Imaging device Raw data Preprocessing Improved data Reconstruction Images/volumes Interpretation Quantitative/qualitative inf. physician Decision

(6)

Medical imaging pipeline example

patient/subject Physical property Imaging device Raw data Preprocessing Improved data Reconstruction Images/volumes Interpretation

(7)

Imaging device

Physical property: density, absorbance,

conductivity, . . . _{Raw data}

Examples:

CT, X-rays, MRI, ultrasound, . . .

(8)

Preprocessing

A priori knowledge Preprocessing Raw data Preprocessed, cleaned, enhanced data Examples:

(9)

Reconstruction

Physics of the acquisition device

Reconstruction

Preprocessed raw data

Physically meaningfull images, volumes,

sequences

Inverse problem

Examples: Tomographic reconstruction

MRI, ultrasound . . .

(10)

Interpretation

A priori knowledge Interpretation High-level processing Images, volumes, sequences Quantitative

& qualitative inf. size, changes, metabolic inf.

(11)

Modalities

(12)

Microscopy

(13)

Microscopy — examples

placenta cross-section

(14)

(15)

Microscopy — examples

crocodile ear slice

(16)

Microscopy — examples

(17)

Microscopy — types & trends

Electron microscopy

Electron transmission microscopy

Confocal microscopy – reject out-of-focus light, scanning Phase-contrast microscopy

Fluorescence microscopy fluorescent dyes

multiple sensing channels/filters multiple light sources – visible, UV CCD cameras

supercooled superresolution

Moveable specimen tray Auto-focussing

Automated acquisition, mosaicking

(18)

Microscopy

Advantages

High-spatial resolution

Colour and texture information Affordable (optical microscopy)

Proven technique – large body of experts available Disadvantages

Difficulties of in-vivo observations Inherently 2D

(19)

X-rays / Radiography

1895, W. Röntgen B. Röntgen hand modern hand

(20)

X-rays / Radiography (2)

(21)

X-rays / Radiography (3)

Chest X-ray:

(22)

X-rays / Radiography (4)

Advantages

Widely used and available Experts available

High-spatial resolution

Excelent imaging of hard tissues (bones) Disadvantages

Radiation exposure

Difficulty in imaging soft-tissues 2D projection, hidden parts

(23)

X-rays / Radiography (5)

New trends

CCD sensors replace film

higher sensitivity, faster exposure, lower dose dynamic imaging

(24)

Computed tomography

(25)

Computed tomography (2)

Principle:

?

(26)

Computed tomography (3)

(27)

Computed tomography (3)

Example, head:

(28)

Computed tomography (4)

Advantages 3D volume

High-spatial resolution Disadvantages

Increased radiation dose Longer acquisition times

Shadows, difficulties imaging soft-tissues Costly equipment

(29)

Ultrasound – Equipment

(30)

(31)

Ultrasound (2) – Examples

Fetus

(32)

Ultrasound (2) – Examples

(33)

Ultrasound (2) – Examples

Heart, Doppler

(34)

Ultrasound (3)

Advantages Low-cost

Proven technique – Experts available Not invasive, no harmful effects

Good imaging of soft tissues Easy dynamic acquisition Disadvantages

Low-signal quality, speckle noise

(35)

Ultrasound (4)

New trends

3D ultrasound

High-frequency ultrasound Doppler ultrasound

(36)

MEG and EEG

(37)

MEG sensors

SQUID array Gradiometers

(38)

Currents in the brain

100 mV 1 ms -80mV Repolarization Propagation Depolarization Quadripolar structure of -80 mV K+ K+ _Na+ + 20 mV Na+ Action Potential action potential Axon Membrane Ion-pump Ion-pump

(39)

Magnetic measurements

(40)

Source localization

(41)

MEG and EEG

Advantages

EEG widely available

Excellent temporal resolution,

can observe brain activities at ms resolution Disadvantages

MEG extremely expensive

Poor spatial sampling / resolution

Good reconstruction algorithms not yet available Necessity of a good head model

(42)

Magnetic resonance imaging (MRI)

(43)

MRI – Principles

ω

=

γ

B

(44)

(45)

MRI – Example

Brain slice:

(46)

MRI

Advantages

Good spatial resolution True 3D acquisition

Excellent contrast for all tissue types In vivo, non-invasive, no radiation Disadvantages

Very expensive

Acquisition somewhat long (3D) Poor temporal resolution (3D)

(47)

MRI – Types and trends

Sequences: gradient-echo, spin-echo,. . . Weighting: T1, T2, PD, BOLD,. . .

Acceleration: EPI, FLASH, parallel, . . . Modalities: fMRI, perfusion, flow, . . .

(48)

Other imaging methods

Positron emission tomography (PET)

Single positron emission tomography (SPECT) Gamma camera

Termography . . .

(49)

Medical data processing

(50)

Medical imaging specificities

(wrt. Computer vision)

We want the physical reality, not to emulate human perception.

We deal with many modalities, optical intuition often does not apply. Data are rare and expensive — maximum information must be extracted. Approximations to be avoided if possible.

Data are continuous, result is continuous — we do not decide, we help to decide

Algorithms must be robust and safe — give no answer or a conservative answer, rather than a wrong answer.

(51)

Basic data processing tasks

Low level operations Segmentation

Registration Classification

Reconstruction / Inverse problems . . .

(52)

Contract enhancement

(53)

Edge detection

retinal cells

(54)

Segmentation

(55)

Segmentation (2)

Easy for humans, difficult for computers.

We do not know how brain perform segmentation.

Algorithm is image and task dependent, needs a priori knowledge.

Useful for:

Separating objects, masking background Image simplification

Shape, size and other statistics

(56)

How do we segment?

(57)

How do we segment?

Cells are homogeneous connected components.

(58)

How do we segment?

(59)

Segmentation from MRI (T1)

10/25

BRAIN MRI, SEGMENTATION RESULT

original, gray and white brain matter, CSF

(60)

Image registration

Find corresponding points

Manual, automatic, semi-automatic Useful for:

Comparing images from different times

Comparing images from different methods Comparing images from different subjects Analyzing movement

Segmentation

(61)

Image registration (2)

reference image test image

deformation

warping registration

warped image

(62)

Image alignment

reference test

(63)

Image alignment

reference test

before warped

(64)

Manual registration

(65)

Registration as minimization

deformation optimization test image reference image deformed image difference criterion E deformation function g(x)

(66)

Registration examples

EPI distortion MRI atlas CT alignment SPECT atlas Ultrasound MRI heart sequence MRI perfusion

(67)

Registration examples

EPI distortion MRI atlas CT alignment SPECT atlas Ultrasound MRI heart sequence MRI perfusion Atlas

(68)

Registration examples

(69)

Registration examples

(with University Hospital in Geneva)

(70)

Registration examples

(71)

Registration examples

(72)

Registration examples

EPI distortion MRI atlas CT alignment SPECT atlas Ultrasound MRI heart sequence MRI perfusion or iginal uncorr ected corr ected

(73)

Tomographic reconstruction

?

The problem

(74)

(75)

Tomographic reconstruction

Another solution

(76)

Radon transform

si,1 si,2 si,j angle θi u t

(77)

Radon transform

si,1 si,2 si,j angle θi u t

(Rf )(θi, uj) = R f t cos θi − uj sin θi, t sin θi + uj cos θi dt

I = I0 exp − Z r∈S ρ(r)ds(r)

(78)

Reconstruction algorithms

Phantom 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 Phantom 50 100 150 200 250 50 100 150 200 250

(79)

Reconstruction algorithms

Backprojection Variational reconstruction

0 10 20 30 40 50 Backprojection nt=8 nu=32 50 100 150 200 250 50 100 150 200 250 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 Variational reconstruction nu=32 nt=8 gamma=1

50 100 150 200 250 50 100 150 200 250

8 angles (each 22◦_{), 32 samples per angle}

(80)

Medical imaging – Conclusions

Medical imaging experiences a recent boom: Many modalities

Instruments are digitalized and more widespread More powerful and affordable computers

Large quantity of data is produced. Computers can help to:

make data meaningful

extract previously unavailable information visualize, compare and analyse

scan for abnormalities

provide the right information for physicians to make the decision

(82)

Medical imaging – Conclusions

Medical imaging experiences a recent boom: Many modalities

Instruments are digitalized and more widespread More powerful and affordable computers

Large quantity of data is produced. Computers can help to:

make data meaningful

extract previously unavailable information visualize, compare and analyse