Image quality issues in digitization projects of historical documents

(1)

Istituto di Fisica Applicata “Nello Carrara”

National Research Council (CNR)

Firenze, Italy -

www.ifac.cnr.it

Image quality issues in

digitization

projects of historical documents

Franco Lotti

(2)

IFAC activity in Italian digitisation projects of ancient docume

nts

¾

IMAGO-II

projects

(1998-2000).

State Archives of Florence and Lucca

(more than 100,000 parchment rolls of the

Diplomatico

fonds, VIII-XIV

century).

http://www.archiviodistato.firenze.it/progetti/attivite.htm

http://www.comune.lucca.it/archiviostato/supp-inf.html

¾

Candido

project

(2000-2003).

University Library of Pisa

(about 85.000

images of manuscripts, drawings, printed books on paper and parchment,

collection of letters of famous personalities, periodicals from XIV to XIX

century).

http://www.cab.unipd.it/eventi/pisa.php3

;

http://www.pisa.sbn.it

¾

Datini

project

(2001-2004).

State Archive of Prato

(450,000 images of

Francesco Datini’s collection of private and trade letters, bookkeeping notes,

management books, etc. – XIV century).

http://www.archiviodistato.prato.it

¾

Monumenti Nazionali

project

(2002-2004).

State Archive of Frosinone

(feasibility study for the digitisation and web consultation of parchments

preserved by various Middle Ages abbeys and monasteries in Central Italy).

(3)

D

igitisation

p

olicies

of the European Union

Lund

-

4th April 2001: EC expert’s meeting

to accomplish coordination mechanisms of digitisation

policies and programmes across Europe

⇒

Lund Principles

⇒

Lund Action Plan

“The European culture can be freely

accessible through the digitisation of

cultural content.

This will also support and promote the

cultural difference in a global scenario”

(4)

D

igitisation

p

olicies

of the European Union

MINERVA PROJECT

(coordinator

:

Italy)

a network of Member States’ Ministries

--

to implement the Lund Action Plan;

--

to discuss, correlate and harmonise activities in

digitisation of cultural and scientific content;

--

for creating agreed European common

recommendations and guidance about:

--

digitisation

--

metadata

--

long

-

term accessibility

(5)

Good practices in digitisation projects

- Analysis of d

ocument

s

--

Safe document treatment and management

--

Definition of user’s requirements

THE FIRST STEPS

--

Q

uality

of acquired images

--

Efficient and fast retrieval

(6)

Simplified layout of a

digitisation

project

IMAGE ACQUISITION METADATA COMPRESSION . . . .

Docu-ment

Master

• Indexing - formatting and organisation of metadata

• Accessibility and dissemination strategy

• Maintenance plan

Development of the project

(8)

COSTS

Image quality

⇒

Image size

⇒

Storage

Good practices in digitisation projects

⇓

(9)

--

Illumination system

--

Type of benches and document

supports

--

Sensor characteristics

--

Preprocessing

--

Calibration procedures and stability

--

Optical and mechanical performances

--

Compression

F

actors affecting image quality

(10)

--

Material typology

--

Binding type

--

Status of conservation and fragility

DOCUMENT CHARACTERISTICS:

--

Light sensitivity

--

Size and type of content

--

B&W, grey or colour

Choosing the right instrumentation

(11)

--

S

ampling

rate (

ppi

-

pixel per inch)

--

Number of elements and geometry (

li

near,

matrix)

--

Spatial resolution, MTF

(cycles/mm)

--

Bit depth (

bpp

-

bit per pixel)

SENSOR CHARACTERISTICS:

--

Type of scan (one shot, three s

hots

)

--

Type of view (fixed

-

planetary, flat

-

bed, …)

(12)

Sine

-

wave patterns:

to evaluate the

modulation transfer

function (MTF)

Square

-

wave

patterns: to

evaluate the

contrast

transfer

function (CTF)

Test charts for spatial resolution

Evaluation of the sensor performances

(13)

Computing the Modulation

Transfer Function (MTF), by the

acquisition of sine

-

wave test

charts, is the correct way to

evaluate the

spatial resolution

.

The evaluation of the Contrast

Modulation Function (CTF)

,

by the

acquisition of square

-

wave test

charts,

tends to over

-

estimate the

quality of the sensor.

MTF and CTF

MTF

CTF

(14)

Colour

Test charts

(15)

Choosing the right instrumentation

Light, UV fraction and temperature measurements

(16)

Image compression

A

A –

–

Reversible (

Lossless

) methods

:

•

Bitmap

•

TIFF uncompressed

•

TIFF LZW (Lempel Ziv Welch)

•

PNG

•

GIF

•

(17)

Image compression

B

B –

–

Lossy

methods

:

•

JPEG

•

(18)

JPEG Compression 1: 20

-

artifacts

300

(19)

JPEG Compression 1: 20

-

artifacts

1 cm

(20)

JPEG Compression 1:50

(21)

JPEG 2000 file format

•

Compatibility with ISO standard

•

Openness

•

Interoperability (systems compliant with the standard)

•

Non

-

proprietary

•

Supports embedded metadata

•

IPR and XML boxes for property and vendor information

•

Scalability, both in quality and resolution

•

Lossy

and

lossless

decompression

•

Progressive display

•

(22)

Example:

Comparison

between

JPEG

and

J2

K

Case A

:

Bit rate = 2.55 (CR = 9.4)

Ggood

quality for LAN

consultation (intranet)

Case B

:

Bit rate = 0.46 (CR = 51,8)

Low quality: for web

dissemination (internet)

Datini

Project

(State Archive of Prato)

(23)

High

-

Q JPG

–

(intranet)

Low

-

Q JPG

–

(internet)

Example:

(24)

0.39

186

9.53

51.83

0.46

Low quality

J 2K

0.81

40

4.12

9.40

2.55

High Quality

J2K

0.33

187

10.93

51.83

0.46

Low quality

JPEG

0.73

55

5.05

9.40

2.55

High Quality

JPEG

Q-index

(**)

PE (*)

RMSE

(*)

CR

Bit rate

(*) Green channel

Example:

(25)

Some examples

1 - Parchments

3 - Printed paper

4 - Manuscripts

5 - Drawing

2 - Seals

(26)

Example 1

-

Project:

IMAGO II

Digitisation

of the

Diplomatico

fonds

State Archive of

Florence:

More than

140,000

Parchment rolls

681 provenances

VIII

–

XIX century

A box of parchment rolls of the Diplomatico.

State Archive of Florence

(27)

Example 1

-

Project:

IMAGO II

Digitisation

of the

Diplomatico

fonds

State Archive of Lucca:

About

23

,

000

parchment rolls

70 provenances

VIII

(28)

Example 1

-

Project:

IMAGO II

Digitisation

of the

Diplomatico

fonds

To ensure safety of

document handling,

suitable sliding

windows have been

designed for the

acquisition of

parchment rolls on

both sides

(recto and verso)

(29)

Presentation 1:1

Example 1

Project:

IMAGO II

Digitisation of parchment

manuscripts

Sensor :

Matrix, Three shots

(no interpolation)

Spatial sampling:

200

ppi

Colour depth:

36 bit/pixel,

rescaled to 24

bpp

Compression:

JPEG

average CR: ~ 10

Access:

Intranet.

The quality of compressed

images showed a hardly

appreciable degree of loss,

Enlargement: 4:1

(30)

Example 2

-

Project:

IMAGO II

Digitisation

of the

Diplomatico

fonds

ACQUISITION OF

WAX SEALS

Sensor :

Matrix, Three shots

(no interpolation)

S

p

a

tial

sampling

:

4

00

ppi

Colour depth:

36 bit/pixel

rescaled to 24

Illumination: 2 flashes,

a

symmetric

C

ompression

: JPEG

,

average CR

(31)

Example 3

- CANDIDO

Project

–

Periodicals (XIX Century)

Sensor :

Three

-

linear scanner

S

patial

sampling

:

200

ppi

Colour depth:

24 bit/pixel

C

ompression

:

JPEG, CR

≈

10

Access

(32)

Example 4

- CANDIDO

Project

–

Correspondance Rosselmini-Gualandi

Sensor

: Scan back, Three

-

linear

array

S

patial

sampling

:

300

ppi

Colour depth

: 48 bit/pixel

rescaled to 24 bpp

C

ompression

: JPEG

, CR

≈

10

Access

(33)

Example 4

- CANDIDO

Project

–

(34)

Example 4

- CANDIDO

Project

–

(35)

Example 4

- CANDIDO

Project

–

(36)

Example 5

- CANDIDO

Project

–

Rosellini - Drawings

Sensor

: Scan back, Three

-

linear

array

S

patial

sampling

:

30

0

ppi

Colour depth

: 48 bit/pixel,

rescaled to 24 bpp

C

ompression

: JPEG

, CR

≈

10

Access

(37)

Conclusions

- Image quality depends on a number of factors related to the

capture methods and the further processing steps.

- Difficult to weight all those factors in a rigorous way.

- Suitable test procedures advisable, also tuned to the

specific targets.

Key points:

Correct acquisition of masters

Adaptive and progressive compression and smart,

robust IRP methods

(38)

Istituto di Fisica Applicata “Nello Carrara”

National Research Council (CNR)

Firenze, Italy -

www.ifac.cnr.it

… Thank you for your kind

attention !