How To Write A Store On A Microsoft Database On A Flash (Orchestra) On A Linux Computer (Ortho) On An Ipro (Orroster) On Your Computer Or Your Computer (For Ahem) On The

(1)

There and Back Again

As Quick As a Flash:

Tuning Performance in CDO

Stefan Winkler

Independent Software Developer and IT Consultant CDO Committer

[email protected] @xpomul on Twitter

(2)

CDO in a nutshell

Server

Client

Storage

(3)

CDO in a nutshell

Server

Client

Storage

(4)

CDO in a nutshell

Server

Client

Storage

CDOID attribute reference CDOID

(5)

CDO in a nutshell

Server

Client

Storage

(6)

CDO in a nutshell

Server

Client

Storage

(7)

Storage

CDO in a nutshell

Server

Client

Storage

(8)

Storage

CDO in a nutshell

Server

Client

Storage

attribute reference CDOID attribute reference

(9)

Storage

CDO in a nutshell

Server

Client

Storage

attribute reference CDOID CDOID attribute reference

(10)

Storage

CDO in a nutshell

Server

Client

Storage

(11)

Storage

CDO in a nutshell

Server

Client

Storage

(12)

Storage

CDO in a nutshell

Server

Client

Storage

(13)

Layers

App

Logic

CDO

Client

EMF

Model

CDO

Server

IStore

Code

Storage

(14)

App

Logic

CDO

Client

EMF

Model

CDO

Server

IStore

Code

Storage

(15)

Choose the

Right Storage!

MEM Lissome DB (h2 in-proc) DB (h2 net) DB (MySQL) Hibernate (MySQL) MongoDB (net) DB4O (in-proc) 1.701 519.063 22.314 9.381 4.269 1.486 1.475 1.121

time to read 10,000 objects (msec)

2.268 1.503 10.151 6.686 5.546 721 431 283

(16)

Choose the

Right Storage!

Store

Usage Scenarios

Pros

Cons

MEMStore prototyping, testing supports all CDO _features not persistent LissomeStore custom CDO file-based store, _{optimized for performance} _{full support planned}performance, _{(no external access)}custom file format

DBStore common/default solution supports all CDO

features DB selection and tuning adds to complexity HibernateStore custom DB schema required custom DB schema

and mapping only supports subset of CDO features, slower MongoDB Store No SQL DB wanted,

fast writing speeds required fast writing speeds slow reading speeds,limited commit size DB4OStore No SQL DB wanted,

fast reading speeds required fast reading speeds slower writing speedsdual licence OODB with commercial performance ? commercial licence,

(17)

Choose the Right

Database!

h2 (net) MySQL (net) PostgreSQL (net) 4.231 9.381 4.269

time to read 10,000 objects

6.606 6.686 5.546

(18)

Choose the Right

Database!

6.606 6.686 5.546

time to write 10,000 objects (msec)

(19)

Choose the Right

Database!

6.606 6.686 5.546

time to write 10,000 objects (msec)

Standard Configurations !!!

(20)

App

Logic

CDO

Client

EMF

Model

CDO

Server

IStore

Code

Storage

(21)

Choose the Right

Repository Mode!

Time

v1 v2 v3 v4 v5 v6 v7

Audit Mode

(22)

Choose the Right

Repository Mode!

Time

v1 v2 v3 v4 v5 v6 v7

Non-Audit Mode

(23)

Choose the Right

Repository Mode!

Time

0:v1 v2 v3 v4

v5 v6

Branching Mode

1:v1

2:v1

1:v2

2:v2 2:v3

(24)

Choose the Right

DBStore Mapping!

•

Audit:

•

plain vs. range-based list mapping

•

Branching:

•

plain vs. range-based list mapping

(25)

Choose the Right

DBStore Mapping!

date

sum

Order

date

sum

Product

date

sum

Product

number

price

Product

Order Order Order Order

id version date sum

1 2 3/29/12 102.34

Product Product Product Product

id version number price

2 2 1-423-4 39.12 3 2 1-485-3 60.71 4 2 1-584-2 2.51 Order_Product_List Order_Product_List Order_Product_List Order_Product_List

id version index value

•

non-audit repo mode

•

no historic („audit“) views

(26)

Choose the Right

DBStore Mapping!

id version date sum

1 2 3/29/12 102.34

2 2 1-423-4 39.12 3 2 1-485-3 60.71 4 2 1-584-2 2.51 Order_Product_List Order_Product_List Order_Product_List Order_Product_List

id version index value

1 2 0 2

1 2 1 3

•

audit repo mode

•

classic

•

always writes complete list

•

range-based

(27)

Choose the Right

DBStore Mapping!

id version date sum

1 2 3/29/12 102.34

2 2 1-423-4 39.12 3 2 1-485-3 60.71 4 2 1-584-2 2.51

•

audit repo mode

•

classic

•

always writes complete list

•

range-based

•

writes deltas

Order_Product_List Order_Product_List Order_Product_List Order_Product_List Order_Product_List

id from to index value

1 1 null 0 2 1 1 1 1 4 1 2 null 1 3 1 1 1 2 3

(28)

Choose the Right

DBStore Mapping!

•

branching repo mode

•

classic vs. range-based

•

copy-lazy (default) vs. copy-on-branch

Order_Product_List Order_Product_List Order_Product_List Order_Product_List Order_Product_List Order_Product_List

id branch from to index value

1 1 1 null 0 2 1 1 1 1 1 4 1 1 2 null 1 3 1 1 1 1 2 3

(29)

Choose the Right

DBStore Mapping!

•

branching repo mode

•

classic vs. range-based

•

1 1 1 null 0 2 1 1 1 1 1 4 1 1 2 null 1 3 1 1 1 1 2 3 Order_Product_List Order_Product_List Order_Product_List Order_Product_List Order_Product_List Order_Product_List

(30)

Choose the Right

DBStore Mapping!

•

branching repo mode

•

(31)

Choose the Right

DBStore Mapping!

•

branching repo mode

•

1 1 1 null 0 2 1 1 1 1 1 4 1 1 2 null 1 3 1 1 1 1 2 3

(32)

Choose the Right

DBStore Mapping!

•

branching repo mode

•

1 2 1 null 0 2 1 2 1 null 1 5 1 2 1 null 2 4

(33)

Choose the Right

DBStore Mapping!

2.250 4.500 6.750 9.000 721 1.491 1.481 688 4.581 1.473 1.483 736 8.941 1.506 1.442 681

time to insert 100 objects at beginning

time to insert 100 objects in the middle

time to insert 100 objects at the end

(34)

App

Logic

CDO

Client

EMF

Model

CDO

Server

IStore

Code

Storage

(35)

Choose the Right

Transport Layer!

CDO Client CDO Server CDO Client CDO Server Computer 1 Computer 2

TCP (true client/server)

TCP (same computer)

(36)

Choose the Right

Transport Layer!

CDO Client CDO Server CDO Client CDO Server Computer 1 Computer 2

TCP (true client/server)

TCP (same computer)

(37)

A Word On

Caching

•

Revision Caches in CDO

use Soft references

•

Caches grow until memory

limit is reached

•

View Cache can be

configured to use strong or

weak references instead

CDO

Client

CDO

Server

View Cache (Soft Ref) Session Cache (Soft Ref) Repository Cache (Soft Ref)

(38)

App

Logic

CDO

Client

EMF

Model

CDO

Server

IStore

Code

Storage

(39)

Choose the Right

Model Configuration!

Native Reflective Legacy 11.302 8.882 8.581

4.460 3.076

2.802

(40)

App

Logic

CDO

Client

EMF

Model

CDO

Server

IStore

Code

Storage

(41)

Make Use of

CDO Loading Policies!

•

Lazy Collection Loading

session.options().setCollectionLoadingPolicy( CDOUtil.createCollectionLoadingPolicy(

/* initial = */ 5, /* onResolve = */ 5));

•

Explicit Prefetching

CDOObject fromCDO = getMyObjectFromCDO(); fromCDO.cdoPrefetch(/* depth = */ 3);

•

Prefetching by Policy

(42)

Leftovers

•

Things I did not talk about

•

unordered collections

•

store queries (use with caution!)

(43)

Summary

•

Choose the Right Storage!

•

Choose the Right Database!

•

Choose the Right Repository Mode!

•

Choose the Right DBStore Mapping!

•

Choose the Right Transport Layer!

•

Choose the Right Model Configuration!

•

Make Use of CDO Loading Policies!