Kiwibank ib at Startupt

Kiwibank: Startup to Enterprise in 7 years

Wayne Knowles

Ki ib

k t St

t

2

Kiwibank at Startup

ƒ Established by Labour-Alliance Coalition Government

ƒ Low Startup BudgetLow Startup Budget

ƒ Leveraged NZ Post for Infrastructure

ƒ Branch Network using existing Postshop’s and franchiseBranch Network using existing Postshop s and franchise network

ƒ Leverages ISP Services

Ki ib

k t St

t

2002

3

Kiwibank at Startup - 2002

ƒ IBM p-Series for “Core” banking database

ƒ IBM x-Series machines running Microsoft Applications

ƒ File & Printer services

E h M il

ƒ Exchange Mail server ƒ Citrix

ƒ Several SQL Server databases

ƒ Entry Level IBM SAN

Fil & P i t l SQL S D t b (< 1TB) ƒ File & Print plus SQL Sever Databases (< 1TB)

R

d T S

Road To Success

• Acquisitions • Many customers

• Guess

requirements

Not sure what we

• Brand developed

• Building market knowledge

• Many products

• Know what customers expect

• Not sure what we would become!

Ki ib

k G

th

5

Kiwibank Growth

U d

t

d Th A

li

ti

6

Understand The Application

Understand how CPU, memory and storage is impacted by growth

-ƒ Linear: resource_usage = n x customers

ƒ Exponential: resource_usage = n x customers2

Silicon is faster than Disk (Ferrite):

ƒ Keep “working set” in memory for 24 hours

ƒ Expect high I/O and performance impact after a reboot

U d

t

d Th A

li

ti

7

Understand The Application

Eliminate the high cost operations: ƒ Disk I/O ReadsDisk I/O Reads

ƒ Process startups (forks)

Best way to find bottlenecks: perform full load test

ƒ Commercial tools available (Tivoli Loadrunner etc)

ƒ Commercial tools available (Tivoli, Loadrunner etc)

ƒ Use Standard OS Tools (truss, sar and perfmon)

ƒ Especially when sharing performance data with external vendorsEspecially when sharing performance data with external vendors

U d

t

d St

G

th

8

Understand Storage Growth

Data can be classified into 2 categories: ƒ StructuredStructured

ƒ Unstructured

Most organizations recognize that unstructured data Most organizations recognize that unstructured data

growth is out of control.

Multiple copies of the structured data can also result in a large storage requirement.

St

G

th

Storage Growth

Stage MIS ODS

Consider the storage requirements of multiple test g

environments!

A hi i d D D h l thi th

S

t C

it M

t

10

Smart Capacity Management

ƒ “Radar Screen” forcasting

ƒ One should understand how the application growsOne should understand how the application grows ƒ Track the upper limitTrack the upper limit

ƒ Ie. xx Months before next impact

ƒ When highlighting issues to management use correctWhen highlighting issues to management use correct terminology

ƒ Understand Vendor Lead-Times

B ildi

M

bl I f

t

t

11

Building Manageable Infrastructure

Engineer upfront for cater for future upgrades

ƒ

CPU or Memory upgrades are easy

ƒ

CPU or Memory upgrades are easy

ƒ

Capacity on Demand

Consider future Forklift upgrades

ƒ

Put the Customer first and minimize downtime.

ƒ

Smart choice of technology to minimize downtime

ƒ

Kiwibank has performed 2 major forklift upgrades

Bi B

U

d

12

Big Bang Upgrades

ƒ Triggered by Batch Window timeTriggered by Batch Window time

ƒ Migrated to new machine using mksysb migration (AIX)

ƒ Implemented in p-Series LPARImplemented in p Series LPAR

ƒ Resources allocated to allow future AIX upgrades

ƒ Outsourced the solution – implemented in 8 weeks

ƒ Outsourced the solution implemented in 8 weeks

R li bilit

13

Reliability

ƒ Measure and improve reliability

ƒ Full analysis of all problemsu a a ys s o a p ob e s

ƒ Concentrate on repeat problems first

More Fingers = More Problems

ƒ Reduce amount of touching on the system to a minimumg y

ƒ Automated monitoring and reporting to eliminate daily

checks

Wi k d P

bl

14

Wicked Problems

"Wicked problem" is a phrase used in social planning to describe a problem that is difficult or impossible to solve because of incomplete, contradictory, and changing

requirements that are often difficult to recognize.

Moreover because of complex interdependencies the Moreover, because of complex interdependencies, the effort to solve one aspect of a wicked problem may

reveal or create other problems.p

F t

P

f I f

t

t

15

Future Proof Infrastructure

ƒ Thin Provisioning

ƒ Virtualization (VMWare, LPAR etc.)Virtualization (VMWare, LPAR etc.)

ƒ De-dup

ƒ RedundancyRedundancy

ƒ In place scale-out and scale up

ƒ Aim for zero outage upgrades & changes

B ildi

M

bl A

li

ti

16

Building Manageable Applications

ƒ Use established industry standards, where possible

ƒ SOA Architecture & XML MessagesSOA Architecture & XML Messages

ƒ Use Message Queuing

ƒ Implement solutions that are Horizontal & VerticallyImplement solutions that are Horizontal & Vertically scalable

ƒ Break into small manageable componentsg p

B

i

C

ti

it

17

Business Continuity

ƒ Consider sharing with pre-production environment

ƒ DR hardware can used as a pre-production environment a d a e ca used as a p e p oduc o e o e for testing and production support.

ƒ Visualize with VMWare / LPAR

ƒ Implement replication as high up the stack as possible

ƒ Ie. Oracle Data-guard, SQL Server Replication

ƒ Can be implemented (or tested) in stages with minimum

BCP / DR

Cl

i A

h

BCP / DR – Classic Approach

BCP / DR

Cl

i A

h

BCP / DR – Classic Approach

BCP / DR Fi

G

i

d

20

BCP / DR Fine Grained

Kiwibank uses a “Fine Grained” DR strategy

ƒ Any system can be separately DR’edy y p y

ƒ Separate DR procedure for each component

Kiwibank has future plans to use active-active where ibl

BCP / DR

Fi

G

i

d

BCP / DR – Fine Grained

BCP / DR

Fi

G

i

d

BCP / DR – Fine Grained

BCP / DR

Fi

G

i

d

BCP / DR – Fine Grained

Ki ib

k C

t & F t

24

Kiwibank Current & Future

ƒ Has completed several upgrades due to customer growth

ƒ Agile and able to react to change

ƒ Significant investment in IBM x-Series and p-Series hardware

S

25

Summary

Understand your applications Think Smarter, not Harder

Maximize the use of your assets

Adopt new technology and use it to grow your business Minimize outages and Customer Impact

26

Th

k

Thank you

Questions?

Questions?