Contingency Planning
Contingency Planning
for Senior Management
for Senior Management
What you need to know about
What you need to know about
your business recovery
Agenda
Agenda
Current Regulatory EnvironmentCurrent Regulatory Environment
Risk Management Risk Management
What is Contingency PlanningWhat is Contingency Planning
Components of a solid recovery programComponents of a solid recovery program
Lessons from Sept 11Lessons from Sept 11
Questions to ask your team before the disaster Questions to ask your team before the disaster happens
happens
Eleven Steps to a Business Contingency PlanEleven Steps to a Business Contingency Plan
Current Regulatory Environment
Current Regulatory Environment
Interagency White PaperInteragency White Paper
Privacy regulationsPrivacy regulations
Sarbanes OxleySarbanes Oxley
Corporate GovernanceCorporate Governance
Prudent Man RulePrudent Man Rule
More to follow
More to follow
-
-
The rules will not get any
The rules will not get any
easier to comply with
Governance
Governance
Governance: regulators, corporate governance requirements – Turnbull, Sarbanes-Oxley
• Typically: CEO or Board must certify that risks are understood and under control
• Requirements:
• aggregate reporting from all areas • good lines of communication
Governance
Governance
A concise summary of your key risks A concise summary of your key risks
A common vocabulary to discuss risk A common vocabulary to discuss risk
A means to have productive discussions with your team A means to have productive discussions with your team
A roadmap to help you in your business planning A roadmap to help you in your business planning
A set of actions to help improve your risk management A set of actions to help improve your risk management process
Risk Management Program
Risk Management Program
Three components of RiskThree components of Risk
ThreatsThreats
AssetsAssets
Elements of Risk
Elements of Risk
Threats
Threats
--
Events or situations which would cause financial or Events or situations which would cause financial or
operational impact to the organization.
operational impact to the organization.
Threats are measured in probabilities, such as “may Threats are measured in probabilities, such as “may
occur 1 time in 10 years”.
occur 1 time in 10 years”.
Each threat has a duration of time that the business Each threat has a duration of time that the business
or operation would not be able to function in it’s
or operation would not be able to function in it’s
normal manner, if at all
Elements of Risk
Elements of Risk
Assets
Assets
-
- Assets are composed of many elements Assets are composed of many elements
Physical assets that are owned by the organizationPhysical assets that are owned by the organization
Information assetsInformation assets
Financial assets Financial assets
Revenues lost for the duration of the incidentRevenues lost for the duration of the incident
Additional costs to recoverAdditional costs to recover
Fines and penalties incurredFines and penalties incurred
Elements of Risk
Elements of Risk
Mitigating Factors
Mitigating Factors
--
Mitigating factors are the protection devices, Mitigating factors are the protection devices,
safeguards, and procedures which are in place
safeguards, and procedures which are in place
that reduce the effects of the threats.
that reduce the effects of the threats.
They do not reduce the threat; they only
They do not reduce the threat; they only
reduce the effect of the threat.
reduce the effect of the threat.
Examples of mitigating factors in use are UPS (
Examples of mitigating factors in use are UPS (UninterruptableUninterruptable Power Supply) and Generator backups for replacement power,
Power Supply) and Generator backups for replacement power,
sprinkler systems to control the spread of fire, Assess Card
sprinkler systems to control the spread of fire, Assess Card
Readers to control physical access to Fidelity space, etc....
Risk Mitigation Strategies
Risk Mitigation Strategies
Protecting People and Workspaces
Protecting People and Workspaces
Protecting Information
Protecting Information
Protecting Reputation
Protecting People and Workspaces
Protecting People and Workspaces
Access ControlAccess Control
Alarm MonitoringAlarm Monitoring
Floor WardenFloor Warden
Evacuation DrillsEvacuation Drills
Background InvestigationsBackground Investigations
Landscape DesignLandscape Design
LightingLighting
CamerasCameras
Visitor proceduresVisitor procedures
Backup Power systemsBackup Power systems
Facility designFacility design
Protecting Information
Protecting Information
Information Security policy and proceduresInformation Security policy and procedures
Privacy PolicyPrivacy Policy
FirewallsFirewalls
Intrusion DetectionIntrusion Detection
Strong PasswordsStrong Passwords
Controlling access to informationControlling access to information
Vendor ManagementVendor Management
Secure offsite storageSecure offsite storage
Proprietary Waste DisposalProprietary Waste Disposal
Protecting Reputation
Protecting Reputation
Strong GovernanceStrong Governance
Media trained Media trained
Communication PlansCommunication Plans
Internal and external auditsInternal and external audits
Operational ManagementOperational Management
RecoverabilityRecoverability
Financial
Strategic Organizational Technology Operational Legal/Regulatory Risk Definitions
Market Credit
Liquidity, Cap
& Funding People Process Events
Risks associated with the use of systems and technology, including availability, capacity integrity, operational support, functionality systems integration and change manage-ment
Risks that are an inherent part of the business environment and have an effect on business objectives and performance
Risks that are part of a unit’s environment
relating to people, culture, organizational structure and values
that can impact overall organization effectiveness Risks relating to enforceability of contracts, interpretation of laws, compliance with law and impact
of regulation
Inability to raise debt or equity capital as
needed for short-term liquidity or long-term growth, as well as uncertainty in pricing or sales of assets or liabilities Exposure to loss relating to a change in the credit-worthiness of a counter-party, collateral, customer or partner that may impact the counter-party’s ability to fulfill its obligations under a contractual agreement
The uncertainty in the future market value of a portfolio
of assets and / or liabilities
The risk of loss resulting from
people
The risk of loss resulting from inadequate or failed processes
The risk of loss resulting from unique,
unusual or extraordinary
Event Management
Event Management
Contingency Plans are what we exercise when all other Contingency Plans are what we exercise when all other mitigating factors fail
mitigating factors fail
Contingencies start with Event ManagementContingencies start with Event Management
If you do not properly manage Events, all the other If you do not properly manage Events, all the other Risks may occur
Risks may occur
Event Management is about Communication and Event Management is about Communication and Response
Response
Event Management Requirements
Event Management Requirements
Strategy must be consistent regardless of eventStrategy must be consistent regardless of event
Need to establish an assessment processNeed to establish an assessment process
Event Ownership needs to be definedEvent Ownership needs to be defined
Management teams identifiedManagement teams identified
Response Teams identifiedResponse Teams identified
Process for gathering of key decision makersProcess for gathering of key decision makers
Event Management example
Event Management example
Facility event
Facility event
Built by BuildingBuilt by Building
Three teams :Three teams :
Assessment TeamAssessment Team
First Escalation TeamFirst Escalation Team
General TeamGeneral Team
Permanent standing conference bridge that always Permanent standing conference bridge that always has the same phone number
has the same phone number
Event owners definedEvent owners defined
Contingency Planning
Contingency Planning
If you are reading your plan for the first time and you are
If you are reading your plan for the first time and you are
in the middle of a disaster……….
in the middle of a disaster……….
You are in trouble
How ready is your business?
How ready is your business?
If you were evacuated from your building and you were If you were evacuated from your building and you were standing in the evacuation area and they announce that
standing in the evacuation area and they announce that
you could not work at that site for at least the next 2
you could not work at that site for at least the next 2
weeks,
weeks,
Do you know what to do next?
Do you know what to do next?
Does your staff?
What is a Contingency Plan
What is a Contingency Plan
The documented process for The documented process for
continuation/recovery of business functions in
continuation/recovery of business functions in
the event of an unexpected disruption of service.
the event of an unexpected disruption of service.
The plan describes the preThe plan describes the pre--planned sequence of planned sequence of events that allows for the continuation/recovery
events that allows for the continuation/recovery
of business functions, computer resources,
of business functions, computer resources,
networks, and facilities.
Components of a Solid Business
Components of a Solid Business
Continuity Program
Continuity Program
Deliverables
Deliverables Due dateDue date Emergency Notification List
Emergency Notification List QuarterlyQuarterly
Business Functions/ Resource Requirements
Business Functions/ Resource Requirements SemiSemi--AnnuallyAnnually
Business Resumption Plans with sign
Business Resumption Plans with sign--offoff AnnuallyAnnually
Training & Awareness
Training & Awareness QuarterlyQuarterly
Vital Records Program
Vital Records Program OnOn--goinggoing
Technology Reviews
Technology Reviews AnnuallyAnnually
Strategy for loss of site/systems
Strategy for loss of site/systems AnnuallyAnnually
Procedures for loss of site/systems
Procedures for loss of site/systems AnnuallyAnnually
Call Exercise
Call Exercise SemiSemi--AnnuallyAnnually
Walk
Walk--Through ExerciseThrough Exercise AnnuallyAnnually
Simulated Or Actual Exercise
Simulated Or Actual Exercise SemiSemi--AnnuallyAnnually
Compact Exercise
Compact Exercise AnnuallyAnnually
Systems Loss Test
Business Impact Analysis
Business Impact Analysis
The process used to identify what would happen if a risk The process used to identify what would happen if a risk occurred
occurred
The end result is to determine the Recovery Time The end result is to determine the Recovery Time
Objective (RTO) and the Recovery Point Objective (RPO)
Objective (RTO) and the Recovery Point Objective (RPO)
of all processes within your organization
of all processes within your organization
Includes technology and nonIncludes technology and non--technology functionstechnology functions
Results should be signed off by Senior Management as Results should be signed off by Senior Management as evidence of review
evidence of review
RTO and RPO drive the recovery strategies available for RTO and RPO drive the recovery strategies available for each business process to be recovered
Lessons from September 11
Lessons from September 11
The events of September 11th and the resulting The events of September 11th and the resulting business disruptions have highlighted the need for
business disruptions have highlighted the need for
companies to revisit the assumptions underlying their
companies to revisit the assumptions underlying their
disaster recovery and business continuity plans. Such
disaster recovery and business continuity plans. Such
plans have primarily focused on the loss of systems and
plans have primarily focused on the loss of systems and
information or the inability to access a main processing
information or the inability to access a main processing
facility. After September 11th, planning considerations
facility. After September 11th, planning considerations
have expanded to include:
have expanded to include:
loss of key employees or emotionallyloss of key employees or emotionally--impacted staff,impacted staff,
loss of access to major business districts,loss of access to major business districts,
longlong--term operation at backterm operation at back--up sites,up sites,
need for alternative backneed for alternative back--up sites,up sites,
availability of contact information for key employees,availability of contact information for key employees,
loss of paper documentation, andloss of paper documentation, and
How Close Were We?
How Close Were We?
Ariel View After the Attack
Another View
What We Learned from 9/11
What We Learned from 9/11
Testing was the key to the success of the recoveryTesting was the key to the success of the recovery
Critical operations in a single site are bad businessCritical operations in a single site are bad business
We don’t have problem by business, we have problems by We don’t have problem by business, we have problems by building
building
Transportation was a major issue in the first few daysTransportation was a major issue in the first few days
Incomplete/inaccurate inventories make the insurance claim Incomplete/inaccurate inventories make the insurance claim difficult
difficult
People do not want to travel away from their familiesPeople do not want to travel away from their families
Very few business operations stand aloneVery few business operations stand alone
Voice is harder than data to recoverVoice is harder than data to recover
Some of our vendors were in trouble tooSome of our vendors were in trouble too
We experienced no loss of life in New York and injuries were We experienced no loss of life in New York and injuries were not serious. All the required personnel were available.
not serious. All the required personnel were available.
The Full Market remained closed, allowing initial recovery The Full Market remained closed, allowing initial recovery efforts to be augmented and the business to prepare to
efforts to be augmented and the business to prepare to
conduct business for a long period of time in the alternate
conduct business for a long period of time in the alternate
sites
sites
TwoTwo--way pagers. They worked consistently when other way pagers. They worked consistently when other forms of communication were either busy or completely
forms of communication were either busy or completely
unavailable.
unavailable.
Buildout of all of the alternate sites occurred very quickly to Buildout of all of the alternate sites occurred very quickly to allow critical business functions to resume
allow critical business functions to resume
Bench Strength Bench Strength -- All of the people involved exhibited All of the people involved exhibited
teamwork, flexibility, availability and an excellent attitude.
teamwork, flexibility, availability and an excellent attitude.
Many volunteered to work longer hours and additional shifts
Many volunteered to work longer hours and additional shifts
to get the job done.
to get the job done.
The ability to use alternate network resources to get the New The ability to use alternate network resources to get the New York Operations back online quickly and to provide
York Operations back online quickly and to provide
redundancy for network lines running in backup mode.
redundancy for network lines running in backup mode.
The Good News
Planning Assumptions to re
Planning Assumptions to re
-
-
think
think
Assume only one disaster strikes at the same timeAssume only one disaster strikes at the same time
We lost access to WFC and simultaneously lost access We lost access to WFC and simultaneously lost access
to key buildings in Boston that were evacuated as a
to key buildings in Boston that were evacuated as a
precaution
precaution
This led to multiple disaster declarations in diverse This led to multiple disaster declarations in diverse
locations that had to staffed at the same time by
locations that had to staffed at the same time by
multiple support groups
multiple support groups
Assume infrastructure required for recovery is in Assume infrastructure required for recovery is in
place
place
Telecommunications, power and transportation were all Telecommunications, power and transportation were all
impacted. No one had ever imagined a scenario where
impacted. No one had ever imagined a scenario where
all the planes in the country would be unavailable
Planning Assumptions to re
Planning Assumptions to re
-
-
think
think
Assume your disaster recovery team and the rest of Assume your disaster recovery team and the rest of
the corporation survive the attack
the corporation survive the attack
We was unaffected by this, but other New York based We was unaffected by this, but other New York based
corporations lost entire recovery teams and the
corporations lost entire recovery teams and the
documentation required to recover
documentation required to recover
Other corporations are struggling to do required day to Other corporations are struggling to do required day to
day business functions because those responsible died
day business functions because those responsible died
in the event and the training materials for the position
in the event and the training materials for the position
were stored in the building
were stored in the building
Assume the ability to get required equipment from Assume the ability to get required equipment from
your vendors very quickly
your vendors very quickly
This did not impact We, but the drop in the economy This did not impact We, but the drop in the economy
has left many vendors with little or no inventory. The
has left many vendors with little or no inventory. The
ability to obtain obtain required equipment quickly was
ability to obtain obtain required equipment quickly was
hampered.
Planning Assumptions to re
Planning Assumptions to re
-
-
think
think
The disaster recovery plan should be built for a The disaster recovery plan should be built for a
short interruption in business and only for the
short interruption in business and only for the
data center, not a long term disaster
data center, not a long term disaster
This type of planning assumption led many business This type of planning assumption led many business
units to assume that plans only needed to be done for
units to assume that plans only needed to be done for
very small numbers of employees or only for their
very small numbers of employees or only for their
technology infrastructure. This led to scrambling
technology infrastructure. This led to scrambling
during a disaster and not necessarily the best
during a disaster and not necessarily the best
recovery plan for the employees involved. Recovery is
recovery plan for the employees involved. Recovery is
for the whole business.
Questions to Ask Your Team BEFORE
Questions to Ask Your Team BEFORE
the Disaster
the Disaster
Are we recoverable or just “green”?Are we recoverable or just “green”?
What is our Recovery Time Objective?What is our Recovery Time Objective?
What is our Recovery Point Objective?What is our Recovery Point Objective?
Are we prepared for “loss of people”, not just loss of Are we prepared for “loss of people”, not just loss of site?
site?
Are we prepared for losing a critical application?Are we prepared for losing a critical application?
Where is the alternate site?Where is the alternate site?
How will you communicate during the event?How will you communicate during the event?
When did you last test?When did you last test?
Have we identified our critical vendors and do we know Have we identified our critical vendors and do we know what their recovery plans are?
Don’t get caught without a plan
Eleven Steps to having a Contingency
Eleven Steps to having a Contingency
Plan for your business
Plan for your business
Follow these steps to a solid recovery
Follow these steps to a solid recovery
program for your business
Step 1
Step 1
Identify Business Recovery team
Identify Business Recovery team
•
• Identify your team and make certain they Identify your team and make certain they
know how to reach you in an emergency
Step 2
Step 2
Identify business vital records
Identify business vital records
•
• Identify vital recordsIdentify vital records
Procedure manualsProcedure manuals
formsforms
vendor listsvendor lists
contact listscontact lists
customer listscustomer lists
contractscontracts
Step 3
Step 3
Identify Business Functions
Identify Business Functions
•
• Identify the business functions for functional Identify the business functions for functional
areas
areas •
• Perform risk and business impact analysis for Perform risk and business impact analysis for
each function
each function •
• Establish the recovery time for your business Establish the recovery time for your business
functions
functions •
• Identify minimum staff requirementsIdentify minimum staff requirements
•
Step 4
Step 4
Identify desktop requirements
Identify desktop requirements
•
• Minimum desktop configurationMinimum desktop configuration
•
• Application connectivityApplication connectivity
•
• Voice RequirementsVoice Requirements
phonesphones FaxFax ModemsModems •
• Print RequirementsPrint Requirements
•
Step 5
Step 5
Define Recovery Strategy
Define Recovery Strategy
•
•
Develop recovery strategy for
Develop recovery strategy for
business functions based on the
business functions based on the
recovery priority
Selecting the Right Recovery Strategy for
Selecting the Right Recovery Strategy for
your business
your business
Recovery strategies will be driven by the recovery
Recovery strategies will be driven by the recovery
timeframe of the function. Recovery options might
timeframe of the function. Recovery options might
include the following:
include the following:
SelfSelf--service service -- A business unit can transfer work to another of its own locatioA business unit can transfer work to another of its own locations ns
which have available facilities
which have available facilities
Internal Arrangement Internal Arrangement -- Training rooms, cafeterias, conference rooms, etc.... Training rooms, cafeterias, conference rooms, etc....
may be equipped to support business functions.
may be equipped to support business functions.
Reciprocal Agreements Reciprocal Agreements -- Other business units may be able to accommodate Other business units may be able to accommodate
those affected. This could involved the temporary suspension of
those affected. This could involved the temporary suspension of nonnon--critical critical functions at the business units not affected by the outage.
functions at the business units not affected by the outage.
Dedicated alternate sites Dedicated alternate sites -- Built by your company to accommodate critical Built by your company to accommodate critical
function recovery.
function recovery.
External Suppliers External Suppliers -- A number of external companies offer facilities covering A number of external companies offer facilities covering
a wide range of business recovery needs.
a wide range of business recovery needs.
No arrangement No arrangement -- for low priority business functions it may not be cost for low priority business functions it may not be cost
justified to plan to a detailed level. The minimum requirement w
justified to plan to a detailed level. The minimum requirement would be to ould be to record a description of the functions, the maximum allowable lap
record a description of the functions, the maximum allowable lapse time for se time for recover, and a list of the resources required.
Step 6
Step 6
Internal Site Survey
Internal Site Survey
•
•
Survey existing sites
Survey existing sites
•
•
Identify equipment/phone services
Identify equipment/phone services
•
•
Identify desktops to be used for
Identify desktops to be used for
contingency
contingency
•
•
Identify staff to be displaced or moved to
Identify staff to be displaced or moved to
off shift
off shift
Step 7
Step 7
External Site Recovery
External Site Recovery
Prepare RFP which includes all
Prepare RFP which includes all
requirements
requirements
Identify essential vs. “nice to have”
Identify essential vs. “nice to have”
Receive proposals from vendors
Receive proposals from vendors
Compare for requirements and costs
Compare for requirements and costs
Visit sites identified as potential vendors
Visit sites identified as potential vendors
Step 8
Step 8
Internal Systems
Internal Systems
•
•
Identify all platforms and applications
Identify all platforms and applications
supported by internal systems group
supported by internal systems group
•
•
Identify recovery priority for each
Identify recovery priority for each
application
application
•
•
Identify recovery strategy which meets the
Identify recovery strategy which meets the
business requirements
business requirements
•
•
Develop recovery procedures for critical
Develop recovery procedures for critical
applications
applications
Step 9
Step 9
Document Plan
Document Plan
•
•
Pull the information together into a plan
Pull the information together into a plan
document and distribute
document and distribute
Step 10
Step 10
Train staff
Train staff
Everyone should know the answer to the question :
Everyone should know the answer to the question :
If you couldn’t get back in your building today,
If you couldn’t get back in your building today,
what would you do next?
what would you do next?
Step 11
Step 11
TEST, TEST, TEST
TEST, TEST, TEST
•
• Event Management testsEvent Management tests
•
• Alternate site testsAlternate site tests
Don’t be the one taken by storm!!
Websites
Websites
Industry Group WebsitesIndustry Group Websites
DRI International DRI International www.drii.org www.drii.org Continuity Insights Continuity Insights www.continuityinsights.com/conf.cfm www.continuityinsights.com/conf.cfm Contingency planning and Management
Contingency planning and Management www.contingencyplanning.com
www.contingencyplanning.com Disaster Recovery
Disaster Recovery JounalJounal
www.drj.com
www.drj.com//
Global Association of Risk Professionals (GARP)
Global Association of Risk Professionals (GARP)
www.garp.com
www.garp.com
Professional Risk Managers International Association (PRMIA
Professional Risk Managers International Association (PRMIA))
www.prmia.org
www.prmia.org
Institute of Internal Auditors
Institute of Internal Auditors
www.theiia.org