Using Security Metrics Coupled with
Predictive Modeling and Simulation to
Assess Security Processes
© 2008 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice
Assess Security Processes
Yolanta Beres, Marco Casassa Mont, Jonathan Griffin, Simon Shiu
Systems Security Lab
•
How does and how should an enterprise form and
justify its security decisions and investments?
−
How well are our processes protecting us?
−
Even in the face of imperfect or patchy data
−
How well will we be protected if the threat level changes?
(and it’s changing all the time)
Security Investment & Decision Making
(and it’s changing all the time)
−
How much difference will it make to our overall level of
protection if we add extra security measures?
−
How do we get appropriate input and buy-in from all the
stakeholders?
•
Let’s start with an easy example... Software
vulnerabilities.
15 October, 2009 2
Problem Statement
•
How do we evaluate the effectiveness of our
vulnerability management processes and policies?
−
when we have a combination of protections and
processes: patch management, AV, HIPS, emergency
escalation, temporary workarounds
3 15 October, 2009
•
How do we estimate in advance the impact on
overall protection of a change in policy or the
addition of a new security mechanism?
The Solution: Build a Model
•
Stochastic model of threat environment
•
Process model of organization’s protections
•
Validate with experts and against known data sources
•
Select metric(s)
−
For software vulnerabilities: Time until “risk mitigated”
4
−
For software vulnerabilities: Time until “risk mitigated”
•
Execute the model as a discrete event simulation
−
Enough for statistically significant results (100K vulnerabilities
should do the trick)
•
Now adjust the model to reflect proposed changes in
policy and see how well they perform
Vulnerability related Timeline
Discovery
Disclosure
Mitigation Deployed
5 15 October, 2009
Risk Reduced Windows
Discovery
Disclosure
Cannot be
measured
Some
public
data
A lot of public
data
Patching
processes
Minus zero
day exploit
Public exploit
code
Patch Available
Patch Deployed
Only some groups aware,
no data yet
Model: BAU Patch Deployment
Vulnerability
Disclosed
Exploit
Available
Malware
Patch
Available
likelihood: 0.97 mean delay: 23 days,exponential distribution likelihood: 0.84
mean delay: 25 days, exponential distribution
likelihood: 0.72 mean delay: 5 days,
exponential distribution
duration: 0.5–1.5 days,
uniform distribution
Vulnerability
Assessment
Test
Solution
Patch
Deployment
duration: 10–20 days, uniform distributionmean time to target: 136 days, take-up curve:
Vulnerability
Disclosed
Exploit
Available
Malware
Patch
Available
Test
Solution
Vulnerability
Assessment
Model: Vulnerability Management
Exposed?
Early
Y
Malware
Reports?
N
N
Y
Patch
Deployment
Accelerated
Patching
Emergency
Patching
Early
Mitigation?
Accelerate?
Patch
Available?
Workaround
Available?
Implement
Workaround
Y
N
Y
Y
Deploy
Mitigation
Y
Risk reduced window (from disclosure time) across all vulnerabilities
0.15
0.2
0.25
0.3
0.35
P
ro
p
o
rt
io
n
o
f
v
u
ln
e
ra
b
il
it
ie
s
Risk Reduced Window Overall
Policy dictated deadline
8 15 October, 2009
0
0.05
0.1
timeline
P
ro
p
o
rt
io
n
o
f
v
u
ln
e
ra
b
il
it
ie
s
Proportion mitigated
in early days
Proportion not
mitigated after the
policy deadline
Exploring
Investment Options
Investment in HIPS
Current situation
policy
Investment in patch automation
policy
Case Study:
User Account Provisioning
•
Provisioning Management deals with Managing User Accounts and Setting and
Removing Permissions/Rights
•
Poor User Provisioning could:
−
Give more rights than necessary to users
−
Prevent users from accessing legitimate resources
•
The Provisioning could be subject to various failures due to:
−
User and Administrators’ Misbehaviours
−
Cultural Attitudes
−
IT and Solutions Failures
−
Attacks …
•
Provisioning of User Accounts can be carried out with different levels of
Automation:
−
Ad-hoc processes
User Provisioning:
Policy Decision Makers
•
The CIO or CISO is likely to define the policies
•
However policy analysis and decisions require inputs and consent
(buy-in) from several stakeholders, who have different priorities and
concerns
•
A set of
High-level Security Metrics
has been identified, relevant to
•
A set of
High-level Security Metrics
has been identified, relevant to
the different stakeholders
Stakeholder
Metrics
Security/Compliance Officers:
•
Access Accuracy
•
Approval Accuracy
Application Owner (Business)
•
Productivity Cost
IT Operations (IT Budget Holder)
•
IAM Provisioning Costs
Metrics
Formula
Description
Access
Accuracy
1-(w1*UAD+w2*UAM+w3*UAH)/ (UAA)
w1, w2, w3 are relevance weights in the [0,1] range, UAD is the number of denied user accounts, UAM is the number of misconfigured user accounts, UAH is the number of hanging user accounts and UAA is the overall number of user account provisioned (for which either there has been approval or the approval process has been bypassed);Approval
Accuracy
#Approved_Provisioning /
(#Approved_Provisioning + # Bypassed_Approvals)
User Provisioning:
Identifying Security Metrics
Accuracy
(#Approved_Provisioning + # Bypassed_Approvals)
Productivity
Costs
[(join_appr_time+ change_appr_time) + (join_prov_time +
change_prov_time)] *
Unit_cost_per_day
+ [(#loss_join_appr +
#loss_join_prov) + (#loss_change_appr+#loss_change_prov)]
*
Unit_cost_lost.
takes into account loss of productivity due to waiting time (for the approval and deployment phases) and for lost of approval and deployment activities. The impact of these costs are weighted by constants for “unit cost per day” and “unit cost per loss”.
IAM
Automation
Cost
Fixed_Costs + Variable_Costs*Num_IAM_Automated_Apps
Estimated costs of running automated IAM provisioning processes, depending of fixed costs (e.g. fixed yearly fee) and variable costs (e.g. additional license fees depending on the number of provisioned applications)IAM Effort
#
IAM_automated_provisioning_activities
Ad-hoc Effort
User Joins User User Leaves Changes Role
Events
For each affected:
User Profile - Role- Set of req. Apps - Location/Region
App Profile
- ad-hoc/centrally managed - Admin Location/Region - Entitle mgmt team & profile - Available IAM Controls
User Profile
- Role
- Set of req. Apps - Location/Region
User Profile
- Roles
- Set of req. Apps - Location/Region
For each
Application/Service Profiles
- ad-hoc/centrally managed - Admin Location/Region
- Provisioning mgmt team & profile - Available IAM Controls
Types of Changes on Affected apps? “Leaving” Application/Service Profiles - ad-hoc/centrally managed - Admin Location/Region
- Provisioning mgmt team & profile - Available IAM Controls
Provisioning Model: Details
Application
:
For each affected Application: “Joining” For each affected Application: “Changing”User
Joining:
IAM Provisioning
Management
Process
User
Changing Role:
IAM Provisioning
Management
Process
User
Leaving:
IAM Provisioning
Management
Process
Request for each affected Application: Prob. Loss Approval Request? Measure: # Lost Approval Requests (Denied Access) YES YES Application Profile - ad-hoc/centrally managed - Admin Location/Region
- Provisioning mgmt team & profile - Available IAM Controls
User Joining: Provisioning Management Process
Dependency on:
- regional/local attitudes
- presence of automation (e.g.
notification workflow)
Dependency on:
- regional/local attitudes
- available resources (admin, mgmt).
- presence of automation (e.g.
IAM provisioning solution)
- type of applications
Carry on,
Provisioning Model: Details
Waiting time To Process Approval
Request
Measure: User Joins - time to get
Approval Request? Waiting time To Deploy/COnfig Measure: time to deploy (conf. account) Prob. Loss Deployment Activity? NO NO (Denied Access) YE S Prob. Misconfig? Measure: #Misconfigured Account YE S YES Measure: #Lost Deployment Activities NO YES
Dependency on:
- regional/local attitudes
- available resources
- presence of IAM automation:
provisioning & deployment
Dependency on:
- regional/local attitudes
- available resources
- presence of IAM automation:
provisioning & deployment
Dependency on:
- regional/local attitudes
available resources
- presence of IAM automation:
provisioning & deployment
Carry on, without auth.
