Tyson Jarrett
About Me
•
Master’s Degree Information Systems – Cyber
Security
•
Reviewed 1562 CIP CMEP items
•
CIP Analyst 4 years
•
CISA certified
•
New Marathon and Half Marathon
Personal Records!!
•
Half: 1:57:36
•
Full: 4:56:48
Agenda
•
Why is Security Testing
Important?
•
Overview of Security Testing
•
Best Practices on
Conducting Security Testing
Some Causes for Authentication
servers not tested
Device configurations changed and only tested on
the primary site
Passwords changed without adequate testing done of impact
on functionality of critical programs
New program tested on test system, but not appropriately
tuned for production prior to implementing
Batch file directly installed on production system, causing communications
from the RTUs to stall
Due to inherent design issues in the code, critical services were
Benefits of Security Testing
•
Leads to more secure and reliable operations
•
Prevent introduction of vulnerabilities
•
Promotes a more reliable BES
•
Reduces likelihood of compliance issues with
other requirements
–
CIP 007 R2: Ports/Services
–
CIP 007 R4: Anti-Virus
–
CIP 007 R5: Accounts
Create testing procedures Determine what devices can have testing environment Determine Production test to minimize adverse effects Create test environment that reflects production environment Update test environment after production changes Test all significant changes Document test results
Create
testing
procedures
Test for
impacts to
security
controls
Document
test
results
Create Testing Procedures
•
Document when testing is needed (patch applied,
new device, new software/hardware)
•
How testing will be performed (test or production
environment)
•
The scope of testing (ports and services,
accounts, antivirus, logging)
–
Or repeatable process to determine existing security
controls potentially impacted
•
Additionally, test procedures should include
Common Testing Scope
Networking
equipment
Rule sets/ACLs Ports/services (on and
through) SNMP Accounts
Servers
Ports/Services Accounts Event Logs Malware Applications Functional testingRelays
Confirm if testing should be Automated/Manual
Configuration review Ports/Services Passwords/Default
Create a Test Case
•
Describes steps that need to be performed on the
device(s) to analyze the vulnerabilities identified in
the scope
o
Verify ports/services, accounts, and configuration settings
didn’t unexpectedly change. Confirm antivirus, and logging
are still working as intended
•
Have/Create a baseline for these controls
o
Gives the SME performing the tests a basis of comparison
Verifying Security Controls
•
Security testing should be sufficient to ensure
potentially impacted security controls are not
adversely affected
–
Can be a manual or automated process. Sometimes a
manual process is required due to sensitive devices
•
Testing should be conducted in a manner that
gathers raw data and can compare it to existing
baselines
•
Testing can be conducted in a Test or Production
Using a Test Environment
•
A test environment should be used if
possible
–
Have process to determine if test environment
reflects production
•
Have steps to identify and account for differences
–
Ensure test environment is updated along with
production as part of change control
–
Have clear documentation identifying where a
Test Environment
•
Test environment should model the baseline
configuration of Production
–
May have different set of components (devices)
•
Example: Database on one component and
web server on another component
–
Test environment may have both database and
webserver on same component; as long as
operating system, security patches, network
accessible ports and software are same
Testing in Production
•
At a minimum, take steps to minimize
adverse effects on the production
environment
–
Network segmentation
–
Scheduled outages
Testing in Production Vs. Test environments
Testing in Production
Pros
• Eliminates potential for unforeseen
events going unnoticed due to
differences in environments
• Initially more cost effective(don’t need
to duplicate equipment)
Cons
• May Increase exposure to vulnerabilities
• If vulnerability is introduced can cause
delays or damage even with strong
controls in place to minimize adverse
effects
• In depth testing can break or crash
Testing in Test Environment
Pros
• No/limited exposure to harming
production system during testing
• Vulnerabilities introduced to test
environment are much less expensive
and easier to fix
• Events/Issues during testing have less or
zero impact to reliability
Cons
• Increases likelihood of testing missing
certain vulnerabilities due to not being
identical
Document Security Testing
•
Require evidence to be captured as part of
testing practices
–
Should ideally be linked and/or captured in
Change Management System
•
There should be a review and approval of the
test results in order to move forward with the
Change (applying the patch and/or deploying
the new device)
–
Require peer reviews by independent SME’s prior
Quality of documentation
•
Ensure documentation adequately
demonstrates what was done during the test
phase
•
Documentation should include captured
evidence supporting testing was done
Baseline Example
C:\Documents and Settings\HMI-1>netstat -b -o -a -n > netstat_boan.txt
Active Connections
Proto Local Address Foreign Address State PID TCP 0.0.0.0:135 0.0.0.0:0 LISTENING 952 C:\WINDOWS\system32\svchost.exe TCP 0.0.0.0:445 0.0.0.0:0 LISTENING 4 [System] TCP 0.0.0.0:6002 0.0.0.0:0 LISTENING 428 [spnsrvnt.exe] TCP 0.0.0.0:7001 0.0.0.0:0 LISTENING 248 [sntlkeyssrvr.exe] TCP 0.0.0.0:7002 0.0.0.0:0 LISTENING 248 [sntlkeyssrvr.exe] TCP 127.0.0.1:1025 0.0.0.0:0 LISTENING 1656 [dirmngr.exe] TCP 127.0.0.1:1029 0.0.0.0:0 LISTENING 2484 [alg.exe] TCP 127.0.0.1:5152 0.0.0.0:0 LISTENING 1764 [jqs.exe] TCP 127.0.0.1:33333 0.0.0.0:0 LISTENING 1856 [PGPtray.exe] TCP 172.16.105.220:139 0.0.0.0:0 LISTENING 4 [System] TCP 127.0.0.1:2111 127.0.0.1:33333 ESTABLISHED 1616 UDP 0.0.0.0:7001 *:* 248 [sntlkeyssrvr.exe] UDP 0.0.0.0:500 *:* 700 [lsass.exe] UDP 0.0.0.0:4500 *:* 700 [lsass.exe] UDP 0.0.0.0:445 *:* 4 [System] UDP 127.0.0.1:123 *:* 1084 c:\windows\system32\WS2_32.dll UDP 172.16.105.220:6001 *:* 428 [spnsrvnt.exe]
Good Raw Data Example (ports/services)
root@bt# nmap -sT -sV -p T:0-65535 172.16.105.220
Starting Nmap 5.59BETA1 ( http://nmap.org ) at 2012-01-03 10:28 EST
Nmap scan report for 172.16.105.220
Host is up (0.00084s latency).
Not shown: 65528 closed ports
PORT STATE SERVICE VERSION
135/tcp open msrpc Microsoft Windows RPC
139/tcp open netbios-ssn
445/tcp open microsoft-ds Microsoft Windows XP microsoft-ds
777/tcp open multiling-http?
6002/tcp open http SafeNet Sentinel License Monitor httpd 7.3
7001/tcp open afs3-callback?
7002/tcp open http SafeNet Sentinel Keys License Monitor httpd 1.0 (Java
Console)
MAC Address: 00:0C:29:07:09:3B (VMware)
Service Info: Host: HMI-1; OS: Windows
Evidence supports date
testing was conducted
Post Testing (Best Practice)
•
After the change is implemented, there should
be an acceptance test to confirm that the
settings are still correct.
o
Primarily applies if the testing was done in a test
environment and test and production
Environments are different.
o
This can be a lighter test, to confirm the device is
working and security controls are functioning as
expected
Key Takeaways
•
Consider all factors and scenarios when
creating security testing procedures
•
Capture evidence to support that testing was
done per procedures
•
Where possible conduct post testing to
Create testing
procedures
•Identify Scope and create test case
•Have procedures for testing in both production and test environments
•If testing in production how will adverse effects be minimized?
•May involve scheduled outage
•Have change control procedures in place to ensure test environment
Test all changes
•Define what types of security testing needs to occur
•Ensure both security testing and functional testing are conducted
•Ensure proper measures are taken to minimize adverse effects prior to
implementation
•Outage, Test environment, network segmentation, etc.
Document test
results
•Checklist or automated solution that require test results to be documented •Ensure documentation
captured includes actual evidence of testing
•Gather and store raw data evidence to support testing •Usually baseline prior to
change and raw data after change comparison
