Security OperaCons with Splunk App for Enterprise Security

(1)

David Casey, Vice President,

IT Security OperaCons Manager Flagstar Bank

Security OperaCons

with Splunk App for

Enterprise Security

(2)

Disclaimer

During the course of this presentaCon, we may make forward-‐looking statements regarding future events or the expected performance of the company. We cauCon you that such statements reﬂect our current expectaCons and

esCmates based on factors currently known to us and that actual events or results could differ materially. For important factors that may cause actual results to differ from those contained in our forward-‐looking statements, please review our filings with the SEC. The forward-‐looking statements made in the this presentaCon are being made as

of the Cme and date of its live presentaCon. If reviewed aSer its live presentaCon, this presentaCon may not contain current or accurate informaCon. We do not assume any obligaCon to update any forward-‐looking statements we may make. In addiCon, any informaCon about our roadmap outlines our general product direcCon and is subject to change

at any Cme without noCce. It is for informaConal purposes only, and shall not be incorporated into any contract or other commitment. Splunk undertakes no obligaCon either to develop the features or funcConality described or to

(3)

Personal Background

!   CISSP/CISM/SnortCP/Dr.Evil

!   Joined Flagstar Bank in early 2013 !   15+ Yrs – IT Security

!   18 Yrs – U.S. Army Military

Intelligence

Experience in the following sectors:

!   DoD (Lockheed MarCn, NCI, SAIC) !   EducaCon

!   Energy !   Finance

Specialize in building Security OperaCons programs from the ground up and major security ops overhauls due to compliance failures

(4)

Company Background

Flagstar Bank

!  

Full-‐service bank (Troy, Michigan)

!  

$9.4 billion in total assets

!  

100 + Branches in Michigan

!  

39 Home loan centers in 19 states

!  

NaConwide mortgage lender

(5)

In the Beginning There was Darkness…

!   2009-‐2012 Flagstar expanded

business operaCons very fast

!   Infrastructure changes, mind sets,

technology could not keep pace

!   As Flagstar bank grew federal

oversight shiSed

!   New auditors were assessing security

in the same manner as the Chase’s and Bank of America’s

!   Flagstar had many, many audit

ﬁndings

!   People, processes, and technology

had to change

!   IT Security OperaCons grew

signiﬁcantly in order to meet compliance requirements eﬀecCvely

!   A SIEM was a criCcal component !   The ‘One Ring’ to rule them all!

(6)

SIEM Technology Decision

When looking for a SIEM solu7on for Flagstar leveraged 12+ years of SIEM deployment experience as its guide.

Lessons learned:

•  Diﬃcult geong data in (ingesCng data)

•  Hard to get clear results from ad-‐hoc queries

•  Limited plaporm opCons

•  Costly to operate/maintain

•  Inﬂexible

•  SIEM sales hype. Product vendors only want to

sell you their product. No interest in truly helping you protect your organizaCon.

Splunk Experience…

ü  Easy to get all machine data

into the system

ü  Simple plain language search ü  Uses commodity hardware ü  IntuiCve, easy to use

ü  Flexible and easy to customize ü  They actually want you to be

successful and take great strides to make it so!

(7)

Splunk Deployment

Current Design:

!  2 Search heads, 3 indexers, 300+ GB/day !  Data sources (current)

–  All servers via forwarders – Windows, 4 ﬂavors of UNIX

–  All networking devices (switch, router, wireless, VPN, etc.)

–  Syslog systems

–  Firewall, IPS, DLP, AnC-‐Virus

–  Web proxy logs

–  DNS, DHCP

–  eMail

Applica7ons

–  Splunk for Windows apps (3) –  Splunk for UNIX app

–  Various vendor security apps (<10) –  DBConnect

(8)

Splunk Deployment

Disaster Recovery (DR) Design:

!   Overall Splunk ecosystem managed from HQ site !   2 Search heads, 2 indexers

!   DR site forwards all logs to HQ site

!   HQ replicates last 72 hours of logs to DR

Future State:

!   All data 100% replicated

!   Heavy Forwarders Deployed to both HQ

and DR sites

(9)

Security OperaCons Monitoring Challenges

Some7mes security technology is simply not enough… it takes a

human to help it all make sense

!  

The cyber security threat landscape is constantly morphing, ever

changing, with threat actors intent on by-‐passing common security

controls that rely on known payerns and detecCon techniques

!  

Humans are primarily a visual-‐based species

!  

Splunk can provide a visual that “speaks a thousand words” by

taking the complex and making it simple to understand

(10)

Case Study #1 – Are We Being Targeted?

!   Flagstar’s IT Security OperaCons Team uses the Splunk Enterprise Security

(ES) App to monitor for advance threats, including exploits, malware

infecCons, monitoring blacklists, and responding to spikes in threat trends

!   One common threat gathering technique is ﬁnger prinCng/mapping out a

targets public facing systems, its ports and services

!   Being “scanned” is very common and generally considered background

noise… just a part of doing business on the internet

!   But when the scan is coming from a country that is frequently a hosCle

cyber threat, and the scan is performed slowly, non-‐aggressively, it can oSen by-‐pass security controls that are designed to block more

(11)

Case Study #1 – Are We Being Targeted?

!   Sample Splunk search:sourcetype="[hidden]" earliest=-‐1m inbound | geoip

src | search src_country_code!=US | stats count AS count by src_country_name | sort -‐ count top limit=5

(12)

Case Study #1 – Are We Being Targeted?

!  

Upon closer inspecCon we

were able to isolate the

scans as originaCng from

the City of Nanning, China

!  

We have no legiCmate

customers in China…

!  

Answer?

(13)

Case Study #1 –

Con$nued

There are many hosCle actors all over the world. Some of the top actors are Russia, Ukraine, and China. Take Russia for example. Sure seems like there are a lot of outgoing connecCons to a Russian IP address. Could this be a compromised host?

Using Splunk we can watch closely outbound desCnaCons, by IP locaCon, and respond more quickly when we see an increase in potenCally risky IP traﬃc to known hosCle actor countries.

(14)

Case Study #1 –

Con$nued

!  

Sample Splunk search (Russia Inbound): sourcetype=“[hidden]"

src_ip!=“[internal networks excluded]" | iplocaCon src_ip | search

Country="Russia" | where Country="Russia" | chart count by src_ip

| sort -‐ count top limit=5

!  

Sample Splunk search (Russia Outbound): sourcetype=“[hidden]"

src_ip!=[exclude DNS server IP, web proxies, etc.] " | iplocaCon

dest_ip | search Country="Russia" | where Country="Russia" |

stats count by src_ip,dest_ip | rename src_ip AS "Client" dest_ip

AS "Russia IP Address" count AS "Count" | table Client,"Russia IP

Address",Count | sort -‐ count by Count top limit=5

(15)

Case Study #2 – Firewall Control AyestaCon

!   Flagstar’s IT Security OperaCons Team uses the Splunk Enterprise Security (ES)

app to help meet regulatory requirements (for IT Security Dept. only… at this Cme)

!   One example was where federal auditors wanted to see where changes to the

perimeter ﬁrewall were being monitored against approved ﬁrewall changes

!   If a change occurred outside of the change control process it should be noted

and invesCgated

!   Splunk was used to help idenCfy all ‘write’ and ‘execute’ commands issued on

the perimeter ﬁrewalls and graphically displayed for easy idenCﬁcaCon

!   This soluCon was accepted by the federal auditors

(16)

Case Study #2 – Firewall Control AyestaCon

!   Sample Splunk search: evenyype=“[hidden]_privileged_acCvity" "write" OR

"111010" OR "101008" NOT ("Teardown" OR "connecCon" OR "exit" OR "ping" OR “[hidden]") | Cmechart span=15m count(host) by user | sort _Cme

(17)

Case Study #3 – Metrics Across Security Technologies

!  

Flagstar’s IT Security OperaCons Team uses the Splunk Enterprise

Security (ES) App to track security metrics

!  

Security metrics are commonly requested as proof that the $$$

invested in security technology is actually producing results

!  

Rather then running separate reports from each security

technology to determine the “metrics”, using Splunk simpliﬁed the

process greatly

(18)

Case Study #3 – Metrics Across Security Technologies

!  

Sample Splunk search (IPS): index=

[hidden]

sourcetype=

[hidden]

(19)

Case Study #4 – 24 x 7 Monitoring

!   Flagstar’s IT Security OperaCons Team uses the Splunk Enterprise Security (ES) app

to provide 24x7 monitoring

!   Instead of spending $$$ on an external Managed Security Services provider that

provides aSer hours support, Splunk can be used to develop acConable dashboards monitored by the internal Network OperaCons Support Team (which works 24x7)

!   PotenCal savings can go towards other criCcal security budget items

NOTE: This case study is currently being developed and tested within Flagstar. It has not yet reached a point where it is ready to replace an external MSS provider

(20)

Case Study #4 – 24x7 Monitoring

(21)

Case Study #5

!   Flagstar’s IT Security

OperaCons Team uses the Splunk Enterprise Security (ES) app to detect `Brute Force Login Ayempts’ and send automated alerts in real-‐Cme when

(22)

Case Study #6

!   Flagstar’s IT

Security

OperaCons Team uses the Splunk Enterprise Security (ES) app to detect `Malware

InfecCons’ and send automated alerts in real-‐Cme when detected

(23)

Case Study #5 & 6

!   Sample Splunk search (Brute Force Ayempt Email Alert): EventCode=4625

sourcetype="WinEventLog:Security" earliest=-‐6m latest=now | bucket

_Cme span=5m | stats count by _Cme, Account_Name, src_ip,dest | where count > 500

!   Sample Splunk search (Malware Email Alert): index=[hidden]

sourcetype=[hidden] NOT ("Actual acCon: Cleaned*" OR "Actual acCon: QuaranCned" OR "Actual acCon: Deleted") | rename "event_Cme" as "Detected" actual_acCon as "AcCon" dest_nt_host as "Host" dest_ip as "Host IP" user as "User" risk_type as "DetecCon Type" signature as

"Malware Name" | table "Detected" "Host" "Host IP" "User" "DetecCon Type" "Malware Name" "AcCon" | sort by "Detected"

Security OperaCons with Splunk App for Enterprise Security