After the Attack: RSA's Security Operations Transformed

(1)

Ben Smith, CISSP RSA Field CTO (East), Security Portfolio

Senior Member, ISSA Northern Virginia

(2)

• ~ 2,000 security devices

• ~55M security events per hour

• ~60K employees

• 350 sites

• 85 countries

• Core intellectual property

(3)

• Eyes on Glass

• Analysis

• Forensic

• Coordination

• Remediation

• Rule/Report Creation

• Workflow Development

Structure of Pre-Breach CIRC

RSA CIRC

2009-2011

(4)

Technology Pre-Breach

RSA SIEM SharePoint File Servers Databases NAS/SAN Endpoints Data Discovery (RSA DLP) Security Operations (RSA Archer) Windows Clients/Server s Incident

Tracking Vulnerability Risk Management Firewall IPS Proxy AV Security Controls Log Analysis Reporting Event Forensics

(5)

The Initial Vector in the Attack

1

Two rounds of “phishing” emails

Some clues about the email lead us to believe that this was from some slightly dated research on employees

2

3

Attacker gains access to other machines

Zero-day exploit installs backdoor (Poison Ivy RAT Variant) which enables extraction of memory resident password hashes

Launch Zero-day attack

One user opened email attachment (an Excel spreadsheet) which launches a Flash zero-day

X

(6)

From Compromise to Exfiltration

Attacker initiates separate network connection using

credentials obtained in the earlier phase of the attack

4

Attacker moves laterally through organization, relying heavily on escalation of privileges, to systems containing disparate information that when combined, allow

compromise of targeted information

5

Attacker removes data and stages it on a file share within the network

6

Files are encrypted and attacker tries to exfiltrate from several servers before finding a successful exit path from within the organization

7

ATTACKER

(7)

• Focused and coherent attacks

– Times of attacks may be carefully choreographed

– Attackers move rapidly to the target: they know what they want, and the order in which to get it – Months or years of reconnaissance and preparation allow the attacker to move with exacting

precision

• With prior experience, preparation and tools, attackers exploit people and processes more

than weaknesses in infrastructure

– Remote (attacker) hosts may be modified to match internal naming structure

– Attackers may exhibit very detailed knowledge of the people, processes and infrastructure – People continue to be the easiest target in any organization

• Fresh malware may be used

– Compiled just hours before the initial attack event

– Specifically crafted malware yields no known signature to block

(8)

8

• ~$70 Million Write-Down

– Cost of remediation, IT, investigation, consulting, lost revenue,

lost productivity

• Six months focused on remediating authenticators

– All Authentication-related marketing & sales activity stopped for

six months

• Impact to trust

– Customers: How could you do this to us? Why didn’t you contact

us first?

– Lost some customers permanently

• All despite no risk to customers

(9)

• Nobody fired!

• Changed product/services portfolio

–

Spent more on investigative tools vs. checkboxes

• Gained authority - now more than just a token seller

• Recognized for honesty by announcing breach

(10)

Biggest Impact: CIRC Reorganization

CIRC L1 L2 L3 - Eyes on Glass - Analysis - Forensic - Coordination - Remediation - Rule/Report Creation - Workflow Development Advanced Tool & Tactics Cyber Threat Intelligence CIRT Content Analytics - Specific functions

- Reduces “scope creep” - Focused workflow

2009-2011

(11)

• The Ability to Measure

– Event time to Assignment to

Escalation to Resolution

– Average time to closure

• Scan every image file for .exe content

• 55M events per hour become 2-3K incidents per month

– 90 incidents for 94 person hours per day

• Time to closure is now about 1 day

• Program can scale!

(12)

• Define “dwell time”

– From point of trigger to eyes

on (analyst assigned)

• Separation of duties

– Analytics (CIRT Tier 1) – Advanced Tools & Tactics

• Threat intelligence

– Integrated investigation – Visible to analyst

(13)

• Post-breach analysis indicated that threat intelligence would

have played a major role in detecting this activity earlier on

• Actions:

– Built a dedicated Cyber Intelligence group

– Bought multiple commercial intelligence feeds – Joined multiple threat sharing groups

– Custom developed a threat intelligence portal / database – Developed in-house OSINT gathering program

• But at that time, we found…

(14)

• Some threat intel vendors don’t understand the difference

between Intelligence and Information

– A “bad” IP with no context is not actionable!

• Impact: Resources wasted on searching

– Why am I searching proxy logs for the IP address of a mail server

that was used in a phishing campaign?

(15)

• Lack of widely adopted standard for sharing threat intel or IoCs

– Many IoCs are still shared in “unmapped” formats

• CSV, text files, HTML posts, vendor-specific XML

– Impact: Resources wasted on logging into various portals, mailing

lists, feeds and then normalizing the IoCs

– Impact: Human errors when transferring data

(16)

• Limited platforms/applications to house threat intel

– Avalanche, MITRE CRITs, ThreatConnect – Sharing, reviewing/approving, “retiring”

• Have you ever retired an IoC?

• How big are your block lists?

– Impact: IoC lifecycle management difficulty – Impact: Increased burden on security controls

(17)

• Quality of product from vendors varies

– Some do a good job of vetting indicators – However, we still see 8.8.8.8 listed as bad

• Impact: Potential for operational issues

• Impact: Developed custom tools to vet IoCs

(18)

• Justifying the expense to management

– Lack of obvious “wins”

– Early failures due to poor third-party intelligence – Still not finding “all the bad stuff”

– A lot of custom development

(19)

• Reviewed Threat Intel sources

– Removed those that fail to provide context

– Took a hard look at those who don’t provide structured IoC

delivery, regardless of context

– Understood each vendor’s focus area

• Do you need Cybercrime Intel or just APT?

• Migrated from custom Portal to CRITs

– Still required substantial code changes to support EMC workflow – Developed capability to integrate with multiple sharing standards

(20)

• Tracking incident false positive rate based on threat

intelligence source

– Assign confidence values to sources – Feedback to source vendor

• Correlating alerts across multiple data sources to add

contextual elements to Incident record

– When alert from DNS fires, check proxy / firewall logs for

contextual data and add to Incident

(21)

• Malware Intelligence Program

– Leverages Yara, VirusTotal, Cuckoo, Internal DB

– Search for new samples of specific Threat Actor tools each night

and programmatically extracts IoCs

• Passive DNS

– Internally generated and commercial

– Used to pivot on known IoCs to find more

(22)

• Organizational maturity is required!

– Threat intel isn’t the silver bullet

• Need to manage expectations

– Expensive

• Both in $$$ and human capital

– Requires constant care and feeding

• New vendor offerings, quality of data

– Doesn’t always produce tangible results

• No hits today. Intel failure or nothing going on?

(23)

• If you are shopping for external threat intelligence, understand:

– Threat Intel quality varies widely

• Get some samples before signing the contract • Ask your peers

– Threat Intel requires manual data entry

• Amount is proportional to the number of sources

• This is improving, more support for standards [STIX, TAXII]

– Threat Intel will likely require custom coding

• Portal/database, workflow integration, federation/sharing [CRITs]

(24)

(25)

• Stabilize the patient

• Know what you have and prioritize by risk and value

• Harvest system state information from your production systems • Compare what you have to what you deployed

• Remove suspect systems from the environment and return to a

trustworthy state

• Continuously monitor and validate to prevent re-compromise • Communicate in a way that builds trust and confidence

(26)

• The most valuable assets to any company are informed, aware, and

vigilant employees

• A well-defined security policy will take the guesswork out of “what is

appropriate?” employee behavior

The Single Most Effective Security Control

“If I could have chosen anything, technology or otherwise, that would have prevented or lessened the attack against RSA…it would have been a more aware employee base.”

James Lugabihl

(27)