with Big Security Data
HOW INTELLIGENT AND SCALABLE SIEM SOLUTIONS HELP IT SECURITY PROFESSIONALS
STAY ON TOP OF AN EVER-EVOLVING, DATA-DRIVEN ENVIRONMENT
Technology today has become synonymous with data. As each new tool enters the enterprise, the shear volume of information IT organizations deal with compounds. Gartner estimates the amount of data analyzed by enterprise information security organizations will double every year through 2016. This explosion of data and processing adds not only complexity to the business environment, but also a “big security data” challenge that organizations need to address. As security needs and compliance mandates continue to evolve, the need for context,
analytics and the time period for which data must be stored becomes more critical.
“Expectations for what security professionals should provide to the enterprise are also changing rapidly because of big data,” explains Trevor Welsh, enterprise solutions architect with McAfee, a leading provider of enterprise-grade security solutions. “Security groups are now expected to be experts in a lot of different types of data, including the inner workings of databas-es, applications or security of an application stack,” he says. “And now that it’s possible to extract data from
❱❱
SIEM: Keeping Pace with Big Security Data
2
these places in a meaningful way, the thought is that security as a group will be able to utilize this data in an intelligent way to provide guidance back to the busi-ness. Security teams are tasked not only with protect-ing the business, but with providprotect-ing valuable business intelligence as well.”
As data continues to grow exponentially, the threats facing organizations are evolving as well. Today’s attackers are skilled professionals conducting advanced targeted attacks, meaning prevention alone cannot protect enterprises. “It wasn’t long ago that there were singular bad indi-viduals who wanted to break into big enterprises, cause disruptions and brag,” he says. “However, the scene has changed with advanced persistent threats (APT) and state-sponsored terrorism programs added to the mix. As a result, security professionals are expected to monitor systems as well as parse through mounds of information from various sources to figure out how to best leverage their limited resources.”
One positive aspect of big security data has been the shift in perception around security. “Initially companies did not want to pay for security—not because they didn’t care, but because they deemed security as expensive and non-revenue generating,” says Welsh. “However, the stringency and costs of compliance [for PCI DSS, HIPAA etc.] motivated organizations to make investments and improve the data environment—moving the pendulum towards meeting compliance. Yet, over time as these efforts became more rigorous, companies started to realize that it was cheaper to just become more secure. This was the advent of CSOs becoming more power-ful. They were at the table answering to the CIO and doing security for security’s sake.”
❱❱ UNDERSTANDING SIEM
While the volume of information and number of threats continue to grow, it’s clear that traditional log management systems can’t handle big security data. Fortunately, there are proven technologies capable of helping. The current generation of Security Infor-mation and Event Management (SIEM) technology is a prime example. Solving today’s big security data challenge requires evolving from traditional relation-al databases and time-based flat file systems that
legacy SIEM solutions have leveraged as their core analytic capability.
SIEM technology provides real-time analysis of security alerts generated by network hardware and applications. By definition SIEM focuses on capabil-ities of gathering, analyzing and presenting infor-mation from network and security devices; identity and access management applications; vulnerability management and policy compliance tools; operating system, database and application logs; and external threat data. Key areas of focus include monitoring and managing user and service privileges, directory services and other system configuration changes, as well as providing log auditing and review and incident response.
The purpose of SIEM solutions is to accurately compare in a single location all the data collected by a variety of security devices, applications and data sources. Specifically, with SIEM solutions it’s possible to pool together routers, switches, and virtual machines (VMs) and then normalize the data. “As a result, no matter where the data comes from, it all looks the same, and it’s easier to draw com-parisons,” Welsh explains. “This capability makes it possible to see, for example, what one IP address did across all of the company firewalls.”
SIEM is also instrumental in categorizing data, which is key considering how many different operating sys-tems operate within today’s evolving environment. “Any time someone logs in, it creates an event. The challenge is that all of these login events look different,” Welsh says. Welsh notes that an effective SIEM solution should be able to understand what a login looks like, regardless of platform. As a result, if security wants to see all failed log-on activities, the SIEM should have the ability to provide that insight. ❱❱ RECOGNIZING DIFFERENCES
Of course, it’s important to note that not all SIEM solutions are created equally. In fact, many SIEM solutions in place today struggle to collect and man-age all the required contextual data. At the same time, the data load and analytics pressure has grown beyond what those data management systems can handle.
“ Security
professionals
are expected to
monitor systems
as well as parse
through mounds
of information
from various
sources to figure
out how to
best leverage
their limited
resources.”
—Trevor Welsh, Enterprise Solutions Architect, McAfee
Below are a few key qualities that often serve as differentiators in applications. As such, IT profession-als should consider these needs as it evaluates SIEM solutions:
nUsability. SIEM solution workflow and ease of use must be intuitive and effective. SIEM solutions should present security with a dynamic dashboard envi-ronment that allows them to quickly drill down into data. For instance, if someone clicks on an incident, the dashboard should light up with the details so you immediately know who is involved, the threats, the systems, geographies, etc. “There is an idea that SIEM needs to be complex or really simple. The truth is, it should be in the middle—it needs to be simple for your use cases. You should be able to meet your require-ments after the setup is complete,” Welsh explains. “Of course, SIEM cannot configure itself, so reaching this level will take some work.”
nSpeed. The overall speed to recall data should be a key consideration before selecting a SIEM solu-tion. “The recollection of data for a SIEM is crucial whether you are performing an ad hoc or forensics investigation,” says Welsh. “For instance, one of the most crucial components of a SIEM in today’s environment is its ability run rules at a high speed against all of the data. Considering that data comes in very quickly—up to 10,000 events per second—a SIEM needs to be able to execute and tell the ana-lyst or security group of any issues.”
nScalability. “Will the SIEM grow with the orga-nization? This is only possible if the solution has distributed correlation, which means the installation can be expanded without a rip and replace,” he says. “Given how fast the business environment is evolving, no one can afford to embrace a solution that cannot grow with the organization.”
nVendor engagement. Pay close attention to how many training hours a vendor recommends. Success with SIEM deployments is often closely tied to how many training hours and professional services a company gets relative to the amount recommended by the vendor. “Leverage the vendor to help ensure the organization achieves alignment between goals and actual results,” says Welsh. “You need to make sure you have an ongoing relationship with your vendor if you want to get the most out of the investment.”
❱❱ FINDING SUCCESS
Beyond solution criteria, an organization’s planned approach to embracing a SIEM solution can play a crucial role in determining the outcome. For instance, it’s important for security professionals to set clear expectations before deploying their SIEM solution. “Success here really starts with building the knowledge base. For instance, it’s useful to read what analysts say because it provides insight into what is happening,” says Welsh. “In addition, talking
An intelligent and
effective SIEM
solution can help
your organization:
n Achieve meaningfulsituational awareness through rich context and analysis
n Diagnose and respond to
incidents in seconds, not hours, to reduce damage, prevent data breaches, and lower remediation costs
n Experience fewer security
and compliance incidents and lower per-incident costs
n Simplify compliance policy
processes and reporting to improve operational efficiency
n Reduce training time and
operational cost
Effective Real-time Security
Effective security starts with real-time visibility into all activity on all systems, networks, data-bases and applications. McAfee Enterprise Security Manager enables your business with true, real-time situational awareness and the speed and scale required to identify critical threats, respond intelligently and ensure continuous compliance monitoring. Security teams now have access to real-time, risk-relevant information to obtain a stronger security posture while shortening response time. Other features include:
❱❱Actionable information in minutes instead of hours
❱❱Massive data collection across a wide range of information sources ❱❱Real-time threat and risk data integration and event correlation ❱❱Immediate access to years of event and flow data
❱❱Monitoring and reporting support against more than 240 regulations ❱❱Integrated tools for improved security workflow
❱❱
SIEM: Keeping Pace with Big Security Data
4
with others in your security peer group to learn about actual implementation experiences and use case can help achieve expectations and allow you to go into the project with achievable goals.”
Early on, Welsh recommends focusing on under-standing exactly what a SIEM can do for the busi-ness. “SIEM is not a magical black box that you set and forget. Instead, a SIEM is an integral part of your security operations. The most successful deploy-ments occur when IT involves several groups (e.g. compliance, ...), OS, desktop support, networking, etc.) within the process,” he says. “Involvement from
the early stages is instrumental in securing buy-in and provides varied and insightful input, resulting in a better end product.”
While many see big data as a challenge to SIEM, Welsh sees its presence within the organization as a welcome partner. “Big data can provide increasingly larger amounts of intelligence to SIEM, meaning SIEMs have proportionally more opportunity to gain insight and improve understanding of how critical network assets are being utilized and by whom,” he says.
“ Leverage the
vendor to help
ensure the
organization
achieves
alignment
between goals
and actual
results”
—Trevor Welsh, Enterprise Solutions Architect, McAfee
Success with SIEM
Operating within an industry known for its massive amounts of data and rigorous compliance demands, an effectively deployed SIEM solution is instrumental for Edward Pardo, CISSP, senior IT security engineer with the Roswell Park Cancer Institute located in Buffalo, NY.
“Having the ability to look at events across the entire environment versus a system at a time is crucial today,” says Pardo. “It’s a SIEM that makes it possible to gain access to the goldmine of data that otherwise is ignored.”
Properly implemented, a high-value SIEM solution provides visibility to all the connected systems. “There are a lot of times where we use the system to gain a new perspective as to what is going on. For instance, you can get tunnel vision looking at some of the point solutions and the data they put out,” he says. “SIEM allows you to put everything together, look at it from every angle and verify that existing management tools are actually doing what they are supposed to be doing.”
According to Pardo, the key to success is to get the business and management actively involved from the beginning. “Early involvement helps answer why we are doing this and gets the teams onboard that you are going to connect to the system. Without the big picture, they may see it as duplication of efforts,” he says. “However, SIEM is more like glue that holds everything together. It is the way to truly build IT intelligence. If you look at a lot of the business intelligence architecture, it is heavily dependent upon IT. Having a wide range of people on board with the project in advance simplifies the entire process.”
Pardo also recommends taking the time to do it right. This includes building an accurate inventory of the architecture and infrastructure already in place as well as a solid understanding of the organization’s end goal in embracing a SIEM.
“If you want to get the most out of a SIEM, you need to realize that it is not a black and white project. There are a lot of questions to address along the way: What is the analysis? How much data am I actually bringing in? What are we hoping to do with it?” Pardo says. “It is a situation where until you have a true understanding of your environment, it’s difficult to understand the true areas of concern. Plus, you don’t want to put yourself in a position where you are bringing too much data in too fast. You will end up swamped and will realize that too much of the material you are bringing in is garbage.”
McAfee updates business security
management tools
ADDS REAL-TIME QUERYING CAPABILITIES TO MCAFEE EPO AND ENABLES SIEM TO AUTOMATE SECURITY RESPONSE TO SUSPICIOUS EVENTS
This article orginally appeared in Computerworld, February 2013.
McAfee is enhancing its business security platform by adding near real-time querying capabilities to its Orchestrator (ePO) software and by integrating it with its security information and event management (SIEM) product to automatically initiate endpoint security policy changes.
The ePolicy Orchestrator software is the core of McAfee’s Security Connected framework and strat-egy, that aims to have all security products used in a business environment working together and sharing information. It is a central security management software that lets businesses gather data from end-point systems, update and deploy configurations, initiate endpoint and network security policies, and interact with other security products, not only from McAfee, but also from other vendors in the McAfee Security Innovation Alliance.
Managing tens or hundreds of thousands of end-point systems in an enterprise environment can be a time-intensive task. In order to reduce the time pen-alty, McAfee launched McAfee Real Time for ePO, a technology that reduces query time to seconds and allows businesses to get information from products installed on endpoint systems and investigate possi-ble security events much faster.
“For example, if I want to know if all files are up to date on endpoint systems or some information about registry, I can get that in seconds with Real Time for ePO and with very light load on the network at the same time,” said Gretchen Hellman, director of product marketing for SIEM at McAfee. That’s thanks to a new communication mechanism that uses a chaining query method where instead of que-rying each endpoint individually, the server sends out a single request that gets passed around in a peer-to-peer fashion, she said.
“The performance improvement will vary depending on network environment, Hellman said. On small networks, such operations can now be performed 10 times faster, but on really large networks the per-formance improvement can be up to 1,000 times,” she said.
The second platform enhancement that McAfee announced was the integration of its SIEM product, the McAfee Enterprise Security Manager, with ePO, McAfee Vulnerability Manager and the McAfee Net-work Security Platform.
The SIEM already uses McAfee’s Global Threat Intelligence feed, which contains information about malicious resources such as websites, domains and file servers. This allows the product to analyze logs and event data collected from endpoints and alert the system administrator of any suspicious commu-nication with a potential bad actor.
The new SIEM enhancements also enable the product to also automatically take action based on predefined rules. For example, when the SIEM sees potential interaction with a bad actor it can automat-ically initiate a scan on the affected endpoint to see if there’s malware running on it or can instruct the McAfee Network Security Platform to immediately block the suspicious communication, Hellman said. It can also tell ePO to make policy changes and tag the system for additional investigation.
“What the SIEM actually does now is take intelli-gence and turn it into intelligent action,” Hellman said.
These enhancements are part of McAfee’s Security Connected strategy to focus its efforts on achieving greater integration between its own products and the products of its partners.
“ If I want to
know if all files
are up to date
on endpoint
systems or some
information
about registry, I
can get that in
seconds with
Real Time for
ePO and with
very light load
on the network
at the same
time.”
—Gretchen Hellman, Director of product marketing for SIEM, McAfee
6
Big Data is not only a challenge for customer-facing organizations—but for security teams as well. Over the past decade, the demand for stronger security has driven the collection and analysis of increasingly larger amounts of event and security contextual data. SIEM has long been the core tool that security teams have depended on to manage and process this information. However, as security data volume has grown, relational and time-indexed databases that support SIEM are struggling under the event and analytics load. Legacy SIEM systems have raised doubts about the potential success of SIEM imple-mentations due to their slow performance, inability to manage data effectively, and the extremely high costs associated with scaling.
❱❱BIG SECURITY DATA
Why security data has become a Big Data problem is obvious for anyone who has tried to manage a legacy SIEM, particularly when you look at the defi-nition of Big Data. Big Data consists of data sets that grow so large that they become awkward to work with using existing database management tools. Challenges include capture, storage, search, sharing, analytics, and visualization.
With this in mind, it’s easy to see that IT and IT security have repeatedly wrestled with Big Data challenges. In fact, SIEM itself was invented to address a fundamental lack of data processing capabilities. In the early 2000s, the amount of security information and the level of accuracy of this security data exceeded the capability of existing technologies, and the lack of centralized visibility developed a strong need for automated data analysis. Enter the early SIEM tools, which were de-signed to handle firewall, vulnerability assessment, and intrusion detection systems (IDS) data with the primary purpose of reducing false positives from IDS plus the ability to investigate logs. These early SIEM vendors leveraged existing database management tools and provided specialized analytics on top of event data to enable organizations to eliminate a large number of IDS false positives.
While SIEM initially was adopted by security-con-scious industries—such as large financial services and government—broad adoption did not take off as a viable market until the mid-2000s, when Sarbanes Oxley audit became a reality. Overnight, event management was a core component of the “control framework” in Sarbanes Oxley section 404, and internal and external auditors were requiring it. Sarbanes Oxley was quickly followed by PCI DSS for retail organizations and credit card processors, which introduced log review requirements to pass an audit, inspiring many to turn to SIEM for its promises of automation. And then the regulatory explosion began. The SIEM market exploded along with it—into a billion dollar market.
Compliance not only increased SIEM adoption but also led to a flood of additional security instrumen-tation and increased logging levels. This simultane-ously increased the flood of data SIEM now had to manage and further stretched analytic capabilities. Legacy SIEM systems had always struggled to manage any increases in volume and correlation of security data. This dramatic growth in data and correlation requirements further revealed the inher-ent scale and analytic limitations that these SIEM solutions faced.
Fast forward to a year or two. The demands on SIEM systems continue to intensify. Devastating data breaches at organizations that had passed purportedly stringent compliance-based security audits have pushed IT security to move from “check-the-box” compliance to comprehensive security programs that include perimeter, insider, data, and system security. In response to these increased se-curity controls, innovative and persistent attackers have evolved the sophistication level of their attack methods—creating a need for SIEM to detect low-and-slow attacks, rapidly detect anomalies in event flow, and gain contextual information about data, applications, and databases.
The Big Security Data Challenge
MAKE SIEM WORK FOR YOU
ADDITIONAL READING
❱❱
SIEM: Keeping Pace with Big Security Data
Over the past
decade, the
demand for
stronger security
has driven the
collection and
analysis of
increasingly
larger amounts
of event
and security
contextual
data.
❱❱THE BIG SECURITY DATA CHALLENGE
These increasing demands on SIEM have stretched legacy SIEM solutions to their limit. These legacy SIEM systems were built on databases and archi-tectures with inherent limitations in their ability to handle large volumes of events, historical data, and extensions of relational data. In addition, the analytic capabilities of legacy SIEM systems are insufficient. Many organizations turn off important, but non-es-sential analytic capabilities and spend hours waiting for a single report. These challenges have led to the question: “Does SIEM work?” Given advancements in SIEM today, that question needs to shift to: “Does my SIEM solution meet my current demands and will it scale, in both capacity and analytics, to meet evolving demands?”
Solving today’s Big Security Data challenge requires evolving from the traditional relational databases and time-based flat file systems that older SIEMs leverage as their core analytic capability. Traditional relational databases strain under the stress of simultaneous high-speed insertion rates combined with the added burdens of continuous real-time correlation and historical reporting. Time-based flat file systems fall under the pressure of complex que-ries and, due to their limited indexing, can only offer basic correlation capabilities.
Organizations looking to be successful with SIEM— whether they are first-time adopters or replacing legacy SIEM—need to carefully evaluate the back-end capacity and analytics of SIEM solutions under consideration to understand how intelligent the front-end will be for their needs today and tomor-row. Below are some statistics from the Gartner report, “Information Security Is Becoming a Big Data Analytics Problem, 23 March”:
n The amount of data analyzed by enterprise
infor-mation security organizations will double every year through 2016
n By 2016, 40 percent of enterprises will actively
analyze at least 10 terabytes of data for infor-mation security intelligence, up from less than 3 percent in 2011
Let’s look at some core capabilities of an ideal SIEM, why these capabilities are important, and how to evaluate them in light of the Big Security Data prob-lem organizations face today and will continue to face in the future.
❱❱RELATIONAL DATA EXTENSIBILITY
Because the volumes of event data have grown exponentially and attacks have become more sophisticated, it is critical to enrich event data with relational data about the source, asset, user, and data-intelligent situational awareness. In addition, real-time correlation of this information with event flows needs to be accommodated in the database architecture. If the database architecture can’t han-dle these millions of relational data points, organi-zations will quickly hit a brick wall in expanding the intelligence of their SIEM systems. Extensibility of features such as watch, asset, and user lists should be carefully evaluated, in combination with the ana-lytic capabilities to apply this information intelligent-ly. While many SIEMs have these features, few can support multiple and expansive lists due to database side table limitations. Also, to avoid analytic perfor-mance degradation, many SIEMs will simply provide a look-up of this information, on request of the user, rather than correlate and present it in real time. A strong SIEM will use this information to intelligently create an accurate, real-time picture of risk. ❱❱DYNAMIC ANALYSIS
Requirements for obtaining true situational aware-ness today goes far beyond simple event flow analy-sis, which can tell you the frequency of connections and if there is a change. Today’s SIEM requires dynamic situational that identifies changes in user behavior and dynamically adjusts risk based on source reputation and asset risk, as well as the data, applications, and database activity that relates to it. Dynamic analysis is a critical component of low-and-slow attack detection, and Big Security Data SIEM architectures need to accommodate that.
Solving today’s
Big Security
Data challenge
requires
evolving from
the traditional
relational
databases
and time-based
flat file systems
that older SIEMs
leverage as their
core analytic
capability.
8
❱❱HISTORICAL ANALYSIS
Another key aspect of attack detection and efficient incident response is the ability to analyze historical event data. With attack methods today, it is essential for an SIEM to be able to access years’ worth of data to quickly pinpoint patterns and anomalies, while maintaining real-time analysis without per-formance degradation. It also needs to be able to integrate easily with storage systems and efficiently store event data to avoid extensive storage instru-mentation and costs, offering an architecture that supports simultaneous heavy use of real-time and historical functions.
❱❱EVENT SURGES
Most organizations with SIEM solutions in place will experience event surges—times when event data grows beyond peak expected limits. When an event surge occurs, it is critical that analysts be able to determine whether the increased volume is due to an active attack. SIEMs built for Big Security Data are not only able to handle these surges, but also factor in these surges in their licensing schemes.
SIEMs that do not understand this problem will drop events or lock out analysts from the console when the events per second (EPS) limits are exceeded— preventing security teams from accessing their primary means of situational awareness when it matters most.
❱❱SUMMARY
Automating security monitoring has proven essen-tial in today’s threat environment, and to succeed, today’s SIEM must have the right database back-end and must offer security intelligence that leverages contextual data.
❱❱LEARN MORE
For more information, visit www.mcafee.com/SIEM. ❱❱ABOUT MCAFEE
McAfee, a wholly owned subsidiary of Intel Corpora-tion (NASDAQ:INTC), is the world’s largest dedicated security technology company. McAfee delivers proactive and proven solutions and services that help secure systems, networks, and mobile devices around the world, allowing users to safely connect to the Internet, browse, and shop the web more securely. Backed by its unrivaled global threat intel-ligence, McAfee creates innovative products that empower home users, businesses, the public sector, and service providers by enabling them to prove compliance with regulations, protect data, prevent disruptions, identify vulnerabilities, and continuously monitor and improve their security. McAfee is re-lentlessly focused on constantly finding new ways to keep our customers safe. http://www.mcafee.com ADDITIONAL READING
