Table of Contents
3
|
State of Affairs
3
|
The Challenges
5
|
The Solution
6
|
How it Works
7
|
Solution Benefits
3
State of Affairs
Every successful enterprise requires a myriad of information technologies to function. Whether these are applications, networks, or security devices, every platform is generating a continuous stream of log data. This log data contains vital information about your business, but most of it will go unnoticed. The sheer amount of data makes it difficult to use. This problem, machine big data, can lead to unnecessary spending, complexity, and risk.
As with every form of vital information, machine big data needs to be collected, stored, and distributed to the systems and people who need it. These systems or people have a variety of uses for this data such as security, operational intelligence, compliance, development, and business needs. One such consumer is the ArcSight Security Event Manager (ESM) platform. As with most security event manager products, ArcSight ESM consumes machine data and uses complex rules to correlate multiple pieces of real-time data into a security event. For example, if you visit a company’s corporate web page, at a minimum, a firewall, IDS/IPS, and a web server will log your access attempt. A security event manager has the ability to consume the logs from those three devices and by means of a rule, correlate it into a single event. That event is then analyzed and categorized as either normal behavior or something potentially malicious. Anything potentially malicious would in turn be alerted on so the event could be verified or dismissed by a security professional.
The Challenges
One of the main challenges with using ArcSight ESM, and most security event manager platforms, is scalability. The problem of scalability arises from the complexity and processing requirements that are inherent to security event correlation. At a high level, first the data must be consumed, then parsed and normalized, matched against a list of complex rules stored in-memory, categorized as an event, and alerted on, if applicable. Time becomes a variable in this process as well, because not all data being consumed that may help classify an event will be generated simultaneously or even in rapid succession. Additionally, statistical correlation may be used, which establishes baselines and looks for anomalies and deviations from the baselines. Because of the processing and storage load generated, ArcSight has created other solutions to offload some of the processing. The first is ArcSight Connectors, which can be deployed as software or an appliance, and acts as an initial consumer of machine data. This product performs some parsing and normalization of the data to a proprietary format known as Common Event Format (CEF). ArcSight Connectors then forward the normalized data to ArcSight ESM for correlation. Because ArcSight ESM cannot retain the data for historical analysis or compliance requirements, the data is usually also forwarded to a third platform known as ArcSight Logger. ArcSight Logger can act as an intermediary between ArcSight Connectors and ArcSight ESM. Finally, ArcSight Threat Response Manager is needed to manage the events generated by multiple ESM platforms. A typical ArcSight enterprise deployment is depicted below.
ArcSight Logger ArcSight Logger ArcSight Logger ArcSight TRM ArcSight ESM ArcSight ESM ArcSight ESM ArcSight ESM ArcSight ESM ArcSight Connectors ArcSight Connectors ArcSight Connectors ArcSight Connectors ArcSight Connectors ArcSight Connectors ArcSight Connectors ArcSight Connectors ArcSight Connectors Security Compliance Operations Development Finance
As pictured, the ArcSight deployment can become quite complex and difficult to manage. Even more management overhead is generated by the full-time employees (FTEs) needed to manage the complex correlation rules. Such complex correlation rules need constant fine-tuning to lessen the amount of false positive alerts that are sure to be generated in any environment.
Storage will also be a major factor to consider. As mentioned, ArcSight Connectors normalize machine data into CEF, greatly increasing the size of messages. Some messages are increased 10 times in size, which means that even if an excellent storage compression ratio of 10:1 is achieved, it is of negligible benefit. Also, if there are compliance requirements in the enterprise, most likely a copy of the raw or native message must be kept as well. This means that each message will be stored twice, once in native format and once in CEF.
An additional challenge arises because the ArcSight solution is licensed based on how much data it ingests. The challenge of volume-based licensing is that a fixed cost can rarely be established. As the ArcSight deployment expands and the enterprise grows with the Internet of Things continuing to generate more machine data, the ArcSight licensing cost grows as well. Many times unforeseen events, such as a denial of service (DOS) attack, can cause these costs to increase rapidly. These unknown costs and unforeseen events can pose a real challenge to managing an ArcSight deployment in the enterprise. Also, as pictured above, ArcSight is mainly a security tool with some limited compliance applications. Yet the information contained in an enterprise’s machine big data can have benefit for a variety of use cases across all departments. These concerns can add to the budgetary challenge of managing an ArcSight deployment because in most instances other departments will have their own solution for analyzing the machine
5
To summarize the challenges being faced by enterprises trying to deploy ArcSight, it mainly comes down to cost. In addition to the cost of a volume-based license and its unpredictable growth are factors like procurement, maintenance, and storage. There’s also the cost of FTEs that deploy the platforms and continuously fine-tune the correlation rules. On top of these costs is the chance for unforeseen events that create huge spikes in the amount of machine data being generated. With this in mind, the need for ArcSight volume management solution becomes clear.
The Solution
Just as normal Internet traffic needs to be routed, filtered, and secured, the same is true for machine big data. Similar to a proxy server, load balancer, or any other network device, a true machine big data solution needs to not only collect this data, but also filter and securely forward it to its destination. The TIBCO LogLogic® solution is unique in its filtering and forwarding functionality as well as its enterprise scalability. These features are some of the reasons why many companies choose LogLogic for enterprise logging as a service (LaaS).
How can TIBCO LogLogic’s LaaS solution manage an ArcSight deployment? By using LogLogic as the collection and storage layer for machine big data, you can securely and transparently filter and forward the machine data that consumers, such as ArcSight ESM, receive. This approach will help reduce the costs of an ArcSight deployment. By filtering and limiting the required data that ArcSight needs to meet your enterprise’s security use cases, you no longer need to use ArcSight Logger as a machine data management solution, creating a fixed cost surrounding your ArcSight license and TCO. As depicted in the following graphic, this tactic means less maintenance and a much smaller ArcSight footprint.
TIBCO LogLogic Security Operations Compliance Development Finance ArcSight Connections ArcSight Connections ArcSight ESM
How it Works
The TIBCO LogLogic platform can securely collect machine big data via a variety of methods as required by the log source. For example, data may be transmitted through a secure shell (SSH) connection or retrieved via a secure copy (SCP) file transfer. Once the machine big data is collected, the LogLogic system performs a secure hash algorithm (SHA-256) of the data to prove integrity. Additionally, granular data retention policies allow for custom retention periods for different sets of log data so that only the data your enterprise needs is retained. This data can be retained on the LogLogic system for up to 10 years, as well as searched, reported, and alerted on.
Most enterprises will also need this data filtered and forwarded in real time to a variety of destinations or consumers, including ArcSight ESM. Some other examples of machine data consumers include:
• Security operations centers (SOC)
• Managed security service providers (MSSPs)
• Governance, risk, and compliance (GRC) applications • Data analytics software such as Splunk
• Network monitoring solutions
• Software development tools
The TIBCO LogLogic filtering and forwarding functionality allows for the creation of rules to securelyroute the machine big data to any destination in real time. Now anyone within the enterprise has the capability to access the data they need when they need it, and without a large deployment that only benefits security and compliance.
ArcSight Compliance SEM
M ar 7 0 4 :0 5:0 0 a va s C R O N D [11 23 4 ]: (m ail m a n) CMD M ar 7 0 4 :0 5:0 0 a va s C R O N D [11 23 4 ]: (m ail m a n) CMD M ar 7 0 4 :0 5:0 0 a va s C R O N D [11 23 4 ]: (m ail m a n) CMD Mar 29 2004 09:54:38: %PIX -6-30 2005: Built UDP c onnection f Mar 29 2004 09:54:3 9: %PIX -6-1060 15: Den y T CP (no c onnection) fr o Mar 29 2004 09:54:3 9: %PIX -6-30 2005: Built UDP c onnection f or f addr 194. 66. 12. 71.25 - - [21/F eb/ 20 12 23:44:11] "GET / course/ 18 94 /detail HTTP /1. 1" 200 7 0 66. 12. 71.21 - - [21/F eb/ 20 12 23:44:3 9] "GET / sear ch_b y_author?sear ch_learn_e xp=My 66. 12. 71.25 - - [21/F eb/ 20 12 23:45:21] "GET / course/ 19 /detail HTTP /1. 1" 200 6851 0 .1213 18.4.5. 14 - - [21/F eb/ 20 12 23:45:5 9] "GET / sear ch_b y_subject?sear ch_learn_e xp=algebr a-ii-e xamples HT Mar 7 04:05:00 a vas CROND[11233]: ( cr onjob ) CMD (/ usr /bin/ mrt g / et c/ mrt g/ mrt g. cf g) Mar 7 04:05:00 a
vas CROND[11234]: (mailman) CMD (/
usr /local/bin/ p ython -S / usr /loca Mar 7 04:10:00 a vas CROND[1125 3]: ( cr onjob ) CMD (/ usr /bin/ mrt g / et c/ mrt g/ mr Mar 7 04:10:00 a vas CROND[11254]: ( cr onjob ) CMD (/ usr /lib/ sa/ sa1 1 1) Mar 7 04:10:00 a vas CROND[1125 7]: ( cr onjob ) CMD (/ sbin/ dc cc ollect.sh) Mar 7 04:10:00 a
vas CROND[11255]: (mailman) CMD (/
usr /local/bi8.2 Mar 29 2004 09:54:33: %PIX -6-1060 15: Den y T CP (no c onnectio Mar 29 2004 09:54:3 9: %PIX -6-30 2005: Built UDP c onnection f or 66. 12. 71.25 - - [21/F eb/ 20 12 23:44:11] "GET / course/ 18 94 /detail HTTP / 66. 12. 71.21 - - [21/F eb/ 20 12 23:44:3 9] "GET / sear ch_b y_author?sear ch_lear 66. 12. 71.25 - - [21/F eb/ 20 12 23:45:21] "GET / course/ 19 /detail HTTP /1. 1" 200 685 18.4.5. 14 - - [21/F eb/ 20 12 23:45:5 9] "GET / sear ch_b y_subject?sear ch_learn_e xp=al Mar 7 04:05:00 a vas CROND[11233]: ( cr onjob ) CMD (/ usr /bin/ mrt g / et c/ mrt g/ mrt g. cf g) Mar 7 04:05:00 a
vas CROND[11234]: (mailman) CMD (/
usr /local/bin/ p ython -S / usr /local/ mailman/ cr o Mar 7 04:10:00 a vas CROND[1125 3]: ( cr onjob ) CMD (/ usr /bin/ mrt g / et c/ mrt g/ mrt g. cf g) Mar 7 04:10:00 a vas CROND[11254]: ( cr onjob ) CMD (/ usr /lib/ sa/ sa1 1 1) Mar 7 04:10:00 a vas CROND[1125 7]: ( cr onjob ) CMD (/ sbin/ dc cc ollect.sh) Mar 7 04:10:00 a
vas CROND[11255]: (mailman) CMD (/
usr /local/bin/ p ython -S Mar 7 04:15:00 a vas CROND[113 51]: ( cr onjob ) CMD (/ usr /bin/ mrt g / et) Mar 7 04:15:00 a vas CROND[113 52]: (mailman) CMD (/ usr /local/bin/ p Mar 7 04:20:00 a vas CROND[113 72]: ( cr onjob ) CMD (/ usr /bin/ c 66. 12. 71.25 - - [21/F eb/ 20 12 23:44:11] "GET / course/ 18 94 /detail HTTP /1. 1 66. 12. 71.21 - - [21/F eb/ 20 12 23:44:3 9] "GET / sear ch_b y_author?sear ch_le 66. 12. 71.25 - - [21/F eb/ 20 12 23:45:21] "GET / course/ 19 /detail HTTP /1. 1" 200 68 18.4.5. 14 - - [21/F eb/ 20 12 23:45:5 9] "GET / sear ch_b y_subject?sear ch_learn_e0 Mar 7 04:05:00 a vas CROND[11233]: ( cr onjob ) CMD (/ usr /bin/ mrt g / et c/ mrt g/ mrt g. cf g) Mar 7 04:05:00 a
vas CROND[11234]: (mailman) CMD (/
usr /local/b Mar 7 04:10:00 a vas CROND[1125 3]: ( cr onjob ) CMD (/ usr /bin/ mrt g / et c/ mrt g/ mrt g. cf g) Mar 7 04:10:00 a vas CROND[11254]: ( cr onjob ) CMD (/ usr /lib/ sa/ sa1 1 1) Mar 7 04:10:00 a vas CROND[1125 7]: ( cr onjob ) CMD (/ sbin/ dc cc ollect.sh) Mar 7 04:10:00 a
vas CROND[11255]: (mailman) CMD (/
usr /local/bin/ p ytho Mar 7 04:15:00 a vas CROND[113 51]: ( cr onjob ) CMD (/ usr /bin/ mrt g / et Mar 7 04:15:00 a vas CROND[113 52]: (mailman) CMD (/ usr /local/bi Mar 7 04:20:00 a vas CROND[113 72]: ( cr onjob ) CMD (/ usr /bin
exported29Apr2014
7
Solution Benefits
TIBCO LogLogic’s LaaS platform does not have any volume-based licensing so you never have to worry about unpredictable costs. The LogLogic LaaS solution has a fixed cost that in most cases provides proven savings and ROI in under two years, especially when used to manage your ArcSight deployment. In many scenarios, a single LogLogic appliance can ingest machine data at a rate that requires three to six ArcSight loggers. The following value model shows this scenario.
ArcSight Only ArcSight with LogLogic
GBs of Indexed Data per Day
C os t ($) Lowe r Higher Low High
TIBCO LogLogic & ArcSight Cost
Since LogLogic is now managing your machine big data, you no longer have to worry about massive storage requirements driven by the explosive size of the CEF and the need to store every message twice. Your data retention policies are now quickly and easily managed using LogLogic granular retention rules. Additionally, indexed machine data retention policies can be separated from raw machine data retention policies. This separation means improved use of storage resources and the ability to search through compressed raw machine data during time periods outside of your index retention period. The TIBCO LogLogic LaaS platform offers an effortless lifecycle and is truly plug and play. Setup of the solution is quick and easy, and it does not require an FTE to manage it. With this ease and flexibility, it is never too late to put the brakes on an ArcSight deployment that is growing too rapidly or becoming too costly to scale. A TIBCO LogLogic appliance can be inserted into your environment in front of machine data sources to immediately stem the flow of too much data being sent to ArcSight. Additionally, while the TIBCO LogLogic solution can parse and normalize machine data, it always stores 100 percent of the raw machine data, so it can act as your machine big data system of record. Furthermore, any data modification can occur at the machine data consumer, in this case ArcSight Connectors. LogLogic also contains many enterprise features such as high availability so you never have to worry about losing machine data. Look to TIBCO LogLogic as a true LaaS platform that will provide an ArcSight management solution while managing all of your machine big data, making sure it is delivered to your machine data consumers in real time.
TIBCO Software Inc. (NASDAQ: TIBX) is a global leader in infrastructure and business intelligence software. Whether it’s optimizing
inventory, cross-selling products, or averting crisis before it happens, TIBCO uniquely delivers the Two-Second Advantage®— the ability to capture the right information at the right time and act on it preemptively for a competitive advantage. With a broad mix of innovative products and services, customers around the world trust TIBCO as their strategic technology partner. Learn more about TIBCO at www.tibco.com.
©2014, TIBCO Software Inc. All rights reserved. TIBCO, the TIBCO logo, TIBCO Software, and TIBCO LogLogic are trademarks or registered trademarks of TIBCO Software Inc. or its subsidiaries in the United States and/or other countries. All other product and company names and marks in this document are the property of their respective owners and mentioned for identification purposes only.
www.tibco.com Global Headquarters 3307 Hillview Avenue Palo Alto, CA 94304 Tel: +1 650-846-1000 +1 800-420-8450 Fax: +1 650-846-1005
