Data Sources
O PERATING S YSTEM L OG P ROBLEMS
The main problem with OS logs—Windows or Linux—is the unstructured nature of the log files and the various ways logs are recorded. In Linux, we have syslog, which gener- ates a completely unstructured log file. Writing parsers for it is really hard. You cannot write the parser for a message until you have seen it at least once in your logs. In the case of Windows, there is a tool called Log Parser31that you can use on the command line to
access Windows log files. However, getting your query syntax right with the tool is not easy and requires some knowledge of the different types of log messages that Windows can generate. Furthermore, services on Windows (e.g., DNS or DHCP) are using log files to log their activity. To be correct, some messages are still logged via the event log, but not all of them. This can be fairly challenging, especially because all the logs are kept in a different format and location. I know that in case of Windows, Microsoft is working on a new log format or even a logging infrastructure, which is based on XML. I am not sure that will solve a lot of problems, but it is an attempt to streamline some of the logging in Windows.
No matter what the operating system, they all are conservative about writing log records. Enabling file auditing, for example, requires special libraries under Linux and some configuration under Windows. Certain activity is really hard to get logged, if it is
OPERATINGSYSTEMS
30 A system that is freely downloadable and deals well with log files and output from these types of tools is
Splunk (www.splunk.com).
possible at all. It is frustrating that special libraries are needed to get logs for certain activity. My favorite example is a password change on a Linux system. You need to use a special PAM library to log password changes.32The library is not shipped by default.
As soon as you start working with log files from different systems, you will run into namespace collisions. For example, two people can have the same user name on different machines. The main issue you want to solve with user names is the attribution of an activity back to a human person. If each person is assigned a globally unique user name, you can achieve that. However, reality is different. The same user on different machines can belong to two different people. There is also the problem of local versus domain accounts. A login by the Administrator account could be from the local or the domain account. How do you differentiate them? Hopefully, the log file is explicit enough to do so. What about users with multiple accounts? How do you correlate their activities? I do not know exactly how many different user accounts I have on different machines. There are quite a few. Without knowing what all users belong to me, it will not be possible to build a comprehensive picture of my activities. Do your users share their logins? This is yet another problem that can throw analysis off. Do you have any service accounts enabled on your systems (users like test, for example). Who owns them? Who is account- able for these activities? And finally, if you allow direct root or Administrator logins to your machines, you have no idea who was actually behind that activity. You need to make sure that root cannot log in directly, but the user has to log in with his or her user first and can then either change to root or execute commands with something like sudo. That way you can attribute information back to individual people.
A last problem around OS logs I want to mention has to do with correlating network- based logs with OS logs. Operating systems are really bad about recording IP addresses. For activity caused by a network connection, they will barely ever log IP addresses. How do you correlate these log entries with network activity? You can use the destination address from the network log and use it to get all the logs from that machine. However, if you have multiple sources connecting to that machine, how do you know which connec- tion is responsible for what OS log entry? This is a hard problem. The only solution for you as a user is to regularly dump the socket states of a machine via netstat, for example. This will allow you to stitch the OS and network logs together. The more elegant solu- tion is to fix OS logging, but that’s not so easy (see Figure 2-5).
32 Snare (www.intersectalliance.com/projects/Snare/) can also be used to achieve logging of password
Figure 2-5 To uncover who is responsible for a certain activity on a host, the following steps are neces- sary: 1) Use the list of processes to identify the user’s command. Most likely, the user’s command was exe- cuted in an interactive shell.To uncover the shell’s process ID, find the process’ parent ID. 2) Use a list of open sockets to associate the process with a socket that in turn identifies the remote entity’s IP address.