The Use of Digital Forensic Case Studies for
Teaching and Assessment
Harjinder Singh Lallie
School of Computing, University of Derby, Kedleston Road, Derby DE22 1GB [email protected]
Abstract
This study analyses the use and development of Digital Forensic case studies for the purpose of teaching and assessing Digital Forensics students and practitioners. Within this study, case studies are categorised and a number of available case studies are explored. The importance of evidentiary and non-evidentiary artefacts within the case study are examined. Mechanisms for integrating case study development and/or investigation with student assessment are proposed, the benefits and the challenges of this approach are examined. Practical and technical issues involved in the development of case studies are examined. The study concludes by proposing guidelines for the development of Digital Forensic case studies.
1
Introduction
Digital Forensic (DF) case studies can be used for:
the teaching and assessment of digital forensic students and practitioners;
testing and validating digital investigations software/tools;
research and development.
This study focuses on the use of DF case studies for teaching and assessing DF students and practitioners.
DF case studies are enacted on a computer system and represent a particular scenario that requires analysis and/or investigation. The computer storage system(s) on which the case studies have been enacted are subsequently converted to DF images which can then be analysed and investigated using the appropriate tools.
The DF Image is a bitstream copy of an original hard disk. The image structure begins with a vendor specific header, is then interspersed with vendor specific control information (such as CRCs) and superceded with a footer. Images
conforming to standardised formats (such as .E01 or .dd) are readable by popular disk investigation tools and are therefore popular destination image formats.
1.1
Specific Skill and Holistic Skill
Case Studies that are used to teach/assess a particular and specific skill are referred to herein as Skill Specific Case Studies whilst those that are used to develop and assess a range of skills are referred to as Holistic Skill Case Studies.
Specific skill case studies serve a valid but limited purpose and are easier and faster to develop than holistic skill case studies. Specific Skill case studies can be used to teach/assess particular concepts such as file system analysis, partition Analysis, Metadata analysis or E- mail investigation.
Holistic skill case studies are based around specific scenarios which require a more detailed and thorough investigation. These are designed to teach/assess a collective range of specific skills including the overall investigation of a case possibly through to the production of a case report. Holistic Skill case studies are more difficult to construct and require careful planning. However if used for assessment purposes can encourage more enthusiasm and motivation amongst students.
1.2
User behaviour patterns
The use of a computer system generates a User Behavioural Pattern (UBP) which is subsequently 'captured' within the case study. The UBP is the system state that is generated after the operating system has been installed/configured and when the user begins to use the system. The UBP is a continually evolving pattern which reflects the installation of applications, the creation and use of e-mail accounts, access of web sites, the downloading of data etc. For students to properly understand the context of a particular case, they should develop some knowledge of UBPs.
There are two types of UBP - those that reflect 'normal usage' and those that reflect the usage of a computer system for the planning or perpetration of an incident or crime. The former can be referred to as a normal UBP and the latter as an incident related UBP. An incident related UBP will deposit items of evidence in specific locations of the computer hard disk system as a natural course of the users actions, it is these particular artefacts that then form the basis of the investigation.
Case studies that reflect a normal UBP are easy to construct and serve a useful purpose for teaching specific skills whilst having a limited use for teaching/assessing holistic skills. A typical windows installation followed by two months of normal usage might be sufficient to construct such a case study. Whilst there is good academic and practical rationale behind developing case studies which demonstrate a normal UBP, there are two challenges:
The developer must ensure that he/she does not leave personal data behind through this 'normal' usage. This is almost a contradiction in terms as it is not actually 'normal' usage.
The motivation levels behind students who are subsequently required to 'investigate' these may be quite low.
A normal UBP is important for the generation of 'noise'.
1.3
Evidentiary and Non-Evidentiary Artefacts
A case study based around an incident related UBP will contain evidentiary or non-evidentiary artefacts. There are two types of non-non-evidentiary artefact:
System files and folders generated by the operating system or software application installation. These are artefacts that have not been created or altered by computer usage and can largely be ignored during the investigation. A new operating system installation will result in such non-evidentiary artefacts.
The normal usage of a computer system generates non-evidentiary artefacts which are referred to as noise. Noise refers specifically to artefacts that have no evidentiary value, i.e. they do not form part of the evidence supporting the case but have been created/modified/accessed by users of the computer system and not the operating system or software applications.
A small degree of noise can be generated through „normal‟ system usage. Typical windows installation followed by around four to eight weeks of normal usage might be sufficient for this purpose. The investigation will often involve the analysis of some of the noise, but ultimately the noise may be ignored. Without the noise, the eventual investigation would be simplistic and superficial and the evidentiary element of the case study would be easily distinguishable.
Evidentiary and non-evidentiary (particularly noise) artefacts of the case study must be generated in parallel and therefore require careful planning. It is important to understand that the proportion of noise to evidentiary value within the case study might be very high as it is unlikely (but not impossible) that a computer system might be used primarily for the planning and perpetration of a crime.
2
Existing case studies
There are a growing number of publicly available DF case studies. For legal reasons, realistic DF images (i.e. those images that reflect real cases) are not available for use in academia or for professional training. For related reasons, images that have formed part of a civil case are also generally unavailable. Therefore, most if not all of the available images are artificially created.
A set of particularly useful DF images (mostly in .E01 format) are provided by NIST through the CFReDS (Computer Forensic Reference Data Sets) Project [1]. The images include: a set of Skill specific images, one Holistic skill specific image (the Greg Schardt image) and a set of images that can be used for mobile phone investigation.
The Greg Schardt image is based around a suspected hacker who is alleged to have intercepted credit card numbers, usernames and passwords. A series of questions are posted by CFReDS which can be used as a set of 'ready made' tutorials or even an assessment for students. Sample solutions are available on the website.
Brian Carrier [2] has published a series of skill specific DF images (all in .dd format), these were previously posted to the Computer Forensic Tool Testing group (CFTT) at NIST [3]. These images are useful for teaching\assessing specific skills such as: FAT keyword searches, FAT un-deletions and JPEG searches.
A number of useful holistic skill images have been posted by Lance Mueller [4]. Each of these images is supported by a task definition statement, some are accompanied by sample answers. One of the images involves the investigation of suspected network attacks on a machine with a Windows XP home installation. The image is accompanied by a TCPdump of network traffic and therefore acts as a useful exercise in testing network forensics related skills. Another image in the Lance Mueller collection involves a potential IPT claim. The third image is a particularly interesting challenge involving the recovery of a file from an 'unreadable drive'.
A lightweight holistic skill case study is available from The International Society of Forensic Computer Examiners (ISFCE) [5]. This image must be extracted to a floppy diskette and the case study revolves around an IPT case, sample answers are provided.
Digital Corpora [6] provides a series of particularly useful DF images. One of these is a skill specific image which involves file recovery and carving on an SD card from which certain JPEG images (taken by a Canon camera) were deleted. One of the particularly useful and interesting images is of a USB stick image which contains an Ubuntu 8.10 installation through which the user had browsed a number of US Government websites.
The practice of publishing solutions is followed by many of the DF image publishers, this potentially limits the viability of using these images for investigating within an assessment. Academics can address this problem by:
Students can be tasked to demonstrate their understanding of the techniques and methodology applied to solving the problem rather than presenting the solution in isolation.
Requiring a practical demonstration of the answer so as to indicate that students understand the process and methodology in finding that answer.
Developing further questions in addition to those posed by the publisher.
3
Approaches to Developing Case Studies
Student or self generated case studies can serve many useful purposes. Such case studies have to represent fictional civil cases as opposed to criminal cases as the latter might attract police scrutiny and give rise to negative institutional publicity. Furthermore, there may be serious ethical issues involved in the development of a case study relating to a criminal act.
3.1
Student Generated Case Studies
Student generated case studies can be developed through an assessment which involves a student (or group of students) developing and proposing a case study scenario and then enacting it on live computer systems. This is not a unique approach and others have tried it in the past [Carlin et al., 2005].
Student case study development must be guided by a tight remit which is clearly defined through the assessment specification or through milestone based interaction. This is so that:
Ethical and legal issues relating to the context of the case study are carefully managed.
The tight remit results in a case study that can actually be used. Student generated case studies have a number of benefits:
If handled and enacted effectively, this approach has the potential to generate case studies that can be used for subsequent teaching and training.
Assignment remits can be developed such that all students in the cohort do not develop a case study focusing on the same civil case. Each group could be required to develop a different case study thereby resulting in numerous potentially usable case studies. If this approach is adopted, students might be required to develop initial ideas which are approved by the academic prior to the full development of the case study, this is to avoid duplicate scenarios.
If students are organised as groups, the scope of the resulting case study can be larger than if the case study had been generated by the academic.
This approach has a number of learning benefits. By participating in this process, students engage in the thought process of the criminal/guilty party. Students must understand the 'evidentiary consequences' of their case study, i.e. they must ensure that certain evidence appears in certain locations on the destination hard disk.
There are however some distinct challenges with this approach:
The process is nevertheless a time consuming one and may not be easily achievable in smaller (15-20 credit) single semester modules.
The approach is hardware intensive, particularly so for large cohorts. This could be subverted by developing the case study within a VME platform. This proposal was suggested by Kessler and Schirling [2006], however little further development in terms of exploring this approach seems to have been done. The approach taken by Carling et al., [2005] is one whereby students are given an external hard disk and required to generate the case study on a partition on the hard disk, the windows installation is on a laboratory machine. This approach is interesting in that it does not involve investigating the operating system derivatives of the case study and might have little scope for the generation of noise.
Whilst the secondary aim of the assessment might be to generate a usable case study, one must consider that although students are an excellent academic resource they may not always yield a usable case study. Therefore, academics may have to resort to the self-generation of case studies.
3.2
Academic Developed Case Studies
At this juncture it is useful to refer to a case study developed at the University of Derby by the author and a member of the faculty technical team. The aim was to develop a case study that could be used to teach and assess holistic digital investigation skills. The remit of the case study was agreed in advance and a limited degree of planning took place. The planning included an agreement of general case study actions but not finite instances of interaction with the desktop system (referred to herein simply as an event).
Case study development took place over two months and involved two dedicated networked Windows XP desktop machines each administered by the author and the technician respectively.
The case study centred around the director of engineering in a car manufacturing company who having met a senior design engineer (SDE) in another car manufacturing company sought to encourage the SDE to join his company. During the course of their communications, the SDE is alleged to have supplied current engineering design plans. The case subsequently involves an internal investigation by the respective organisations and involve an IPT claim by the SDE's company.
The majority of evidentiary artefacts within the case study were based on e-mail communications. A number of web-sites were accessed in parallel to the e-mail communication, this access was designed to correlate information and communication within the emails.
A number of problems were found during the enactment of the case study plan:
Clearly the case study had to be realistic, there had to be a series of sporadic communications which needed to take place throughout the day. Due to work commitments by the author and the technician, this proved difficult to maintain over the two month period and as the development of the case study progressed, communication between the two fictional characters tended to be confined within certain hours of the day. In the context of the case study this was unrealistic and limited it's investigative potential.
The OS installations were new and there was no noise in the case study, the lack of noise would limit the investigative potential of the case study.
Often it became difficult to 'think and behave' in the manner of the two suspects. This was difficult but essential.
Whilst regular backups of the case study were maintained, there was always potential for particular actions not to be easily reverted. For instance it was not easy to revert incorrect events (in the context of the case study) which took place after the previous backups had been made - particularly if there had been a large gap (in terms of events) between the backup and the event. If the previous backups were reinstated on one machine, they also had to be reinstated on the second machine. All subsequent events up to the point of the 'mistake' had to be re-enacted.
Whilst the original aim of the case study was to yield two images that could be used to train and assess holistic skills, the resulting DF Images were restricted to being useful for training and assessing specific skills (namely e-mail investigation). For this purpose they have proven reasonably useful.
4
Guidelines to Developing a case study
The experience at Derby directly gives an important lesson which can be used for both the self development and student development of case studies. The activity built into case studies is likely to be conducted within certain time periods determined by when the participants can spare time to work on the case study, the interaction can therefore lack spontaneity.
Furthermore, if the development of the case study is improperly planned it can become overly time consuming. Some of the questions to be asked during the planning phase are:
What is the case study remit, i.e. what is it that the suspects will be accused of?
What is the skill level of the suspects? This will help to determine the depth of the case study, for instance is it necessary to implement anti-forensics techniques within the case study?
What kind of UBP would the combination of suspects and the incident (realistically) generate? This will influence the particular events that need to be generated.
Story-boards have been used for many years in the planning of film/video production. The nature of case study planning lends itself to being better managed through a story-board approach. Key issues pertinent to the case are recorded within the story-board which can be developed as a series of two way communicational dialogues clearly outlining the sequence of events that must take place within the case study. These events have to be recorded to finite detail without necessarily defining the precise detail of each communication.
Further to this, a series of technical issues need to be addressed:
What hardware platform will the case study operate under? Will it for instance involve mobile phones and external storage devices?
What applications will be installed, if so are these readily available?
Will the case study involve a fresh operating system install? How will backups be managed?
How will noise be generated?
Will e-mail accounts need to be configured?
5
Further Study
This research has led to a number of areas for further research:
Using VME as a platform for the development of case studies. A VME can provide an easily manageable and controlled environment within which the case study could be developed. This has numerous benefits, in particular less reliance on a dedicated hardware platform and flexibility in the design of a software/hardware platform within which the case study operates.
Methods for rapid noise-generation. Are there methods or techniques that can be adopted to develop noise rapidly?
Further to this, there is scope for the academic community to develop and contribute towards a repository of case studies.
References
1. NIST: website, http://www.cfreds.nist.gov/ (visited 26th March 2010) 2. Brian Carrier, website, http://dftt.sourceforge.net/ (visited 13th April 2010) 3. CFTT, website, http://tech.groups.yahoo.com/group/cftt/ (visited 13th April
2010)
5. The International Society of Forensic Computer Examiners, website, http://www.isfce.com/ (visited 13th April 2010)
6. Digital Corpora, website, http://digitalcorpora.org (visited 13th April 2010) 7. Kessler, G.C., & Schirling, M.E. (2006). “The Design of an Undergraduate
Degree Program in Computer & Digital Forensics”. Journal of Digital Forensics, Security and Law, 1(3), 37-50.
8. Carlin, A., Curl, S., and Manson, D. (2005). “To catch a thief: Computer forensics in the classroom”. In Proceedings of the 22nd Annual Information Systems Educators Conference (Columbus, OH, Oct.), Association of Information Technology Professionals, Chicago, IL.
Acknowledgments
Jamie Morris (www.forensicfocus.com); Sam Salt (University of Derby); Philip Anderson (University of Northumbria).
