Executive Summary. Methodology

(1)

This document describes test procedures for Diskeeper® 2011 evaluation. Specifically, the tests described in this document are intended to achieve the following objectives: 1. Evaluate the effectiveness of Diskeeper 2011 in preventing and removing disk

fragmentation.

2. Evaluate system performance improvement from using Diskeeper 2011. 3. Evaluate impact of Diskeeper 2011 on system resources while defragmenting

disks.

4. Evaluating the effectiveness of Diskeeper 2011 in making systems more energy efficient.

Note that Diskeeper 2011 offers the revolutionary Instant Defrag™ technology, which immediately eliminates any fragmentation not proactively prevented by IntelliWrite® fragmentation prevention. Virtually any task performed on the system will benefit from the combination of these two features. The first test documents are an example of an IntelliWrite & Instant Defrag test.

Diskeeper 2011 also introduces Efficient Mode, a method of automatic

defragmentation that ensures maximum disk optimization with minimum resource overhead.

Methodology

ReadFile is employed to evaluate the system performance improvement from using Diskeeper 2011. ReadFile.exe, a file access time measurement tool, measures the length of time taken to open, read all of the clusters allocated to the targeted file, and then close it. Results are expressed in milliseconds (msec), and it also calculates the Transfer Rate as expressed in Kb/sec.

Futuremark’s PC Mark Vantage benchmark tool is used to measure the impact

Diskeeper has on system resources while defragmenting disks. PCMark Vantage uses a proprietary scoring system that measures computer performance across a variety of common tasks such as viewing and editing documents, photos, video, music and other media, gaming, communications, productivity and security (higher scores are better). Specifically, the Hard Disk Drive (HDD) test suite is used to evaluate disk system performance. The tool, as well as relevant documentation, can be obtained from the following location.

(2)

Company Private Page 2

In order to successfully document the potential performance gains provided by Diskeeper you must have the following tools:

A) A computer system running Microsoft Windows (XP, Vista, or Windows 7) with a separate test volume.

B) A suitable tool or utility such as FragmentFreeSpace.exe (see attached ZIP file) to cause fragmented free space and or fragmented files. After all, Diskeeper is a solution designed to eliminate the slowdowns caused by fragmented files and free space. Without fragmented files and free space you really don’t have an environment suitable for testing.

C) Diskeeper software installation package.

D) Disk benchmarking performance software, such as SQLIO, SQLIOSIM, & IOMETER.

Fragmentation Prevention and Elimination

The test should be performed on a system that has been exposed to typical usage in a given environment (i.e. a server that was in production for some time). Such a system would already have a significant level of fragmentation. If such a system is not readily available, a utility called CreateTestDisk provided by Diskeeper can be used to create test disks with various levels of fragmentation. Once a test disk is selected (or created), proceed with installing PC Mark Vantage, ReadFile, and Diskeeper 2011 on it.

When installing Diskeeper, make sure to disable IntelliWrite and automatic

defragmentation. This is so that the state of the disk can be analyzed before running defragmentation. After completing all installation steps, make sure to save the image of the test volume for repeated tests (using an imaging product like Norton Ghost). Images are used so true comparisons can be made. The test procedure is as follows:

1. Start this test with restoring the image of the fragmented test volume. 2. Enable IntelliWrite and automatic defragmentation from the Diskeeper UI. 3. Perform typical production activity on the system (i.e. Web browsing,

downloading and creating mail, editing documents, etc.). The test may include installing several system updates. Continue using this system for an extended period of time (at least several hours).

a. NOTE: If Microsoft Office is available, we can provide workload scripts which will automate typical productivity from the MS Office suite. 4. Open the Diskeeper UI and observe statistics in the Dashboard. Notice the

number of fragments prevented by IntelliWrite.

5. After running the product for more than 24 hours observe the number of

fragments prevented and eliminated in the History tab. The History tab will show you what fragmentation has been prevented (including fragmentation that was handled immediately by Instant Defrag) and eliminated over time, so you can see the benefits over a longer period of time.

(3)

SYSTEM PERFORMANCE IMPROVEMENT

The following applications are used to benchmark performance in this test: 1. Iometer (mimics file server/database server)

2. SQLIO (SQL server) 3. SQLIOSIM (SQL server)

4. Microsoft Windows Performance Monitor 5. Diskeeper 2011 EnterpriseServer

Note that it is important to use proper NTFS clusters sizes (64K for database applications such as SQL), and set a proper HBA queue depth. For SQL tests, a third volume can also be created to store log/transaction files, though this will not affect the outcome of the testing.

Environment Setup:

1. Install ALL Windows-based tools and applications to C: drive

2. When installing Diskeeper, make sure to disable IntelliWrite and automatic defragmentation (they are both enabled by default). This is detailed with graphics below:

 s.

sDiskeepersws IWs(swq) .

(4)

Company Private Page 4 vsv(s)sIW IWv.

Disabling these automatic real-time operating features of Diskeeper are done so that a steady state of the disk can be analyzed.

Benchmark Applications Overview

Diskeeper:

Diskeeper is primarily used to collect fragmentation reports for all test cases.

IntelliWrite and Automatic Defragmentation should be disabled for all tests, except as specified in Test Case #2 and #3.

Perfmon:

Perfmon can be called up from the command line via typeperf.exe. Counters of interest include Split IOs, Disk Reads/sec & Disk Writes/sec, Average Disk/sec Read & Average Disk/sec Write, Average Disk Bytes/Read & Average Disk Bytes/Write, Average Disk Queue Length, Disk Read Bytes/sec & Disk Write Bytes/sec, as well as CPU and RAM statistics.

Perfmon is use in these tests as a compliment to the benchmarking tools. For

production environments, PerfMon should be used in lieu of any third party benchmark tools referred to herein.

(5)

Iometer (http://www.iometer.org/):

Iometer is the industry standard I/O measurement and characterization tool to measure system performance. It creates a single test file; w.s. The size of this file can be configured, with a suggested size for these tests of 5GB.

It offers a wide variety of options to mimic reads and writes. It is best suited for

replicating file server activity, but particular patterns can be used to represent access to a database (e.g. see here: http://www.bluesmoke.net/viewArticle.cgi?id=g6&page=6) When using Iometer, it is important to set up numerous “worker threads” to simulate a production level use case with multiple concurrent reads and writes.

(6)

More importantly are the detailed logs it produces, which can easily be compared (via Excel).

Example report on Fragmented File:

Example report on Non-fragmented File

SQLIO:

SQLIO is CLI only. It runs definable R/W patterns to a definable sized file. It provides IOPS, throughput and latency information. Full use information is included in Using SQLIO.rtf file located on the program root folder (part of the installation files).

The param.txt file dictates the volume the test file will be created on and the size of the file.

Example string inside param.txt: “d:\testfile.dat 4 0x0 100” x:

Below are some options. The “LS” parameter is vital. Also important is to use a larger IO request. Experimentation of various patterns will be required.

Option Description

-o Specify the number of outstanding I/O requests. Increasing the queue depth may result in a higher total throughput. However, increasing this number too high may result in problems

(described in more detail below). Common values for this are 8, 32, and 64.

-LS Capture disk latency information. Capturing latency data is recommended when testing a system.

(7)

-s Duration of test (seconds). For initial tests, running for 5-10 minutes per I/O size is recommended to get a good idea of I/O performance.

-b Size of the IO request in bytes.

-f Type of IO to issue. Either ‘random’ or ‘sequential’.

-F Name of the file which will contain the test files to run SQLIO against.

In order to automate the execution of SQLIO, create a batch file with the following text (and save it as SQLIO-batch-routine.bat):

::Test random IO's of 8K and 16K

call sqlio -kW -s600 -frandom -o32 -b8 -LS -Fparam.txt call sqlio -kR -s600 -frandom -o32 -b8 -LS -Fparam.txt call sqlio -kW -s600 -frandom -o32 -b16 -LS -Fparam.txt call sqlio -kR -s600 -frandom -o32 -b16 -LS -Fparam.txt ::Test sequential writes

call sqlio -kW -s60 -fsequential -o1 -b4 -LS -Fparam.txt call sqlio -kW -s60 -fsequential -o1 -b8 -LS -Fparam.txt call sqlio -kW -s60 -fsequential -o1 -b16 -LS -Fparam.txt call sqlio -kW -s60 -fsequential -o1 -b32 -LS -Fparam.txt call sqlio -kW -s60 -fsequential -o1 -b64 -LS -Fparam.txt call sqlio -kW -s60 -fsequential -o1 -b128 -LS -Fparam.txt x: 

(8)

As an example, to validate that fragmentation affects performance, the test below shows stats on a 64KB cluster using 4KB reads and writes (improvement of 7.5% writes, and 26.6% reads).

SQLIOsim (http://support.microsoft.com/kb/231619):

Creates and expands SQL files, runs preset SQL pattern reads and writes. The start and ending size of the databases can be adjusted, but the read/write patterns cannot. It is important to ensure the tool is set to write its files to the D: drive for these tests.

SQLIOsim.com is CLI version and provides means to automate launch and reports. Use the /? Parameter to determine options.

(9)

Manually running the tool provides reports in the form shown below:

********** Final Summary for file C:\sqliosim.mdx **********

CLogicalFile::OutputSummary

Display Monitor File Attributes: Compression = No, Encryption = No, Sparse = No

Display Monitor Target IO Duration (ms) = 100, Running Average IO Duration (ms) = 78, Number of times IO throttled = 6167, IO

request blocks = 7

Display Monitor Reads = 16258, Scatter Reads = 29897, Writes = 956, Gather Writes = 25007, Total IO Time (ms) = 38661523

Display Monitor DRIVE LEVEL: Sector size = 512, Cylinders = 73444, Media type = 12, Sectors per track = 19, Tracks per Cylinders

= 224

Display Monitor DRIVE LEVEL: Read cache enabled = Yes, Write cache enabled = Yes

Display Monitor DRIVE LEVEL: Read count = 48539, Read time = 2613438, Write count = 40611, Write time = 37398658, Idle time

= 21180, Bytes read = 8329734144, Bytes written = 6727626752, Split IO Count = 33, Storage number = 2, Storage manager name = VOLMGR

By fragmenting the database files created, expect to see degradation in performance stats from this tool over an identical environment where the database files are not fragmented.

(10)

Test #1 – Fragmentation impact on SAN disk performance

In order to save time (the time it would take to defragment – which can be many hours), the benchmarks are first run on a freshly formatted volume. This ensures minimal or no fragmentation of the benchmark test file(s). A tool is then used to fragment the file, and the benchmarks are then re-run on those now fragmented files. At the end, the results can be compared to validate that fragmented files incur performance penalties. The test procedure is as follows:

1. Run each benchmark utility to create the necessary workload test files.

a. For IOMETER, create a 5GB (iobw.tst) test file. We suggest using the All-in-one Access Pattern which offers a variety of random and sequential reads and writes.

2. Once complete. From Diskeeper UI perform a manual fragmentation analysis of the test volume. Save the Job Report. At this point the test volumes should have little to no fragmentation.

(11)

Company Private Page 11 svw sss.Svsssw .

3. Simultaneously run all benchmarking tools (Perfmon/typeperf, SQLIO or

SQLIOsim, and Iometer) on server. Test files must be written to a separate test volume in all cases. All benchmark tools (including Perfmon) should be set to run for the same two hours for parity. A batch command is preferred, but manually launching the tools successively within a few minutes will suffice (note that pattern will need to be reproduced for the “fragmented test case” in the second half of this procedure).

Simultaneously Launch: a. Perfmon

b. SQLIO should be launched from the MS-DOS Command Prompt window using a batch command file that performs various read/writes. Increase the time frames for most of the read/writes tests to run for 6 minutes each so the total time running is approximately 2 hours.

c. SQLIOSIM. Again, set the time duration to match the 2 hour time. It is also important to uncheck_{the box so the test database files are not}

deleted on startup or completion of the test.

(12)

e. IOMETER. Set it to run for 2 hours and select the “All in one” test pattern that does a variety of different sized read/write request patterns.

4. Once completed record the results.

5. Next from Diskeeper UI perform a manual analysis of the test volume. Save the Job Report which shows the fragmentation state of the volume and the

degradation it is causing on the read access times on the volume. After the tests complete, run DiskView again to verify the levels of Windows NTFS file

fragmentation. The files will be in the same condition as before (little to no fragmentation) and copyd down the resulting log files showing the performance results.

6. Now the test files that are used must to be fragmented. Run the FragmentFile.exe tool against each of the workload files. The syntax is:

FragmentFile.exe _ 1 1. Iobw.tst (5GB) from Iometer.

2. Sqliosim.mdx (~4GB) and Sqliosim.ldx (~50MB). 7. Reboot the server.

8. Now that the files are fragmented we repeated the same exact tests as documented above.

a. Once completed record the results.

9. Delete the LUN. Ensure the SAN’s cache is cleared (may require a LUN dismount of sorts), otherwise the comparison result will be skewed.

10.Use Diskeeper to record the amount of fragmentation on the server (save the reports).

11.Now compare the before and after data from Iometer and the analysis done by Diskeeper.

(13)

SYSTEM RESOURCE IMPACT: EFFICIENT MODE WITH INVISITASKING

1. Start this test with restoring the image of the fragmented test volume.

2. Run the PC Mark Vantage HDD test suite and collect the score. Repeat this test 4 more times for a total of 5 iterations.

3. Confirm that Efficient Mode is selected in Diskeeper options (it is the active defragmentation mode by default). Enable automatic defragmentation on the test volume from the Diskeeper UI. In this mode, Diskeeper uses InvisiTasking® technology to ensure there is no impact on user and system applications while Diskeeper is optimizing the disk performance. The next step demonstrates this. 4. Shortly after starting automatic defragmentation, close down the Diskeeper UI

and run the PC Mark Vantage HDD test suite again (also 5 iterations) while Diskeeper performs defragmentation using InvisiTasking.

Compare the PC Mark Vantage performance scores before enabling defragmentation and while running defragmentation in the background, to evaluate the impact of Diskeeper on other applications while it is processing the volume. This shows the magic of InvisiTasking.

DISKEEPER SAVES ENERGY

This evaluation shows how Diskeeper makes system more energy efficient by lowering the power consumption of systems. This is done by measuring power consumption with and without Diskeeper running on the system.

1. This test requires an Energy Measurement device (examples: watts up?Power Analyzer, Yokogawa WT210) that connects between the computer system and the power source. It periodically measures and records the energy consumption of the system. We can send you details of the device that we used and send you the device for use if needed.

2. Start this test with restoring the image of the test volume from step 9 of the Automatic Defrag test. We start with a clean test volume as we are measuring the power consumption savings that Diskeeper provides by handling

fragmentation that is occurring from a simulated workload over a 24 hour period.

3. A utility was created to simulate a workload on the system. This includes disk activities of creating, writing, and deleting files (we can provide this utility to you). This utility is run every hour and is run for a 24 hour period.

4. During this time, the Energy Measurement device is recording the power

consumption used every hour. When the 24 hour period is complete, the power consumption data is saved to a spreadsheet:

(14)

5. Repeat test steps 2-4 again, but in step 2, after restoring the test image, uninstall Diskeeper.

6. Compare the power consumption of the system, with and without Diskeeper.