Guidance for Best Practices in Information Security and IT Audit

(1)

Guidance for Best

Practices in Information

Security and IT Audit

(2)

Executive Summary

Practices Covered 2

Key Findings 2

Only One-in-Ten Experience the Best Outcomes 2

Some Practice Guidance Drives Better Outcomes 2

Baseline Practices for Better Outcomes 3

Top 10 Percent: Top 10 Practices 3

Outcome-based Practices 4

Practice Improvements Pay Off 4

Guidance for Best Practices 4

Organization of the Report 5

Benchmark Findings

Distribution of Outcomes 6

Practices and Better Outcomes in IT 6

CobiT, ISO and Practices for Better Outcomes 7

Which Practices Work 8

Differences in Implementation of Practices by Domains 9

Significant Gaps for Individual Practices 9

Baseline Practices for Better Outcomes 11

Practice Improvements Pay Off 12

Top Line Business Results 12

Audit Expenses 13

Time and Labor Costs for Audit in IT ` 14

Business Downtime 15

Financial Exposure and Loss: Data Loss and Theft 16

Taking Action to Improve Outcomes 17

Appendix A: Guidance for Practices 18

Appendix B: Practices Tested 23

About the Research 24

(3)

Executive Summary

Practices Covered

This report covers benchmarked practices within information security and IT audit functions across more than 3,000 organizations that are directly related to better operational and financial outcomes, for managing:

 The integrity of information  Compliance with regulatory audit  Business risks related to the use of IT

In addition, guidance for practices related to information security practices and procedures, and information security policies are also covered in this report.

More than 100 benchmarked practices for information security and IT audit are grouped into 12 practice domains, including: managing communications about directions and aims; managing IT processes, the organization and relationships; managing the information architecture; managing human resources; managing information and data; managing operations; ensuring systems

security; monitoring and evaluating; managing quality; managing risks and governance.

Key Findings

Only One-in-Ten Experience the Best Outcomes

Operating outcomes related to IT

 About 1 in 10 organizations (12 percent) experience the best outcomes for information security and IT audit with the lowest levels of data loss or theft, least business disruption and fewest problems with audit

 A majority of organizations, nearly 7 in 10 (69 percent) are experiencing higher rates of data loss or theft, higher levels of business disruptions from IT failures, and more difficulty with passing regulatory audits in IT

 Almost 2 in 10 organizations (19 percent) experience the worst outcomes with the highest rates of data loss or theft, the highest levels of business downtime, and the most difficulties passing audits in IT.

Financial outcomes among best performing organizations

Organizations with the best operational outcomes are also experiencing better financial outcomes, including:  Fifty percent less money being spent on regulatory audit each year

 Fifty percent less time spent supporting audit in IT

 Financial exposure and loss from customer data loss or theft is the lowest  Customer retention rates are typically six percent higher

 Annual revenue is uniformly eight percent higher  Profits are six percent higher

Some Practice Guidance Drives Better Outcomes

The best performing organizations uniquely rely on primary and secondary forms of guidance to inform and adjust information security and IT audit practices for better results, including:

Primary practice guidance: CobiT and COSO

The use and implementation of CobiT and COSO guidance for information security and IT audit practices is between 20 and 30 times higher among organizations with the best outcomes.

Extent of practices covered

 Communications about management aims and directions

 Managing IT processes, organization, relationships

 Managing the Information architecture

 Managing human resources

 Acquiring and managing IT assets

 Managing information and data

 Managing operations

 Ensuring systems security

 Monitoring and evaluating

 Managing quality

 Managing risks

(4)

Secondary practice guidance: ISO, PCI, NIST and regulatory audit mandates

Supplementary practice guidance from ISO, PCI, NIST and regulatory mandates to guide information security and audit practices is five to ten times higher, even among organizations not subject to PCI and FISMA audits.

Baseline Practices for Better Outcomes

Baseline Best Practices

Twelve key-practices, one in each practice domain, are uniformly implemented for information security and IT audit by organizations with the best outcomes. These twelve are joined by the top 10 practices making the most difference in outcomes.

Top 10 Percent: Top 10 Practices

Of 104 of the most common practices for information security and IT audit tested by the benchmarks (Appendix B), ten percent of these practices are most responsible for the disparity in outcomes between best performing organizations and all others.

Although overlap exists between the baseline and top 10 practices, some baseline practices are unique and as such can be considered a minimum set of practices. Some of the unique baseline practices include:

 Distributing IT policies for exceptions and adoption throughout the organization  Maintaining an updated list of IT assets for managing outcomes

 Controls implemented to detect and prevent sensitive data from leaking

 Automating the detection and prevention of unauthorized changes to critical IT assets

While most baseline and top 10 practices are commonsense, there is nothing common about their

implementation: only four-to-six of these practices are implemented by organizations with normative outcomes, while fewer than three are typically implemented by organizations experiencing the worst outcomes (Table 1). Table 1: Baseline and Top 10 Practices

Baseline Best Practices in Domains Top 10 Practices

Distribute IT policies for exceptions and adoption Frequent control assessments Manage information security from a non-IT

operations role Protecting IT security data

Protect information security, IT audit and customer

data Using common controls between IT security and IT audit

Deliver training to employees and contractors Automating the majority of procedural and technical controls

Maintain an updated list of IT assets Conducting self-assessments of procedural controls

Use controls to detect and prevent sensitive data from leaking

Improving IT controls Automate detection and prevention of unauthorized

changes to critical IT systems Protecting IT audit data

Continuously monitor critical IT assets and verify

against IT security standards Protecting customer data

Conduct daily and weekly assessments of critical IT technical controls

Reviewing changes against IT policies and standards

Conduct monthly assessments of procedural controls

Managing information security from a non-IT operations role

Define business risks based on objectives and standards for integrity, availability and

confidentiality

Conduct ongoing assessments of business conditions, risks and controls

(5)

Outcome-based Best Practices

The findings contained in this report are based on practices that are implemented by organizations posting the best outcomes. The benchmarked outcomes used to identify the practices of the best performing organizations include:

 Lowest losses or thefts of customer and other sensitive data  Most resilient IT systems and highest IT service levels  Least problems with audit in IT

The findings contained in this report are based on a few simple

measures: 1) organizations with best, normative and worst outcomes are identified; 2) practices are grouped by outcomes and 3) differences by size, industry and practices are compared to identify the most impact to outcomes.

Practice Improvements Pay Off

Improving practices for information security and IT audit pay off. Whether measured by lower financial loss and risk from lost or stolen customer data, more resilient systems and less business downtime, less money spent on audit, or less time spent in IT to support and maintain audit results, the financial incentives for making improvements to information security and IT audit practices are multifaceted and compelling (Table 2).

Table 2: Practice improvements pay off Worst practices (2 in 10) Normative practices (7 in 10) Best practices (1 in 10)

Customer retention, revenue

and profit -8 % to -12% lower 0% 6% to 8% higher

Annualized loss rates for data

loss and theft of revenue1% to 7% 0.2% to 2% of revenue 0.1% to 0.2% of revenue Range of business downtime

from IT problems 60 hours and more 4 to 40 hours Less than 4 hours

Spend on audit 75% of norm 100% 50% of norm

Time spent on audit 80% of norm 100% 50% of norm

Source: IT Policy Compliance Group, 2009

Guidance for Best Practices

Organizations with the best outcomes implement the baseline and top 10 practices, while using practice guidance from CobiT and COSO to a degree that is not matched by other organizations. Guidance for best practices includes:

 Implement the missing — or partially implemented — baseline and top 10 practices to improve outcomes  Adopt CobiT and COSO as primary sources of guidance for information security and IT audit practices  Adopt ISO, PCI, NIST, vendor guides, CSI benchmarks and regulatory mandates as secondary sources of

guidance

For some organizations, implementing the baseline and top 10 practices may simply be augmenting existing practices or adding the few that are missing from current standard operating procedures.

However, for other organizations, implementing the baseline and top 10 practices may involve changes to current practices and the addition of practices that are not currently part of standard operating procedures.

Outcome-based best practices

 Lowest losses of sensitive data

 Lowest business downtime

(6)

Guidance for Implementing Best Practices

Based on the findings, guidance for implementing practices for better outcomes includes establishing objectives and standards, orienting the organization, and implementing the baseline and top 10 practices (Table 3).

Table 3: Guidance for implementing best practices Guidance

Establish objectives Define unacceptable business risks related to the use of IT Focus on core business risks and objectives for controls

 Fewer than 30 is optimal Establish

standards

Define internal standards for integrity, availability and confidentiality of information and IT assets.

Define objectives for controls to manage business risks related to the use of IT.

Organizational structure _{Manage information security from a non-IT operations role}

Practices Implement the baseline practices

Implement the top 10 practices

Employ guidance for practices, including: 1. CobiT and COSO

2. ISO and PCI

3. ITIL, vendor guides, NIST, CIS benchmarks, regulatory mandates Protect critical assets

and information Protect critical information, especially IT security data, IT audit data, and _{customer data}

Continuous assessment and reporting

Conduct very frequent assessments of the effectiveness of controls against standards for integrity, availability and confidentiality

- Once per week for sensitive data

- Once bi-weekly for IT general controls

- Once per month for procedural controls

Use common controls between IT audit and information security for

- Backup, archive and data quality tests and reports

- Procedural and technical tests and reports on controls

- Audit and IT security tests and reports

- Procedural and technical controls

Organization of the Report

There are four sections to this report, as follows:

Executive Summary: contains the key findings of the research and recommendations for best practices for information security and IT audit.

Benchmark Findings:covers in detail the principal benchmark findings for practices as these relate to outcomes, and steps to consider taking to improve practices for information security and IT audit.

Appendix A:includes detailed findings related to guidance employed by organizations for managing the integrity of information, managing information security practices and procedures, managing compliance with regulatory audit, managing information security policies, and managing business risks related to the use of IT.

Appendix B:contains the list of 104 of the most common practices for information security and IT audit tested by the benchmarks, and aligned by outcomes across more than 3,000 organizations

(7)

Benchmark Findings

Distribution of Outcomes

The best performing organizations share several characteristic outcomes that set their practices apart from all other organizations. These characteristics include the:

 Least loss or theft of sensitive information

 Lowest business downtime due to IT failures or disruptions

 Fewest audit deficiencies in IT

However, best performing organizations represent just slightly more than 1 in 10 organizations (12 percent). In contrast, nearly 7 in 10 organizations (69 percent) are experiencing normative outcomes with higher rates of data loss or theft, increased levels of business downtime due to IT failures or disruptions, and elevated levels of audit deficiencies in IT that must be corrected to pass audit.

Unfortunately, nearly 2 in 10 organizations (19 percent) are suffering from the highest levels of data loss or theft, the worst business disruptions from IT failures or disruptions, and the most difficulty maintaining and passing regulatory audits in IT (Figure 1).

Figure 1: Distribution of outcomes

The most recent benchmark results covering practice guidance conducted with 319 organizations are nearly identical to operating and financial outcomes being experienced by the larger population of more than 3,000 organizations. A few key results show that although there are slight skews in outcomes based on size and industry, the primary 2-in-10, 7-in-10, and 1-in-10 distribution of outcomes (worst, normative and best) is primarily due to practices implemented for information security and IT audit.

Moreover, the research findings reveal similarity of outcomes across all three measures. For example, almost all (97 percent) of the organizations with the least loss or theft of sensitive information are the same organizations with the least business downtime and the fewest problems with audit deficiencies in IT. A majority (93 percent) of the organizations with normative outcomes for data loss or theft are the same organizations with normative outcomes for the other two. Lastly, a majority (76 percent) of the organizations with the worst outcomes for audit deficiencies are the exact same organizations with the highest rates of data loss or theft, and the highest level of business impact due to IT failures and disruptions.

Practices and Better Outcomes in IT

The findings show that almost all organizations employ an empirical approach to assessing and reporting on the effectiveness of controls, using IT audit and IT security test reports as a basis for evaluating the integrity of information, the integrity of practices and controls, for managing regulatory audit, managing business risks related to the use of IT, and for managing information security policies. However, the characteristics unique to best performing organizations include

(8)

well defined internal standards for two primary focus areas: 1) confidentiality, integrity and availability standards, and 2) unacceptable business risks related to the use of IT (Appendix A).

In addition, best performing organizations uniquely employ CobiT and COSO guidance to inform and guide practices, while supplementing these with guidance from ISO, PCI, NIST, vendor guides and regulatory mandates (Table 4). Although supplementary forms of guidance from ISO, PCI, ITIL, vendor guides, CIS benchmarks, NIST guidelines and regulatory mandates are all employed, CobiT and COSO are used for practice guidance by 10, 20 and 30 times more of organizations experiencing the best outcomes (Appendix A).

Table 4: Information security and audit practices and guidelines for better outcomes

Practice area

Table stakes for all organizations

Primary practice guidance unique to best

performers

Supplementary practice guidance unique to best

performers

Managing the integrity of information

Backup, archive and

data quality test reports CobiT andCOSO ITIL and vendor guides

Managing information security practices and procedures

Procedural and technical controls assessments, audit and IT security test

reports

CobiT, COSO, PCI

NIST FIPS, FISMA, ITIL, vendor guides, CIS

benchmarks Managing

compliance with regulatory audit

Procedural and technical controls assessments, audit and IT security test

reports

CobiT, COSO

PCI and ISO ITIL, NIST, FIPS, FISMA, vendor guides

Managing information security policies

Internal standards for confidentiality, integrity

and availability, legal counsel and advice

Unacceptable business risks, vendor guides,

standards for confidentiality, integrity,

and availability

Regulatory mandates (PCI, GLBA, HIPAA, FISMA, SOX), Industry standards (ISO, CobiT, NIST, FIPS), legal advice and counsel Managing

business risks related to IT

Risk assessment reports, audit and IT

security test reports

CobiT, COSO, Octave,

RiskNav and similar tools NIST

CobiT, ISO and Practices for Better Outcomes

The benchmark outcomes of 3,280 organizations, aligned by outcomes, are mapped against specific practice guidance defined by guidance contained in ISO (ISO 17799) and CobiT (CobiT 4.1). The 104 practices selected (Appendix B) are the most common practices implemented by most organizations for information security and IT audit. Occupying more than 21 practices domains in ISO and CobiT, the 104 practices are simplified into 12 practice domains for ease of presentation.

The 21 domains include: communicating management aims and directions; managing the information architecture; managing IT processes, the organization and relationships, managing human resources, managing quality, assessing and managing IT risks, acquiring and maintaining the technology infrastructure; managing changes; enabling operation and use; installing and accrediting changes and solutions; ensuring continuous service; ensuring

systems security; educating and training users; managing the configuration; managing problems; managing data; managing the physical environment; managing operations; monitoring and evaluating internal control; ensuring compliance with external requirements; and providing IT governance.

Practices in some domains are not tested by the benchmarks. These include: managing the strategic IT plan; managing the IT investment; managing projects; identifying automated solutions; acquiring and maintaining application software;

Practice coverage

- Communicating management aims and directions

- Managing IT processes, organization, relationships

- Managing the Information architecture

- Managing human resources

- Acquiring and managing IT assets

- Managing information and data

- Managing operations

- Ensuring systems security

- Monitoring and evaluating

- Managing quality

- Managing risks

(9)

managing third party services; managing performance and capacity; and managing the service desk and incidents. As such, incomplete or untested benchmarks findings for these domains are not included.

Once aligned by outcomes, implementation levels for each practice are counted and totaled to provide the frequency of implementation: a percentile of implementation by outcome. The differences in the range of percentile implementations are grouped by the 12 domains, with the largest gaps and leading baseline practices for each domain identified.

Which Practices Work

As the benchmark results show, far more organizations with the best outcomes are implementing practice guidance contained in CobiT and COSO. Of the 104 most common practices tested, the findings reveal that on average,

organizations with normative outcomes are half as likely (42 percent to 58 percent) to implement the same practices as the best performing organizations, and with somewhere between 4 and 6 of the top baseline or best practices

implemented. It should be no surprise that organizations with the worst outcomes are one-tenth as likely to implement the same practices as the best performing organizations and with fewer than three of the baseline and top 10 practices. The range of practice implementations is striking in its direct relationship with outcomes being experienced by organizations (Figure 2).

The level of practice implementation by outcomes produces two clear findings, as follows:

1) Better outcomes are directly related to practice improvements for information security and audit, and 2) Some practices matter more than others

The evidence from the benchmarks clearly shows that to improve outcomes, most organizations with normative results will have to improve current practices. However, not all practices will have to be improved to the same extent. For example, among organizations with normative outcomes, some practices in managing the information architecture may already be close enough to the same practice implementation of the best performers, while other practices may have to be improved dramatically.

Figure 2: Range of CobiT and ISO domain practices, by outcomes

The range of practice implementation shows the worst performing organizations have the most ground to cover to improve practices and outcomes, while for a majority of organizations operating at the norm the amount of effort to improve

outcomes and practices is going be much less.

The actual practices that need to be improved depend on whether practices are implemented or not, and the level of implementation. Even the best performing organizations are not uniformly implementing all 104 of the most common practices. Rather than attempt to implement all 104 of the most common practices, it makes more sense to identify the practices most commonly implemented by the majority of the best performing organizations for comparison.

Ideally, a self-assessment comparing current practices and the most commonly employed practices of the best performing organizations will deliver prescriptive guidance into practices that are close enough to defer and practices most in need of

Outcomes and practices Better outcomes are directly related to

practice improvements for information security and IT audit: some practices matter more than others.

(10)

improvement. Gaps between the most commonly implemented practices by the best performing organizations will identify how close, or far away, current practices are to achieve best outcome levels.

Differences in Implementation of Practices, by Domains

Knowing which practices need to be improved, and by how much, as well as practices that can be deferred or ignored, is extremely helpful when aiming to improve outcomes and results.

Based on the benchmarks with 3,280 organizations, some stark differences in practice areas become readily apparent when the practices of organizations with different outcomes are compared with those being implemented by the best performing organizations.

When the practices in the domains are grouped, differences in practice implementations reveal two important points: (1) some practices are adequate enough, and 2) other practices are in need of more improvement, with some in need of the most improvement to replicate the success being enjoyed by organizations with better outcomes for information security and audit (Figure 3).

Figure 3: Differences in practice domain implementations

Source: IT Policy Compliance Group, 2009 Practice improvements among nearly best performing organizations

Organizations experiencing close to the best outcomes exhibit very little difference in practices when compared to the best performers. Nevertheless, small incremental improvements can be made to specific practices for monitoring and

evaluating, quality, governance and managing the information architecture to improve outcomes further. Practice improvements for organizations with normative outcomes

In comparison, most organizations experiencing normative outcomes exhibit much greater differences in the extent of practice implementations for information security and IT audit when compared with the best performers. The primary focus areas of improvement among normative performing organizations should be practices for monitoring and evaluating. After this, practices for governance, managing the information architecture, and managing quality contain the next most

deficient practices that need to be addressed to further improve results. Practice improvements for organizations with the worst outcomes

Organizations experiencing the worst outcomes related to information security and IT audit exhibit the largest differences in the extent of practices implemented when compared with the best performers. The primary focus areas of improvement among worst performing organizations include monitoring and evaluating, governance, managing the information

architecture, and managing quality.

Significant Gaps for Individual Practices

In addition to comparing practice domain implementation levels, significant gaps exist between the level of implementation of specific practices between most organizations and the best performers. For example, the practices with the most consistent significant gaps include:

(11)

 The frequency of controls assessments

 Employing common controls between IT audit and information security

 Conducting self-assessments of procedural and technical controls

 Automating procedural and technical controls

 Protecting IT security data

These five practices contain the largest gaps between the levels of practice implementation when compared with the best performers for almost 9 of every 10 organizations. The rank ordered list of the top “must improve” practices with the largest gaps compared to the best performing organizations changes as organizations approach “nearly best-in-class.” However for most organizations, the 9 in 10 not close-enough to best-in-class results, the top 5 practices only change order and are not as nuanced (Table 5).

Table 5: Prioritized practices, based on gaps to best outcomes Prioritized

improvements

to practices Organizations with the worst outcomes normative outcomesOrganizations with nearly best outcomes*Organizations with

1 _{Increase the frequency of}

controls assessments Increase the frequency of controls assessments Automate procedural and technical controls

2 Protect IT security data Protect IT security data Produce IT operational level quality reports

3 Conduct self assessments

of procedural controls

Use common controls between IT security and IT

audit

Produce Electronic dashboard reports

4 Automate procedural and

technical controls Automate procedural and technical controls Produce IT audit and IT security test reports

5 Use common controls

between IT security and IT audit

Conduct self assessments of

procedural controls Produce financial and business impact summaries

6 Protect customer data Improve IT controls and

procedures

Produce IT policy compliance reports

7 Improve IT controls and

procedures Protect IT audit and reporting data Use common controls between IT security and IT audit

8 Protect IT audit and

reporting data Protect customer data Produce financial and business impact summaries

9 Implement strong

information security and audit functions outside of the IT operations function

Review and communicate changes to IT policies and

standards Protect design data

Source: IT Policy compliance Group, 2009

Missing from the table of prioritized practices are important targets for practices that are being implemented by the best performing organizations. Covering a range of metrics collected by the benchmarks that include frequencies of control assessments, and the number of control objectives that are directly linked to audit costs and operating outcomes, the practice targets provide a way to assess how well practices — and outcomes — align with best performing organizations, and by how much practices may need to be improved.

Experience of one large organization

For example, a large entertainment organization with multiple business divisions went through the process of identifying all of its control objectives managing risk related to the use of IT. Across multiple business lines the organization found more than 120 different and distinct control objectives. The organization found duplication of effort and additional costs due to the number of control objectives. After a period of review, the organization whittled the list of 120 objectives to 30 that were common across multiple divisions.

After another period of business and financial review, the original 120 control objectives were reduced to 36 after the first year, and 22 in the second year. The 22 business risk control objectives are now making it possible for the organization to

(12)

be laser-focused in the procedural and technical controls needed to manage business risks related to IT, while instituting new practices to improve outcomes.

Most of the “best” practices identified by the benchmarks are now implemented by this organization with substantial reductions in audit costs, vast improvements in the protection of sensitive information, and much better assurance that information assets are better protected today than in the past.

As the experience at this organization illustrates, and the benchmark findings demonstrate, knowing which practices are more important than others needs to be accompanied by specific targets for practice improvements to yield better outcomes (Table 6).

Table 6: Practice Targets based on Best Outcomes

Practice Targets Focus

Once per week Sensitive information

Once bi-weekly

IT general controls (includes technical controls for applications, systems and information security)

Frequency of control assessments

Once monthly Procedural controls

Risk reporting _{Coincident with assessments} _{Core business, regulatory and legal}

risks Improvements to controls _{At least a 50/50 mix of procedural}

and technical controls. Higher levels of technical controls –75 percent – improve outcomes. Automation of controls _{At least 50 percent of controls and}

reporting fully automated

Higher levels of automation – 75 percent of all procedures and technical controls – improve outcomes

Common controls between IT

audit and IT security At least 60 percent of controls for IT _{audit should be identical to those} employed for information security

Higher levels of common controls – 80 percent and more – improve outcomes

Controls _{Key technical controls include user}

access permissions, critical systems, applications and information

Key procedural controls include IT change management, segregation of duties, and employee training Control objectives _{Reduce to core set of control}

objectives base don key business risks

If the list exceeds 30 core control objectives, it is too large

Baseline Practices for Better Outcomes

Irrespective of the largest gaps in implementation rates of practices, or specific practice targets implemented by

organizations with the best outcomes, there are baseline practices implemented by best performing organizations at levels far above most other organizations.

These baseline practices, by domain, include: distributing IT policies to relevant constituents for exceptions and adoption among employees and contractors; and placing responsibility for the management of information security into a

specialized role, including a chief information security officer (CISO), a chief risk officer (CRO) or a chief compliance officer (CCO) outside of IT operations. Whereas operations is goaled and rewarded by what happens, namely maintaining always-on operations for business purposes, the information security function is goaled and rewarded for what does not happen, namely avoiding losses in integrity to information and IT assets, and retaining confidentiality for critical information assets. Working across business operations, legal counsel, IT operations, physical plant security, regulatory audit, internal audit, and finance, the organizations with the best outcomes are consistently managing the information security function in a non-IT operations role.

Another baseline practice among best performing organizations is the use of automation to detect and prevent

unauthorized changes to critical IT assets. By preventing changes to assets and information that are violations of policy, the best performing organizations are eliminating problems and costs associated with recovering from problems that are more prevalent among organizations with less well developed practices.

Further baseline practices include controls being used to detect and prevent the leakage of sensitive data, consistent training delivered to employees and contractors, and continuous monitoring of critical IT assets to maintain adherence to policy.

(13)

An additional baseline practice includes an actively maintained inventory of all IT assets. Knowing what is on the network, and whether the devices and services are valid, and whether an unauthorized change occurred is one of the basic practices being implemented by all of the best performing organizations.

Although the baseline practices may not exhibit the largest gap between best performing organizations and others, these practices are either close seconds, or are the same practices.

As such, the baseline practices should be considered holistically as “best practices” and implemented, even if some of the top 10 best practices may take additional time to fully implement (Table 7).

Table 7: Baseline Practices by Domain

Practice domain Baseline practices in the domain

Management communications

Distributing IT policies to relevant constituents, policy adoption and enforcement

Managing IT processes and

the organization Managing information security from a non-IT operations role Managing the information

architecture Protection of information security, IT audit, reporting, and customer data Managing human resources Delivering training and awareness for employees

Acquiring and maintaining

information assets Maintaining an updated list of assets with continuous monitoring and reporting Managing information and

data

Maintaining controls to manage sensitive data from leaking

Managing operations Automating procedures to detect and prevent unauthorized changes to critical IT assets

Ensuring systems security Continuous monitoring of critical assets and verification with IT security standards

Monitoring and evaluating Daily and weekly assessments of critical IT technical controls Managing quality Monthly self assessments of non-automated procedural controls

Managing risks Defining unacceptable business risks related to IT based on organizational objectives for integrity, availability and confidentiality

Governance Ongoing assessment of changes in business conditions, risk profiles and controls.

Practice Improvements Pay Off

The benchmarks also track financial results being experienced by organizations, including when latent data loss or theft event rates result in financial loss, and loss experiences from lost or stolen customer data, including: capital reductions in shareholder value, direct expenses incurred for litigation, penalties, settlements, cleanup, restoration, customer

notifications, and credit services. Additional metrics tracked by the benchmarks include customer retention, revenue declines, profit declines, overall spend on audit, time spent by IT to demonstrate audit, the frequency of business disruptions due to IT failures or disruptions, and the length of business outages due to IT failures and disruptions. Given the large dataset, consistency of findings and verifications against publicly available data, the findings provide a fairly reliable picture of the financial ramifications for managing outcomes related to information security and IT audit practices, including:

 Top line results, including customer retention, revenue and profitability

 Annual audit expenses

 Time spent – labor – in IT on audit

 IT resiliency and business service levels

 Financial risk from data loss or theft

The next sections summarize some of the major financial findings related to information security and IT audit practices across the thousands of organizations measured by the benchmarks.

(14)

Organizations with the best practices for information security and audit consistently demonstrate the best top-line results when compared to peers and other organizations. When differences in revenue, profit and customer retention are aligned by outcomes for data loss or theft, business downtime occurring from IT disruptions, and audit deficiencies in IT, there is a consistent finding that shows better outcomes are aligned with better practices for information security and IT audit. Whether the metric is the retention of existing customers, revenues or profits, organizations with better information security and IT audit practices outpace all other organizations for top line outcomes (Figure 4).

Conversely, organizations with the worst practices consistently experience the largest declines in customer retention rates, revenues and profits than peers. Furthermore, an almost linear relationship exists between top-line results and the

effectiveness of practices for information security and audit.

Although it can be argued that better managed organizations consistently post better operating financial results with happier customers, the benchmarks do not measure the connection. Rather, the relationship measured by the benchmarks is between practices for information security and IT audit, and top line business results. Obviously,

improvements in revenue between 8 and 12 percent, or improvements in profits between 6 and 8 percent are not solely due to improvements in information security and audit practices.

However, it would be disingenuous not to recognize the relationship between customer retention, revenue, profit, and better protected customer data, or negative brand impacts associated with not protecting information assets and

information. Although information security and audit practices are most certainly not solely responsible for improvements in customer retention, revenue and profit, the benchmarks show a direct relationship between better practices and better results, even if the margin of contribution has not been benchmarked.

Figure 4: Top line business results and practices

Audit Expenses

The total amount of money spent on audit each year differs considerably by the industry and size of organizations: of the two, size plays the most important factor in how much is spent on audit. This is due to several factors identified by the benchmarks, including: more audits, more conformance and more reporting requirements driven by local geographic, political and legal requirements; an increase in the number of audits impacting IT; differences in audit requirements across geographies and regulatory regimes for different lines of business.

In addition to size, there are differences in total spend on audit by industry, with some industries more, or less, than what other industries spend on audit. However, when annual spend on audit is aligned by outcomes being experienced by audit deficiencies in IT, the relative amount spent on audit reveals a consistent parabola-looking relationship with maximum spending occurring at or near the mid-point for outcomes, where the majority of organizations (7 in 10) operate. The consistency of the findings, even within industries and similar size organizations include:

 Organizations with normative practices are overspending, by as much as two times more

 Organizations with the worst practices are overspending, by as much as one-third more

The findings reveal that for most organizations, the 7 in 10 operating in the normative range, the annual savings from reductions in rounds of testing, more focused testing based on a reduced set of objectives linked to core business risks

(15)

and regulatory requirements, reductions in consulting fees, and reduced internal labor costs associated with demonstrating conformance with audits can be as much as 54 percent annually.

Similarly, for 2 in 10 organizations with the worst practices, the annual savings on audit expenses can be as much as 33 percent annually (Figure 5).

Figure 5: Money spent on annual audit expenses by practices

The organizations with the best outcomes are uniformly spending much less on audit each year, with all of these best performing organizations being from different industries and of different size. The results clearly indicate that it is the practices that are responsible for lower spend in audit, not size or the industry of an organization. Even when

organizations of similar size in the same industry are compared, the same relative spending curve, based on practices, is replicated. The benchmarks confirm that improved practices in information security and audit result in lower audit fees. Although spend by industry cannot be changed for a given organization, and spend by size is unlikely to change for a given organization, lower spending on audit for any organization is directly related to the practices implemented for information security and IT audit. While impressive, achieving a 50 percent reduction in annual spend on audit is going to depend on several factors, including:

1) Implementing better practices for information security and IT audit 2) Near-term expense reductions from fewer rounds of audit testing 3) Longer-term reductions as contracts and labor expenses are reduced

Actual experience from the benchmarked organizations shows that improved practices for information security and IT audit are directly resulting in lower expenses for audit.

Time and Labor Costs for Audit in IT

The amount of time spent by IT to support audits depends on the number of audits impacting IT, the size and industry of an organization. Of the three, size is the most important factor. A narrow range of between 25 percent and 39 percent of time is spent in IT to support audit for most organizations, with an average of 32 percent being spent by IT to support audit. In addition, some variation occurs by industry, with some industries spending more or less time to support audits in IT than other industries. However, this variation is similarly constrained to a narrow band varying by 5 to 10 percent.

However, when the time spent by IT to support audit is aligned by outcomes and practices for information security and IT audit, the relative amount of time being spent to support audits in IT reveals a similar parabola with the maximum time spent to support audit in IT occurring near the mid-point, where a majority of 7 in 10 organizations are operating. Much like money spent on audit, time being spent to support audit in IT reveals:

 Organizations with normative practices spend almost twice as much time in IT to support audit

(16)

For most organizations, the 7 in 10 operating in the normative practices range, the annual savings from reductions in internal labor costs can be as much as 52 percent annually. Similarly, for 2 in 10 organizations with the worst practices, the annual savings on audit expenses can be as much as 30 percent annually (Figure 6).

Figure 6: Time spent to support audit by IT by practices

The research reveals mixed approaches are being used by organizations for the savings in time. For example, some of the best performing organizations are reducing operating expenses because skills are not transferrable, while many others are redeploying skills to improve practices and outcomes. Of the two approaches, the organizations spending the savings from IT audit on improvements to practices dominate among best performing organizations.

Business Downtime

Improving practices for information security and IT audit are accompanied by less business downtime from IT failures or disruptions. Whether the reason for business downtime is due to malware, losses of information integrity, or to natural events or loss of power impacting service-levels, the findings show organizations with the best practices experience much less business downtime (Figure 7).

Figure 7: Reductions in business downtime

(17)

The evidence indicates a close relationship exists between business service levels, operational resiliency and best practices for control objectives, controls, operational efficiency, and ultimately customer retention rates, revenue and profit.

Financial Exposure and Loss: Data Loss and Theft

Benchmarked financial loss, occurring after the loss or theft of sensitive customer is directly related to information security and IT audit practices. Improve the practices and the loss decreases and the likelihood of loss is lowered. Annualized loss exposure tracked by the benchmarks and actual industry experience reveals declining rates as practices in information security and IT audit are improved (Figure 8).

For example, an organization of $1 billion in annual revenue with the worst information security and IT audit practices has an expected annualized loss rate that is slightly more than 2 percent of annual revenue. In contrast, such an organization with normative practices is exposed to an expected annualized loss rate that is slightly less than 1 percent of revenue. An organization with best practices however, has a much lower annualized exposure rate that is between 0.1 and 0.2 percent of revenue.

Whether the experience is more recent such as the breaches that are reported to have occurred at Amuse, Mitsubishi, Fiserv, the U.S. Army, Network Solutions, HSBC, Anthem, Virginia Department of Health, or less recent experience such as the reported data breaches at TRW, Heartland Payment Systems, CheckFree, Norwegian Tax Authority, or the U.K. Ministry of Defence, the publicly available data shows some interesting findings, including:

 Organizations with the worst practices experience multiple loss events in shorter periods of time

 Roughly half of the losses are due to inadequately managed procedural controls

 About half of the losses occur due to inadequately managed technical controls

Improving practices for information security and IT audit are shown to reduce the occurrence rates of data loss or theft and the onset of these events from resulting in financial loss. The lesson from the benchmarks and industry experience is clear: improve practices for information security and audit and financial exposure and loss is dramatically lower, and occurs much less frequently.

Figure 8: Reductions in financial risk from data loss or theft

Although some of the publicly documented loss experiences are higher — and lower — than the benchmarked experience of 3,280 organizations, the benchmark findings and publicly reported loss experiences fall within the predictable ranges as depicted. In addition to the benchmarks, a reliable source for industry experience can be found in the data maintained by OSF at www.datalossdb.org.

Annual accruals of expected loss from the risk of customer data loss or theft, while not common, are being recorded internally against operating expenses at some of the organizations with the best practices. Among most other

organizations however, these losses are only recorded on publicly visible quarterly operating statements after latent loss and theft rates from poorer practices result in financial loss from litigation, clean-up expenses, fines and penalties.

(18)

Whether the objective is reducing financial exposure, increasing customer retention, maximizing revenue and profit, avoiding business disruptions, reducing audit fees, or reallocating labor in IT for more critical projects, the findings provide strong financial evidence it is worth the effort to improve practices for information security and IT audit.

Taking Action to Improve Outcomes

Current practices may have to be modified, augmented, or added to improve outcomes and results. Change is always difficult, but is a necessary part of improving results for the 9 of 10 organizations not operating with the best outcomes. While useful, practice guidance from CobiT, COSO, PCI, ITIL, NIST and regulatory mandates can only point an organization in the correct direction: guidelines cannot substitute for action

that results in better outcomes.

The primary actions taken by the best performing organizations to improve practices and results include: (1) prioritizing and managing the risks;

(2) establishing objectives and measuring results; (3) rationalizing IT policies and procedures; (4) delivering training for employees; (5) segmenting and limiting access to customer and sensitive corporate data; (6) improving IT controls and procedures; (7) automating the collection of IT audit and information security data; and (8) frequent assessment and reporting. The guidance for improving practices for information security and IT audit

contained is this report is shown to improve operational outcomes, reduce costs and financial risks. This guidance, based on the results of more than 3,280 organizations, will hopefully serve as guideposts for improvement (Table 8).

Table 8: Guidance for Best Practices in Information Security and Audit Guidance

Establish objectives Define and maintain what constitutes unacceptable business risks _{related to the use of IT.} Focus on core business risks and objectives for controls: fewer than 30 is optimal.

Establish standards

Define and maintain internal standards for confidentiality, integrity and availability of information assets.

Maintain objectives for controls being used to manage the primary business risks related to the use of IT.

Organizational structure _{Manage information security from a non-IT operations role.}

Practices Implement the baseline practices

Implement the top 10 practices

Employ guidance for practices, including: 1. CobiT and COSO

2. ISO and PCI

3. ITIL, vendor guides, NIST, CIS benchmarks, regulatory mandates

Protect critical assets

and information IT security data, IT audit data, customer data, critical IT assets, and _{sensitive organizational data, systems, applications, networks, and user} accounts.

Continuous assessment and reporting

Conduct very frequent controls assessments against internal standards for confidentiality, integrity and availability (CIA standards

- Once per week for sensitive data

- Once bi-weekly for IT general controls

- Once per month for procedural controls

Use common controls between IT audit and information security for

- Backup, archive and data quality tests and reports

- Procedural and technical tests and reports on controls

Improvements worth the effort The findings provide strong financial

evidence it is worth the effort to improve practices for information security and IT audit.

(19)

Appendix A: Guidance for Practices

Recent benchmarks completed with 319 organizations determined the form of guidance being employed by organizations for practices implemented in information security and IT audit. The results are organized as practices and guidance for six key focus areas, and an assessment of the level of customization required, as follows:

 Managing the integrity of information

 Managing information security practices and procedures

 Managing compliance with regulatory audit

 Managing information security policies

 Managing business risks related to the use of IT

 Level of customization required

Managing the integrity of information

Perhaps the most critical area of focus for most organizations is how well information and IT assets are maintained free from corruption or unintended change. The two dominant practices employed by most organizations to maintain the integrity of information include:

 Backup and archive test reports

 Data integrity checks and quality test reports

These test reports are followed by vendor guides and the ISO 9000 and ISO 20000 quality standards at much lower incidence levels. Given the large proportion of organizations employing backup, archive, and data integrity test reports to ensure information integrity, these practices can be considered as “table-stake” practices that must be implemented. When the findings are directly linked to outcomes being experienced by organizations, the picture that emerges reveals a dominant use of CobiT and COSO as practice guidance for ensuring the integrity of information among organizations with the best outcomes. The findings are conclusive: almost thirty times the number of organizations with the best outcomes rely on CobiT and COSO for practice guidance for managing the integrity of information (Figure 9).

Figure 9: Guidance for managing the integrity of information, by outcomes

In contrast, a majority of organizations with normative outcomes are relying on SCAP, CVE, and SDLC guidelines for managing the integrity of information. As a protocol for updating patches to systems and a common method for classifying vulnerabilities, these two guidelines, while useful for classifying and remediating problems, are apparently not contributing materially to maintaining the integrity of information.

(20)

Lastly, organizations with the worst outcomes are relying on combinations of backup, archive, integrity test reports, vendor guides and ISO quality standards as the primary practice guidance for information integrity without relying on assistance from CobiT, COSO, ITIL SCAP or CVE.

The findings clearly show that backup, archive, data integrity and quality reports, along with practice guidance from CobiT and COSO form a foundation for best practices to maintain the integrity of information and better outcomes.

Managing information security practices and procedures

The two dominant guidelines employed for managing information security practices and procedures for most organizations include:

 IT audit and IT security test reports

 Assessments of procedural and technical controls

Between four and five of every ten organizations rely on these two empirical approaches — test reports and assessments of controls — to manage information security practices in the organization. Although other forms of guidance are

employed, such as vendor guides and NIST guidance, these are employed at much lower levels than test reports and assessments of controls.

When practices are directly linked to outcomes being experienced, the evidence reveals organizations with the best outcomes are relying primarily on guidance from CobiT, COSO, and to a somewhat lesser extent on PCI, NIST, FIPS and FISMA. More than 20 times the population of the best performers employ CobiT and COSO for guidance on information security practices and procedures, followed by supplementary use of PCI, NIST, FIPS, and FISMA practice guidance (Figure 10).

Figure 10: Guidance for information security practices and procedures, by outcomes

In contrast, the primary guidance employed by a majority of organizations with normative outcomes is the practice guidance contained in the ISO information security practices, including ISO 17799, 27001 and 27002). However, fewer organizations with normative outcomes use CobiT, COSO, PCI, or ITIL as guidance for information security practices. Lastly, organizations with the worst outcomes are relying primarily on vendor guides and not much else. Although most vendor guidelines are critically important to the effective implementation and maintenance of technical controls and among some the maintenance of non-technical or procedural controls, by themselves vendor guidelines are apparently insufficient to improve outcomes.

The findings indicate the to be effective, vendor guidelines have to be led by audit and IT security test reports, procedural and technical control assessments, and guidance from CobiT, COSO, PCI and NIST among other forms of guidance. When the practices are aligned by outcomes and checked against audits, the results show that PCI is being employed for guidance among best performing organizations even when there are no PCI audits.

(21)

Managing compliance with regulatory audit in IT

The primary practices for managing compliance with regulatory audit among all organizations include:

 IT audit and information security test reports

 Procedural and technical controls assessments

While necessary, these empirical approaches to managing audit in IT are insufficient by themselves. The organizations with the best outcomes augment the use of test reports and control assessments with practice guidance from several sources, including CobiT and COSO, PCI guidelines, the ISO security practice standards (including ISO 17799, ISO 20001, and ISO 20002, and ITIL (Figure 11).

The intriguing insight about these findings is that CobiT, COSO, PCI, and ISO are very focused on information security and audit best practices, despite being associated with either specific audits (credit card data for example in the case of PCI, or government related audits, guidance, or specifications in the form of NIST, FIPS and FISMA).

Figure 11: Guidance for managing compliance with regulatory audit, by outcomes

When the findings for PCI, FIPS and FISMA are evaluated against regulatory audits among organizations, the best performing organizations are shown to employ these secondary forms of guidance, even when not facing PCI, FIPS and FIMSA audits.

Among most normative performing organizations, the primary guidance being used for managing compliance with regulatory audit includes NIST, FIPS and FISMA, with slightly fewer of these organizations relying on PCI. The defining difference among normative performing organizations is a lack of use of CobiT, COSO, PCI, ISO, and ITIL guidance to inform and guide practices for regulatory audit.

In contrast, organizations experiencing the worst outcomes are not using practice guidelines to inform and vet practices for managing compliance with regulatory audits. The results clearly reveal that until the worst performing organizations integrate best practices identified in CobiT, COSO, PCI, ISO and ITIL, any improvements are likely to be due to luck.

Managing information security policies

Among a majority of organizations, practices for information security policies are guided by two primary considerations that include:

 Internal standard for confidentiality, integrity and availability

 Legal advice and counsel

However, the best performers rely primarily on unacceptable business risks, vendor guidelines, and then on internal standards for confidentiality, integrity and availability for defining and managing information security practices.

After these three, best performing organizations consult practice guidelines that are pertinent to regulatory mandates that include PCI, GLBA, HIPAA, FISMA, SOX, and sources such as ISO, CobiT, NIST, and FIPS. It is not until these guidance sources are utilized that legal advice and counsel weighs in importance for practices related to managing information security policies among best performing organizations (Figure 12).

(22)

In contrast, a majority of organizations operating at the norm rely primarily on legal advice and counsel for guidance to manage information security policies, and pay much less attention to other factors when defining and managing

information security policies. The reliance on legal advice among these organizations is in stark contrast to organizations with the best outcomes, where legal advice, while employed, is the last form of guidance being employed. For a majority of organizations experiencing normative outcomes, the challenge will be to rely more on internal business value and risk standards, while considering legal counsel within a broader context of business risks.

Figure 12: Guidance for managing information security policies, by outcomes

Although industry standard guidelines are successfully being employed for managing information security practices, for managing regulatory audit, and for managing the integrity of information, these guidance sources play a smaller role in managing information security policies.

Rather than relying on strict check-off lists of canned policies based on external sources, organizations with the worst outcomes will have to do the homework necessary to define and manage unacceptable business risks to make forward progress.

Managing business risks related to the use of IT

The primary practices implemented by most organizations to manage business risks from the use of IT include:

 IT audit and IT security test reports

 Risk assessments

After an empirical approach using test reports and risk assessments, the best performing organizations rely on practice guidance defined and outlined by CobiT and COSO, along with procedures and risk assessment tools such as Octave and RiskNav among others to manage business risks related to the use of IT.

In contrast, the majority performing at the norm rely primarily on risk assessments and reports, IT audit and IT security test reports, and very little in the form of external guidance for managing risks associated with the use of IT. Unlike the best performing organizations, organizations experiencing normative outcomes are not using CobiT and COSO for practice guidance, and are not using Octave, RiskNav or similar tools to manage the risks. Flying solo, most organizations are not using tested guidance to better manage the business risks associated with the use of IT.

Similarly, organizations with the worst outcomes rely on test reports, but do not use these in the same extent and in the same magnitude as normative and best performing organizations. More characterized by ad-hoc procedures, these organizations are not using external forms of guidance or tools for managing business risks related to the use of IT. As the benchmark findings clearly show, the business risks of worse practices in information security and IT audit include higher than desirable loss rates for customer data, higher than desirable financial loss from such losses, much higher levels of business disruptions due to failures or disruptions occurring in IT, overspending on audit, and lost opportunity on time spent in IT to improve outcomes.

(23)

Improving results for most organizations will occur by placing more focus on IT audit and IT security test reports, risk assessments, and using the practice guidance from CobiT and COSO as markers to improved practices and ultimately, outcomes for the organization (Figure 13).

Figure 13: Guidance for managing business risks related to the use of IT, by outcomes

Level of Customization Required for Audit Requirements

The audit requirements imposing the most customization include HIPAA and ISO. The audit requirements imposing the least amount of customization include CobiT, the CIS Benchmarks and FISMA. For some organizations there may be no choice but to implement one, two, three or more of these audit requirements. Finding the common denominator between these different audit requirements will reduce customization, organizational impact, and costs (Figure 14).

Figure 14: Level of Customization Required

(24)

Appendix B: Practices Tested

Listed below is the full set of topical practices benchmarked against the outcomes of 3,280 organizations.

Existence and importance of practices Existence and importance of practices

Frameworks aligning IT and business risk Distribution of IT policies

Alignment: T quality management and internal controls Role of the chief information security officer

Risk management and information security function Risk management and IT operations function Risk management and the CIO

Risk management and legal counsel

Risk management and business unit managers Risk management , physical and plant security Risk management , internal and IT audit Classification and valuation of information assets Value of customer data

Value of financial data Value of employee data Value of sales data Value of design data Value of manufacturing data Value of sourcing and logistics data Value of intellectual property Value of audit and reporting data Value of IT security data Data classification schema

Data classification, access control and business continuity Background checks

Awareness and training Critical IT asset inventories Lifecycle control maintenance Storage, archive and retention of data Inventory controls for stored/archived data

Protection of sensitive data for retired/disposed IT assets Controls for data leakage

Training in IT

IT change management Service levels agreements Supplier and contractor services IT continuity plan and tests Logging of IT changes

Failure root cause, tracking and analysis Communication of IT security policies User IDs

User IDs and access rights Permissions and entitlements Segregation controls

Abnormal activity detection and prevention Security incidents

Disclosure of information security data Reviews of systems and internal standards Protection of system audit tools

Monitoring and evaluation of IT performance and risks Monitoring of privileged activities

Frequency of control assessments

Automation of procedural and technical controls Controls common to IT audit and IT security

Change management practices User account controls

Technical controls

Business continuity procedures IT audit and IT security test reports Policies, standards and frameworks Personnel security clearances

IT change management procedures and controls Data indexing, retention and disposal procedures Checksums, integrity checks, and timestamps Change controls for policies, standards, procedures IT quality plan and continuous improvement Measurements and IT quality plan

Control self assessments Automated monitoring of controls

Mapping of controls to standards and frameworks Correcting gaps

Distribution of policies Roles and responsibilities

User acknowledgement and exceptions

Mapping policies, control statements, regulatory requirements Gathering and retaining conformance evidence

Financial and business impact reports Policy compliance reports

Operational quality reports

Legal and regulatory compliance reports Threat reports

IT audit and IT security test reports Electronic dashboard summary reports Outcome scorecards

Risk management practices Risks posed by downtime Risks posed by loss of integrity Risks posed by penalties and fines

Risks posed by outsourced information and assets Risks posed by theft/fraud

Risks posed by theft or loss of customer data Risks posed by information security threats Risks posed by critical skills shortages Risks posed by project delays

Procedures for risk avoidance, reduction, sharing Documentation of controls

Records protection against loss, destruction, falsification Limiting access to sensitive information

Frequency of monitoring Objectives and measurements Employee training

Business risks

IT controls and procedures

Collection of IT audit and controls information