Step 3: Perform Security Audit

Once evidence is collected, the next task is to perform the audit work following ISA 200 and ISA 402 (ISA, 2016) standards. Specifically, the audit involves analyzing evidence in an attempt to evaluate and establish the conformance or non-conformance of CSP controls, i.e. determine whether security procedures and practices in the CSP’s environment sufficiently safeguard assets and comply with security requirements of the organization. To achieve this, the checklist created in the previous step serves as the reference point for reviewing CSP practices and the extent to which the CSP conforms to the organization’s requirements.

6.3.6.3.1 Apply Audit Criteria

Audit criteria is a set of procedures, policies, specifications, or requirements used as a reference against which evidence are compared. The audit criteria can be qualitative or quantitative, general or specific, focusing on what should be according to laws regulations or objectives, sound principles, scientific knowledge or best practices (ISA, 2016). In performing an audit, the Security Auditor uses justifiable audit criteria or select from standard procedures and policies. Reliable sources of audit criteria could be regulations, legislation and standards issued by recognized authorities. For ensuring that evidence is presented in conformance with generally accepted principles, the Statement on Auditing Standards 70 (American Institute of Certified Public Accountants. Auditing Standards Board, 1997) provides essential attributes that are paramount for evaluating the quality of audit evidence and to support the reasonable basis for auditors opinion. Thus, these attributes (as shown in Table 6.19) are adopted and reformed to the context of cloud audit to serve as the audit criteria for assessing evidence. The attributes are considered on the supposition that credence must be established in respect of whether CSP security practices and procedures are done in accordance to specified requirements.

Table 6.19: Attributes for Quality Audit Evidence (Audit Criteria)

6.3.6.3.2 Step 2: Determine Conformance Level

This step involves assessing CSP evidence and performing appropriate analysis to form an opinion that is presented in the audit report. The primary aim is to establish the conformance level associated

Parameter Description

Sufficiency Quality evidence is sufficient quantity has been presented to support assertions made on specific security controls.

Completeness The CSP has presented evidence of all security processes and procedures relating to security controls

Understandability Implementation of security controls, processes and procedures are appropriately presented and described, and disclosures are clearly expressed.

Accuracy Data presented and disclosures made relating to security controls, procedures and processes accurately reflect instances of the cloud operating environment.

Reliability The source of evidence is reliable by nature and is dependent on the individual specific security control area under which it is obtained.

141

with the CSP’s practices based on the evidence produced. In other terms, once a CSP has supplied evidence, an auditor uses professional judgement to measure the sufficiency, reliability and completeness of evidence, and its understandability and accuracy. Essentially, to establish a CSP’s conformance level, a thorough assessment of all evidence presented by the CSP is performed. The evaluation mainly determines the CSP’s conformance in respect of requirements. A simple equation and an assessment scorecard are created, where the Security Auditor, based on expert analysis and interpretation, determines the level of conformance that is associated with a CSP, i.e. the ability to satisfy security requirements. In other words, after evidence have been analyzed and assigned a score about the attributes of quality evidence, a computation is performed to determine the CSP’s level of conformance to the security requirements.

i. Conformance Level to each Question: using the scorecard in Table 6.20, the Security

Auditor assesses evidence presented by the CSP and assigns a value of either ‘1’ or ‘0’ according to how each evidence satisfies the attribute of quality evidence (SCUAR). For example, assuming a CSP had responded with a ‘Yes’ answer to a question and also presented supporting evidence, an auditor assesses the quality of the evidence in terms of its sufficiency, reliability, completeness, understandability and accuracy. If the auditor perceives evidence to validate the CSP’s claim, then he scores the evidence with ‘1’, otherwise ‘0’.

𝑪𝒐𝒏𝒇. 𝑳𝒆𝒗𝒆𝒍 =Sufficiency+Ccompleteness+Understandability+Accuracy+Reliability_{5 (Attributes of Quality Evidence)}

∗ 100

Table 6.20: Evidence Scorecard:

Attributes of

Quality Evidence Possible Values Score Value for Evidences

Sufficiency 1 or 0 Score = 1 if ‘Evidence is Sufficient’, else Score = 0 Reliability 1 or 0 Score = 1 if ‘Evidence is Reliable’, else Score = 0 Completeness 1 or 0 Score = 1 if ‘‘Evidence is Complete, else Score = 0 Understandability 1 or 0 Score = 1 if ‘‘Evidence is Understandable, else Score = 0

Accuracy 1 or 0 Score = 1 if ‘Evidence is Accurate, else Score = 0

Table 6.21 General Scorecard for CSP Conformity Level

Conformance Type Weight Conformance Level

Very High conformance 100 5

High conformance 80 4

Medium conformance 60 3

Low conformance 40 2

Very low conformance 20 1

145

Table 6.22 Security Audit/Analysis

Requirement Target Verification

(Control Domain) (Control Type) Base Measure Specification Question CSP/User Response Verification Means of Criteria Audit Conformance Level

Yes No S C U A R

146

Table 6.22 provides evidence analysis and audit report. The target verification implies the set of control

domains being assessed. For an auditor to perform the required assessments, a base measure is used, that is the type of control being audited. For instance, evidence produced in respect of the application and interface security within (target of verification) to manifest what controls exist in the area of data integrity (i.e. base measure). Therefore, application and interface security become the target of verification, whereas data integrity controls under application and interface security become the target of verification. Furthermore, the means of verification implies the type of evidence that was examined, i.e. supporting evidence produced by the CSP such as log report generated by automated security monitoring tools. The audit criteria highlight the audit criteria (attributes of quality evidence) upon which evidence is compared. Each evidence obtained is compared to the criteria for determining a certain weight or score that will be associated with the evidence. Assessment score assigns a score value to the evidence that has been analysed. Total score cumulates the overall score attained by individual evidence for the base measure in respect of all five audit criteria. Global score cumulates the overall score achieved target verification (control domain) in respect of base measures. The Conformity level determines the level of conformity a CSP has achieved. The outcome is a summary of findings to support the auditor in generating an audit report.