At their final gathering before the exit meeting, audit team members evaluate the accumulated data. They sort the data, see what the data support, and look at the data away from the situation to see what they mean. Any finding should be clear to a reasonable person. The auditor has to look at the evidence found, its relation to the purpose of the audit, and its importance to the entire scope that has been audited.
The lead auditor must make every effort to resolve contradictory evidence or open issues prior to reporting the results of the audit. However, if contradictory evidence has been gathered and the audit team cannot obtain additional data, the contradiction should be included in the audit report as a finding or an observation, and clarification should be requested. A supplemental report can be generated if needed to clarify the original audit report.
Through analysis of audit data, the audit team can distinguish between sys-temic and isolated incidents. An auditor who detects a possible problem looks for a recurrence. If subsequent data show recurrence of the incident in several places, it is a systemic incident. Data that reveal a trend and pattern may also
Part IIb
support a conclusion that an incident is systemic rather than isolated. For example, if one piece of equipment is overdue for calibration by about a month, but all other equipment is in calibration, the auditee may have a good calibration program and the one piece of equipment is an isolated case. However, if a high percentage of equipment has not been calibrated on schedule, the problem is systemic.
Thus, an auditor needs to look at data from a system standpoint and try to assess how the overall system is working. If after preliminary evaluation of data a problem appears to be systemic, the auditor may need to draw another sample.
Once the problem has been identified as a systemic or isolated event, it can be clas-sified for reporting purposes. Typically, when the auditee responds to the audit findings, an isolated occurrence is addressed by correcting the specific instance.
An auditee responding to a systemic issue looks at what aspects of the system exist that permitted the problem to occur. For example, the auditee may need to perform an investigation to determine the root cause of the issue, optimize procedures and retrain personnel, and/or evaluate whether a process should be automated.
The potential impact of an isolated occurrence should be considered when determining which findings to include in the audit report and how they should be rated. If an isolated occurrence of a nonconfor-mance could result in poor quality or unsafe products that may harm the user, it should be included in the audit report and rated accord-ingly. For example, release of one batch of a drug product that did not meet specifications for potency may be an isolated occurrence, but it could cause patients to become more ill or to die. This nonconfor-mance should be included in the audit report and be rated as critical or significant.
Interviews
Interviewing, the process of obtaining information from another person in response to questions, is an important and widely used form of data collection in auditing.
Interviews are used in audits to gain insights, clarify information, confirm or deny suspicions, and elicit details that may not be brought out by other audit activities.
The auditor may employ an interview format such as the six- step interview pro-cess developed by Frank X. Brown:5
1. Put the interviewee at ease 2. Explain your purpose 3. Find out what they are doing 4. Analyze what they are doing 5. Make a tentative conclusion 6. Explain your next step
When conducting an interview, the auditor should begin by providing a personal introduction and putting the interviewee at ease. The purpose of the interview
Part IIb
should be explained, followed by general, open- ended questions that get an inter-viewee talking about the tasks he or she performs. An auditor may simply say,
“What are you doing?” or “What tells you to do that?” If the interviewee’s answers confirm the documentation, the information has been corroborated. If not, the auditor needs to probe by asking questions that are even more specific to ascer-tain whether a work problem does indeed exist, or whether the interview is being hampered by miscommunication. An auditor may not understand the process the interviewee describes, or the interviewee may misinterpret or fail to understand a question. An escort can often help clarify communication problems resulting from unfamiliar terminology by rephrasing a question. An escort should not be permit-ted to answer for an interviewee, however.
When auditors make a conclusion about whether necessary controls are in place, they must communicate the preliminary conclusions to the interviewee and/or escort. For example, an auditor could say, “That’s good. It looks to me as though you have a thorough understanding of your job and are addressing all requirements.” An auditor who suspects a potential finding could say, “It looks like we might have a problem in this area. I can’t see where those controls are placed.” Such comments give the interviewee an opportunity to offer more infor-mation. If the interviewee is able to allay the auditor’s concerns, no problem exists.
In general, people respond readily to inquiries and offer helpful suggestions for improvements if they feel that the audit team is sincere, appreciates their views, and has their needs and interests at heart.
The auditor usually takes notes during an interview and should take a few minutes at the end of each interview to ensure that the notes are complete and will be meaningful when reviewed later in the day. If the interviewee appears apprehensive, the auditor should explain that notes are taken to record informa-tion about the details of the interview. If extensive note taking is necessary, it may be best for the auditor to step out of the room or into a secluded location to record observations so that the interviewee will not become unnerved by the auditor’s recording large amounts of information.
In all communications with the auditee, the auditor must focus on the situa-tion, issues, documentasitua-tion, activities, and/or behavior—not on the person. When communicating with auditee management, the auditor should avoid naming indi-viduals whenever possible.
Interviews are an integral part of process and system audits. There may be no interviews during a product or service audit except to verify status. For product audits, the auditor simply verifies product or service characteristics. A product auditor may audit products to verify that inspection is effective or the reason for a returned or aged product. External audit interviews are more formal than internal audit interviews. Even if an external auditor has audited the same organization for a number of years, the auditor must not become too friendly or fraternize with the auditee personnel.
Also, it is useful to have interviewees agree with records or notations of their comments, and auditee personnel verify the accuracy of the auditor’s observations of activities and status of items, when they are being recorded. This technique will help ensure that the auditee understands the results of the investigation.
Other interviewing techniques include (1) effective interviewing when no pro-cedure exists, (2) what to do when supervisors are present, (3) handling hostile or
Part IIb
difficult situations, (4) interviewing groups, (5) interpreting pauses, and (6) using a translator. Since interviewing is considered an important auditing skill, additional interviewing techniques can be found in Chapter 14, “Interviewing Techniques.”
4. esTabLIshMeNT oF objeCTIVe eVIdeNCe Characteristics
Evidence should be collected on any matter related to the audit objective and scope.
Evidence can include procedures or work instructions and proof of their implemen-tation, examination of personnel training and qualification records, examination of process controls and records, and reexamination of selected work. Evidence needed to support the auditor’s findings may be physical, testimonial, documentary, or ana-lytical. Objective evidence may be obtained internally from the organization under audit, externally from third parties, or by comparing evidence obtained from two or more sources. Evidence obtained from independent external sources is generally more reliable than information obtained from internal sources. Knowledge obtained through physical examination, observation, computation, and inspection is more reliable than interview comments. Evidence derived from records is likewise more reliable than oral evidence. Evidence must be sufficient, complete, and relevant to be useful as a sound basis for audit findings and recommendations. A rational relation-ship should exist between the cost of obtaining evidence and the usefulness of the evidence.6 For example, the costs involved in shutting down a manufacturing plant to obtain data or in traveling to 100 service centers to verify that employees know the quality policy may not be justifiable.
ISO 19011 uses the term audit evidence instead of objective evidence. Any evi-dence collected during the audit should be objective and free from bias. Evievi-dence should be recorded and verifiable. Evidence can be verified or confirmed by (1) records or documents, (2) verbal affirmations, (3) repeat observation, (4) mea-suring or testing, and (5) simulation. Audit evidence is based on the information gathered through sampling. Even though the sampling results are factual, there is an element of uncertainty regarding conclusions. The conclusions are only as good as the degree to which the samples represent the performance or actions of the area being audited.