Notes 4 Session 4. Application Controls. Participant Notes

(1)

Notes 4 Session 4

Participants Notes: IS Security Management 1

Application Controls Participant Notes The System Development Life-cycle (SDLC)

Projects are conceived and grow from a business need. But what seems to be a clear objective at the beginning can easily become blurred and confused as events progress. In the end the product which is delivered might not be what was expected and this may result in significant wasted investment.

Many risks can, to a greater or lesser extent, affect the successful development or acquisition of a new information system. These include the risk that the system will:

• never be delivered; • be delivered late; • exceed budget;

• divert user resources to an unacceptable degree; • not deliver the required functionality;

• contain errors; • be difficult to use;

• fail frequently during operation; • not perform to the required standard;

• be difficult and costly to operate and maintain; • ditto to enhance;

• not interconnect with other systems.

Where an organisation suffers one or more of these undesirable impacts, their cause can more often be attributed to poor management rather than to problems involving technology. Common management problems include:

• no standard approach to project management; in particular :- ∗ unclear project definition, objectives and scope; ∗ an inadequate Business Case to justify the project;

(2)

Notes 4 Session 4

∗ lack of communication :-

♦ of clear responsibilities for line and project management. ♦ between business, user and project management;

♦ of project requirements;

♦ of accurate and reliable information on costs and progress;

• failure to recognise over-ambitious objectives in relation to the organisation’s capabilities to deliver them;

• lack of senior management commitment to the project; • no full time and experienced project manager;

• failure to manage suppliers and consultants (who sometimes take control of the project);

• taking shortcuts due to lack of time (particularly cutting back on training, testing and quality reviews);

• failure to freeze the requirement specification resulting in slipping deadlines and rising costs;

• lack of quality standards;

• reliance on unproved technology.

The number of project failures can be dramatically reduced by breaking down a project into a number of more manageable stages each of which is to produce one or more pre-defined products. The advantage of a structured approach is that it helps to reduce the complexity of planning, monitoring and control. It also offers a number of points during the project where progress against pre-defined deliverables can be reviewed and corrective action taken as necessary (including abandoning the project!)

Collectively these stages, known as the Systems Development Life- cycle (SDLC), are:

• project initiation; • feasibility study;

• requirements specification;

• system design (or external procurement); • system development (or external procurement);

(3)

Notes 4 Session 4

• user acceptance testing; • system implementation; • post-implementation review.

Each of these stages is described in this and the following chapters. Responsibilities in the System Development Life-cycle

Historically, system development projects have often been undertaken by, or under the dominance of, an organisation’s IT department. Where there is little or no end user involvement, the system that is eventually delivered often fails to meet its end users’ expectations in full, and the organisation fails to reap the full return on their investment.

It is now widely accepted that the system’s eventual owner and its end users must be identified before the project commences so that their needs are taken fully into account. Good quality end users should be assigned to the project team at an early stage and, where possible, the project should be under the overall control of a director or senior manager from the end user area.

Internal Audit and Security Management should also be closely involved in the project. This is particularly important during the system specification and design stages, as their input helps to ensure that IT security is adequately addressed. This need applies equally to bespoke developments and to the procurement of packaged software. The external auditor should be consulted on financial audit trail and data retention/extraction requirements, while legal and other expert advice may be necessary during contractual negotiations with suppliers.

To summarise, the various roles and responsibilities during a system development project include:

Top Management: approves the necessary resources to complete the project, and provides commitment to it. This commitment from Top Management helps ensure involvement by those needed to complete the project.

(4)

Notes 4 Session 4

User Management: nominates the System Owner, although during development the system will be owned by the Project Board (on which the System Owner may well be chairman). The users should actively participate in defining requirements, the feasibility study, user acceptance testing and user training. The System Owner should review and approve system deliverables as they are produced.

IS Management: provides technical support for hardware and software environments by developing, installing and operating the required system. Provides assurance that the system is compatible with the organisation’s IS environment and strategic direction. Assumes on-going operating support and maintenance activities after live implementation.

Project Board: group of people representing Business, User and Developer interests and who own the project. They are responsible for ascertaining on-going project viability and ultimate project success.

Project Manager: responsible to the Project Board (but not a member of the Board) for the day-to-day management of the project including monitoring and controlling project costs and timetable.

Figure: life-cycle responsibilities L e a d r e s p o n s i b i l i t y U s e r U s e r U s e r U s e r / I T d e p a r t m e n t I T d e p a r t m e n t I T d e p a r t m e n t I T d e p a r t m e n t U s e r / I T d e p a r t m e n t F e a s i b i l i t y s t u d y R e q u i r e m e n t s A n a l y s i s R e q u i r e m e n t s s p e c i f i c a t i o n L o g i c a l s y s t e m s p e c i f i c a t i o n P h y s i c a l d e s i g n S y s t e m c o n s t r u c t i o n I n s t a l l a t i o n A c c e p t a n c e t e s t i n g P o s t i m p l e m e n t a t i o n r e v i e w

(5)

Notes 4 Session 4

IS Project Team: completes assigned tasks, communicates effectively with users by actively involving them in the development process, works according to local standards, advises the Project Manager of any necessary project plan deviations.

User Project Team: completes assigned tasks, communicates effectively with IS by actively involving them in the development process, works according to local standards, advises Project Manager of expected and actual project plan deviations.

Security Officer: ensures that system design adheres to corporate security policies, consults on appropriate security measures that should be incorporated into the system, tests system security prior to implementation, receives adequate training on any new security systems or procedures to assume security responsibilities after implementation.

Quality Assurance: reviews results and deliverables within each phase, and at the end of each phase, to confirm compliance with requirements and quality standards. The point where review occurs depends on the SDLC methodology used, structure and size of the system, and impact of potential deviations.

Internal Audit: reviews and reports to management throughout the project on the adequacy and application of the SDLC methodology. Assists in defining internal controls.

External Audit: for financial systems, advises on the measures that need to be included in the User Requirements Specification to ensure that the financial statements to be produced by the system will be capable of audit using accepted techniques.

The Feasibility Study

Where a major project is involved, a Feasibility Study might itself represent a significant project and require separate financial approval and project initiation. At the other end of the scale, work on the development of a low cost system which has

(6)

Notes 4 Session 4

no strategic impact and which is clearly feasible might commence without a formal feasibility study first taking place. It might also be the case that a project’s feasibility is established during the IS Strategy Study.

The main objective of a Feasibility Study is to determine whether a proposal is viable and to recommend a suitable course of action if it is. The study might be undertaken by the organisation’s own staff (from both the IT Department and the end use area(s)), by external consultants, or a mixture of both. It will:

• define the problem or need that requires solution;

• define broad or major requirements of the required solution; • determine if a computerised solution is required or desired;

• determine if an existing system can be enhanced to correct the situation; • determine if a commercial product offers a solution to the problem;

• for each alternative, provide an estimate of costs, benefits, technical and business risk, time-scales, and an assessment of the option’s ‘fit’ or compliance with the organisation’s IS Strategy.

During the Study, the Team should consult the Security Manager, the Internal Audit Department and the Legal Department on any requirements that may be necessary to satisfy the organisation’s IT Security Policy or legislation, and which may serve to limit the number of options that are available.

The Feasibility Report might:

• endorse the proposal, identify a suitable solution to the problem, and seek authority to proceed with its development;

• recommend one of the following options: ∗ develop or acquire a demonstration system; ∗ ditto a pilot system;

∗ phased implementation;

∗ acquire a commercially available package; ∗ commission a turnkey system

(7)

Notes 4 Session 4

∗ facilities management; ∗ outsource the service;

• recommend a hybrid solution requiring customisation of a commercial package (this is a high risk option and should be avoided if possible);

• conclude that the project would be feasible if its scope were to be reduced or were it to be split into a number of smaller projects;

• recommend that the proposal is impractical, too expensive, or unattractive for other reasons, and that it should not be taken forward.

Note: phased implementation is applicable to projects that have long

development and implementation time-scales, where changes in business and user requirements will naturally occur before they are fully implemented. Such projects should not be planned in a monolithic way, but be sub-divided into a number of self-contained phases.

The Study will result in a decision by the IS Steering Committee on the course of action to be taken (including abandonment).

The contents and scope of a Feasibility Study will vary considerably between organisations due to their different circumstances and the constraints which apply to them.

Project initiation

The aim of the Initiation Stage is to undertake the groundwork for the future management of a project and to obtain authority for it to proceed. Depending on the scale of the project, Initiation might relate to a study (e.g. a Feasibility Study), to the entire project, or to a single stage of a project (e.g. the Technical Design stage). An initiation stage might therefore occur at a number of points in the SDLC.

(8)

Notes 4 Session 4

Some form of financial authority is normally needed before the Initiation Stage is reached. The level of authority necessary to approve the expenditure will generally depend on the value of the project and on the organisation’s rules on delegated financial approval. For major projects, this authority is likely to be provided by the ISSC.

During the Initiation Stage the following topics will need to be addressed: • the scope of the project;

• appointment of the project’s organisation and management (Project Board, Project Manager, Stage Manager and the Quality Assurance Team);

• appointment of staff and any consultancy support needed to undertake the project;

• any training needs which must be satisfied before the project commences; • any options raised in earlier reports (e.g. the Feasibility Study Report);

• continuing validity of the existing user requirements, and of any assumptions and recommendations;

•

the identification and management of both business and security risks.

Specifying user requirements

Introduction

This chapter provides an overview of the process for preparing the User Requirements Specification, and for obtaining proposals from suppliers where a system is to be provided externally. Audit considerations are covered in the final section.

The User Requirement Specification

If, following their examination of the Feasibility Report, the approving authority (e.g. the IS Steering Committee) decide that the project is to proceed, the next stage in the SDLC is to build on the user’s outline requirements and produce an

(9)

Notes 4 Session 4

accurate and comprehensive statement of the user’s needs, the User Requirements Specification (URS).

The URS is written in non-technical terms and consolidates all the material produced to date relating to the business functions of the required system. It represents the:

• formal declaration of the users’ requirements of the proposed system; • basis on which design work on the proposed system can proceed; • basis for vendors to submit proposals and tenders;

• basis for acceptance testing the proposed system; • basis of User Manuals when the system goes live.

The aim of the Specification Stage is to produce a comprehensive description of the current system, highlighting its deficiencies, describing how these are to be rectified by the enhanced or replacement system, and updated information on costs and investment appraisal (the Business Case must be kept under constant review as the project proceeds to ensure that costs are kept under control and the project remains financially viable). The completed URS should include the following information:

• a description of current problems, and how these will be resolved by the proposed system;

• a narrative describing what is required of the proposed system; • main body of the document, containing:

∗ outline description of business functions; ∗ definition of data input;

∗ definition of data outputs; ∗ description of transactions; ∗ description of processes;

∗ description of current processing regulations; ∗ description of major clerical functions;

(10)

Notes 4 Session 4

• a description of management requirements covering security, change control, etc.

If a decision has been made to buy a system (or indeed a service such as facilities management), the URS should be sufficiently comprehensive to form the basis for advising potential suppliers in full of the organisation’s needs, and enable them to respond with detailed proposals of how they propose to satisfy those needs. As a rule, the URS should therefore be written in such a way that it does not constrain the options open to either designers or prospective suppliers to provide innovative solutions by specifying exactly what technical solutions are to be employed in meeting the users’ requirements.

It is important to ensure that the system’s eventual Owner ‘signs off’ the User Requirements Specification to signify understanding and agreement before the project proceeds further.

System design and development

Introduction

If the project involves building rather than buying a new system, detailed designs and specifications will need to be produced before work can commence on the development of an operational system. During these two stages of the SDLC plans will also need to be drawn up to cover other important activities, such as installing, loading and configuring the new system (often referred to as “system building”), acceptance testing, training and implementation. In a large-scale development project, these latter activities can be considerable tasks in their own right, and will need to be carefully planned and managed if they are to be successfully completed.

This chapter provides an overview of the main activities involved during the design and development stages of the SDLC, some of which are likely to be

(11)

Notes 4 Session 4

technically complex. The final section outlines the questions that the auditor will probably need to consider during these stages of the project.

The Design Stage

The design of any system will inevitably involve some compromise between user requirements and various constraints over what can be achieved in practice. By taking a structured approach to the problem, particularly in separating logical and physical design activities, it is more likely that the optimum solution will be achieved.

During design, the Project Manager’s primary concern will be to ensure that system designs match requirements as closely as is practicable and provide a sound basis for subsequent development work. System design activities are technically complex and although the users need to be involved at specific points, particularly in the design of user interfaces (screens and reports), most of the detailed design work will be undertaken by system analysts and other technical specialists.

Although designs will be based on the User Requirements Specification, this is a high-level specification and is unsuitable for system development purposes. System analysts will therefore need to translate it into detailed system designs and technical specifications. Prototyping techniques might be employed to demonstrate design options to the users, and to refine solutions.

Overall, the Project Team will need to address three broad objectives during the Design Stage. These are to produce:-

a logical design: this is a conceptual or abstract design that provides a solution to the user requirement;

Specifications: define how each user requirement is to be met in terms of the hardware and software to be employed;

(12)

Notes 4 Session 4

a physical design: maps the logical design onto the implementation environment (i.e. the hardware, operating systems, database management systems, programming languages).

The detailed technical designs and specifications produced during the Design Stage will then be passed to the programming teams for coding and development testing.

System development

During system development, the Technical Specification will be used to produce all the configurations items necessary to deliver an operational system. These include :-

• software;

• user and operational manuals; • business continuity plans;

• and plans to cover activities such as system building, testing, training and implementation.

Testing: overview

Testing commences during the design phase, during which designs and specifications should be subject to quality reviews (non-computer testing), and continues during the system development and acceptance testing phases of the SDLC.

Computer systems are tested to prove that they perform to the satisfaction of the various interested parties. This includes the developers, operations staff, and the end-users (including the System Owner). It may also include system administrators, security personnel, and auditors. In practice testing can only give reasonable assurance that all is well and that a system behaves as intended and as predicted; but it cannot give positive proof that any module/program/system is free from error. This is due in part to the extremely high number of possible program paths, and in part to the

(13)

Notes 4 Session 4

practical impossibility of generating test data that will adequately test all paths with all combinations of data.

The overall objective of the testing process is therefore, to ensure that the delivered system is of adequate quality. To meet this objective it will be necessary to confirm that the new system:-

• conforms with the organisation’s technical policies and standards; • performs all the required functions;

• can be used by the staff for whom it is intended; • meets it performance objectives;

• is reliable in operation.

The requirement to demonstrate that a system is reliable implies that it should be tested, not to demonstrate that it works, but to uncover as many defects as possible. Tests must therefore be designed that attempt to demonstrate that the system :-

• does not do what it is supposed to do; • does what it is not supposed to do;

• is not operable by the staff for whom it is intended.

Other important principles that should govern testing - and indeed any quality control - activities are that there is :-

• no testing without measurable objectives; • no testing without recording;

• no recording without analysis; • no analysis without action.

Defects uncovered during testing might be corrected if they are considered to be of sufficient importance to justify the cost and time involved in taking remedial action. But it may be preferable to live with a defect if it is trivial, or defer remedial action until a more convenient time, for example by including a fix in a later release of the software. If a defect is corrected, the system (or perhaps parts of it) will

(14)

Notes 4 Session 4

probably need to be re-tested to ensure that the change has not introduced other unforeseen problems. This process is known as “regression testing”.

Development testing

Software testing is a further aspect of quality control. It falls into two broad categories; “development testing” and “operational testing”.

During the development stage, the Technical Specification is used to produce or acquire the actual configuration items necessary to deliver an operational system. As these become available, they will need to be tested by the development team to confirm that they are of acceptable quality. This process is called “development testing”, and generally comprises two broad areas of testing:-

Unit testing: is the testing of an individual program module in an isolated environment before combining it with other modules to form a program. The objective is to determine whether the module is capable of accepting specific input and producing the correct outputs. Unit testing is normally carried out by the programming team leader. Program testing follows a similar objectives, but with all the modules in place to form a complete program.

Integration testing : is the process of adding new programs to an evolving system and testing to find :-

• errors in the interfaces between programs (sometimes referred to as “link testing”);

• discrepancies between the program functions performed and those specified;

• that no unspecified functions are performed.

Because of its highly technical nature, the users are not generally involved in development testing. However, the users should be involved in “operational testing” which is described in the next chapter.

(15)

Notes 4 Session 4

The Acceptance Testing Plan

Testing needs to be carefully planned and scheduled if it is to be fully effective. The Acceptance Testing Plan is therefore an important product from the development stage.

Acceptance testing should not take place until development is complete. It provides a complete end-to-end test of both the computer system and its interfacing clerical procedures. From the auditor’s perspective, acceptance testing provides a final check on whether the system is auditable (in terms of the adequacy of its audit trails, documentation and data extraction facilities) and is well controlled.

Initial thought needs to be given to what acceptance criteria are to apply to each requirement as the User Requirements Specification is being assembled. These criteria are gradually refined into a plan during system design and development, when it becomes clear exactly what sort of tests will be needed to prove that the delivered system is of acceptable quality.

Acceptance testing is therefore based on an analysis of the User Requirement Specification and any other acceptance criteria defined during design and development. This should aim to identify which requirements, facilities and functions are to be tested (and any which are not), their relative importance, and the method of testing to be adopted for each. This information is incorporated in the Acceptance Testing Plan which should document :-

• individual roles and responsibilities during testing;

• configuration management procedures (there is ample scope for confusion between different versions of software under test, and test data);

• expected versus actual test results;

• requirements for completing test failure reports;

• requirements for prioritising test failures (see Annex 3);

• change control procedures to be applied to the resolution of defects; • procedures for re-testing following changes;

(16)

Notes 4 Session 4

• requirements for signing off test results. Individual tests should clearly show :-

• test objectives and evaluation criteria; • staffing requirements to perform the test;

• the required test environment, including hardware requirements (e.g. the number of terminals or printers needed) and any particular system data conditions or standing data requirements.

The procedures and data for each test (referred to as “test scripts”) also need to be documented. Test scripts comprise a set of systematic instructions to the person performing a test together with the test input and output data. Each test script describes what is to be done, who is to do it, when it is to be done, what to look for, and what to record. The input data specified in each script is that prepared specifically for testing the system, while the output data is a written forecast of the exact data that should result from the test.

It is also important to define the procedures for recording, reporting and fixing faults, and for managing changes to the User Requirements Specification and to any other important system documents (e.g. data flow diagrams, flow charts, etc.)

When the Acceptance Test Plan has been drawn up, the prospective System Owner should sign it off to signify understanding and agreement.

Other development activities

Other activities typically undertaken during this stage of the SDLC include the development of :-

System documentation: staff will need clear and concise instructions on how they are to use, operate and maintain the system, from both an end user and a technical support perspective. Manuals should comply with a formal documentation standard (a technical policy on documentation should have been defined during the IS Strategy Study) to ensure that they cover all necessary

(17)

Notes 4 Session 4

topics. The British Computer Society recommends that the following documents are produced:-

• user’s manuals containing instructions on how to run the system; • details of internal control procedures to be complied with in

financial and accounting systems;

• additional system instructions on the technical aspects of computer and network operations;

• a system flowchart and/or narrative describing the purpose, functions, job scheduling of the system;

• a list of all jobs to be processed;

• a list and description of all databases, files and libraries;

• a list of all input requirements, inclusive of data classifications (for security);

• a list of the reports to be produced by the system, when they are needed, by whom and for whom;

• details of backup requirements;

• details on the administration of any access controls that must be maintained by the user and the data processing functions;

• details of all responsibilities and accountabilities for the operation of the system and its effectiveness;

• details of user clerical procedures including control procedures; • details of interfaces with other systems including control and

security procedures.

Training plans: documentation will need to be produced to cover the delivery of training to a variety of potential audiences including:-

• end users in various functional areas of the organisation;

• middle and senior managers, who may need to use different features of the system (such as management reports and database interrogation facilities);

(18)

Notes 4 Session 4

• technical support staff who will need to operate and maintain the system.

Service level agreement: is an agreement between the user and the service provider who will be responsible for maintaining the software and providing the IT service. Work should be started on the operational characteristics that will be expected from the new system, such as:

• the hours during which it will be available to users; • ditto the days;

• the required availability at any terminal and location;

• maximum acceptable downtime on each type of service failure; • procedures to be followed in the event of system failure; • Help Desk support;

• contingency requirements;

• how the agreement is to be monitored and reviewed.

Implementation plans: should address the creation of the environment in which the system is to run, ensure that it is adequately tested and is functionally acceptable, and define the way in which the new system is to be introduced into production use. The plans must cover:-

• installation of hardware;

• any environmental or building requirements; • the transfer of data from existing to new systems; • setting up the Help Desk;

• backup and standby arrangements; • cut-over from old to new systems. Back-up and standby during system development

System development and configuration management libraries should be backed up regularly, and copies of the back-up stored in a secure remote location.

(19)

Notes 4 Session 4

Where failure to meet the system implementation deadline could have a severe impact on the organisation, it will also be necessary to make plans to ensure that development can continue with the minimum of delay should a major computer failure or disaster occur

Systems development and maintenance

It is essential to ensure that security concerns are taken into account during system development and that the maintenance of systems reckons security issues. The following five controls would take care of security requirements in systems development and maintenance:

Security requirements of systems:

• Business requirements for new systems or enhancements to existing systems should specify the requirements for controls.

Security in application systems:

• The objective is to prevent loss, modification or misuse of user data in application systems.

Cryptographic controls:

• The objective is to protect the confidentiality, authenticity or integrity of information.

Security of system files:

• To ensure that It projects and support activities are conducted in a secure manner.

Security in development and support process:

(20)

Notes 4 Session 4

Security requirements of systems

When a new system is being planned and the user requirements specification is being considered, the users tend to focus on the appearance and ease of use of the new system. It is not uncommon for the less obvious requirements, such as access controls, data processing controls, back up and recovery, check-pointing, to be overlooked. When the new system becomes operational the absence of such facilities becomes apparent. At this stage it is very expensive to re-design the system to incorporate the deficiencies; and it may not be technically feasible. For example, if a new system does not provide an adequate audit trail, it may be impossible for the external auditor to provide an unqualified opinion on the organisation’s accounts. This deficiency will probably be extremely expensive to remedy. The important aspects to be kept in mind are:

• IT security requirements must be considered as a design issue. Statements of requirements for new systems, or enhancements to existing systems should specify the requirements of controls.

• Such specifications should consider automated controls to be incorporated in the system, as well as the need for supporting manual controls.

(21)

Notes 4 Session 4

Introduction to application controls What are application controls?

Computerised financial controls extend beyond the general IT environment. We also need to look at the controls within individual applications.

In the past, when an auditor was confronted with a manual system, the auditor would look for the presence of manual controls which ensure that individual transactions are initiated, authorised, accurately processed and manually recorded in the accounting ledgers. The same is true when the client uses computerised applications, i.e. the auditor should look for controls within an application which ensure that transactions are properly initiated, authorised, processed and recorded.

When a client installs a computerised accounting system the underlying control objectives and concepts remain the same. All the computer does is modifying the nature of risks. Computerised financial systems can also incorporate automated controls which don’t require human intervention. For example, in a manual system a finance clerk would have to manually check the account code on a journal existed. In a computerised application the system would automatically check that the account code keyed in by the user referred to a valid account code. Where it didn’t, an error message would be produced and the transaction could not be processed until a valid account code was entered.

The trend is towards more automated financial systems where transactions are entered into the system at an early stage. In the past the transactions were initiated on paper, authorised on paper and they would only get recorded on the computer system at a late stage, e.g. just before a payment was made. In today’s systems, transactions can be automatically generated or they may be entered onto in the system directly by a user, without the user first having to complete a manual form, e.g. a paper purchase order.

(22)

Notes 4 Session 4

Application controls do not only refer to those controls which are built into an application which ensure that valid transactions, and only valid transactions are processed and recorded completely and accurately in the accounting records. Application controls can also refer to the manual controls which operate in proximity to an application, for example a data entry clerk requires a data input form to be signed before it is entered onto the system. Application controls can be categorised as follows:

• file integrity controls; • application security controls; • data input controls;

• processing controls; • output controls; and

• Master file and standing data controls.

In some situations the auditor may need to consider network and communications controls. The auditor may need to identify any controls which exist to ensure that data transferral is complete and accurate.

Application users, administrators and owners

There are normally three types of user associated with financial applications. Application owner

Firstly, there is the application owner. Application owners are normally a senior user. They are frequently drawn from the highest level of management personnel who use the system on a regular basis. For example the owner of a financial accounting application is normally the head of finance. The owner of a stock and stores system would be the warehouse manager, and the owner of a personnel system would be the personnel manager.

The application owner in effect has overall responsibility for the strategic contribution the system makes to business objectives. For example the finance

(23)

Notes 4 Session 4

manager in charge of the accounting system would be responsible for ensuring that the organisation’s transactions were being recorded completely and accurately and that the system was producing the required output, e.g. management reports on sales, spend vs. budget etc.

The auditor should note that the concept of an application owner is not always apparent to the client. Therefore, when the auditor wants to find out who the owner is, it may be necessary to ask questions such as “who has overall responsibility to the Board (senior management) for ensuring that the accounting system operates as intended?”

The application owner also has responsibility for ensuring that the system is used by the right people and that it continues to operate as intended. The application owner is normally a busy member of staff and hence is unlikely to have sufficient time to look after and administer the system on a day to day basis.

Hence, the auditor will frequently find that the day to day administration of the system is delegated to an application or systems administrator.

Application administrator

This person would be responsible to the system owner. Typical tasks carried out by a systems administrator include:

• maintaining a list of authorised application users;

• adding and removing users from the application’s security profiles; • ensuring that the IT department has backed up the data according

to back-up policies;

• resolving queries from application users;

• identifying, monitoring and reporting significant problems to the application owner and IT department;

(24)

Notes 4 Session 4

• Liaison: liaising with the IT department on system performance, e.g. Storage requirements, response times. The administrator would also liaise with other application administrators and the software suppliers.

The administrator of a finance application may or may not be part of the finance department. There are two trains of thought on this issue. If the administrator is part of the finance department, then there is a risk that he or she will be more aware of user problems, s(he) will know what access rights users should require to carry out their jobs, and s(he) would be more up to date on starters and leavers. However, if the application administrator is part of the finance area then there is a risk that s(he) would know too much about the working practices, staff personalities and the manual controls which operate around the system. This increases the risk that the systems administrator would have too much knowledge of the system combined with high user privilege levels on the system.

Others believe that the application administrator should not be part of the business to which the system relates, e.g. the administrator of the finance system should not be part of the finance department. If they are not part of the finance department they would not be aware of any manual procedures around the system which would pick up attempts of manipulation or fraud.

Most clients adopt the view that the finance system administrator should be part of the finance department. Where this is the case the auditor should ensure that there is adequate segregation of duties, i.e. ensure that the application administrator does not have other line duties within the finance function, e.g. the system administrator should not also be the purchase order clerk.

Ordinary system users

These are the day to day users of an application. They are given access to parts of the system they require to do their jobs. Their training on the system is limited to the user functions needed for their jobs.

(25)

Notes 4 Session 4

Essentially they have to make sure that they know how to use the parts of the system they are authorised to use. They have little knowledge of systems administration duties.

Application types

Applications are normally categorised according to the way data is entered and available to users. The main categories are:

Batch data entry systems:

Batch data entry systems are normally associated with older financial systems. Transactions are normally passed, by users, to a central IT or finance data input section, where they are batched up and entered onto the computer in batches.

Batch data entry with on line enquiry

Transactions are entered in batches as before, but users can make enquiries on individual transactions as they are entered. However, the effect of the new transactions on balances, account totals, summary figures etc. is not calculated until the entire batch has been input and the batch processing cycle completed.

Batch processing with on line enquiry and update

Users enter transactions at workstations. They can amend erroneous transaction data on screen. The user sees what appears to be an updated transaction record. This is because the transaction the user enters updates a copy of the master file. However, the real copy of the master file is not updated until a later time, e.g. overnight updating of the master files.

Real time systems

Real time systems are the latest development in computerised financial applications. Users can enter transaction at a workstation and see an immediate effect on the account balances, i.e. the accounts are updated whenever a user enters a

(26)

Notes 4 Session 4

transaction. Real time systems allow users to correct errors immediately and improve the timeliness of management information. However they don’t include the traditional controls such as batch totals.

External audit reliance on application controls

Going back to the financial audit model. The auditor should plan an efficient and effective audit approach. To do this the auditor should obtain an understanding of the transaction processing cycle and the systems of internal control.

During preliminary planning the auditor should gain an understanding of the client’s systems sufficient to determine whether it would be possible to rely on the systems of internal control to gain audit assurance on the accuracy of the client’s financial statements.

Where the systems of financial control appear to be sufficiently robust the auditor may then test the controls. Depending upon the number of control failures, the auditor may place audit reliance on the controls and reduce the amount of subsequent substantive testing that must be carried out.

The auditor may place reliance on controls which reduce the risks to one or more of the audit objectives. These include Completeness, Occurrence, Measurement, Presentation and Disclosure etc.

Compliance test programmes

To place reliance on a client’s controls the auditor should develop a compliance test programme. These programmes should include the following information:

• a description of the control that is to be tested;

• the evidence that we expect to obtain to satisfy ourselves that the control has operated;

(27)

Notes 4 Session 4

• what will constitute a control failure; and

• how many such failures can be tolerated before the auditor cannot rely their operation to reduce errors.

Testing controls is achieved by gaining evidence that a control has operated. For example, if a computer enforces segregation of duties between various finance functions, the auditor may check the access control lists and the user permissions on those lists. Alternatively, if the computer has automated controls to ensure that purchase order clerks can only order goods from a predefined list of approved products from approved suppliers (regularity assurance), the auditor may check the access controls to the approved products list, together with a sample of new additions to the list.

Evidence that the computer controls have operated will normally be obtained from a combination of observation, enquiry, examination and sampling. The auditor may also be able to use computer assisted audit techniques to assist in the examination of controls. For example, the auditor could download the audit log file and write a routine to extract unauthorised access attempts. The auditor could follow these up to ensure the proper procedures were followed.

Problems with placing reliance on computerised controls

1. A problem with testing computerised controls is that they are often built into an application and they may not provide a history of their operation. In manual systems when a check was carried out by a supervisor or another member of staff, a piece of paper such as an invoice or a goods received note would be signed. The auditor would take the signature as evidence that the control has operated. With computerised systems the control may take the form of the supervisor signing an electronic goods receipt note directly on a computer screen. Evidence that the person electronically signed the electronic document would be obtained from testing the access controls to the good receipt approval screens.

(28)

Notes 4 Session 4

2. The spans of computerised controls are often limited. For example, they cover just one small stage within the processing lifecycle of a transaction, e.g. the data entry phase. There are many other stages in the processing cycle where things could go wrong and where additional controls testing would be required. Auditors normally prefer to place reliance on controls with a wide span of influence, e.g. bank reconciliations.

3. Auditors are often reluctant to place reliance on computerised controls because of the fear of hackers and computer literate fraudsters. Auditor fear that these people can get around many of the automated controls in financial systems. If the auditor places reliance on a control and it subsequently turns out that they were easily bypassed by a computer literate thief then the auditor would feel very uncomfortable.

4. Another problem associated with testing computer controls is that they are preventive in nature, i.e. they check data automatically when a user attempts to enter it and reject erroneous transactions. By their nature, auditors are more reluctant to place reliance on preventive controls. They ask themselves questions such as do we know that the control was working at the time, since no errors were discovered and recorded. This problem can be addressed by the auditor actually testing a control during the accounting period. For example, the auditor would visit the client during the year and attempt to input transactions which the systems controls, if operating correctly would prevent. Auditors should remember to get the client’s approval before doing this as any erroneous transactions which bypass the controls will have to be identified and reversed at a later date.

Amount of controls testing required

The next question the auditor asks is how to gain the required (planned) audit assurance. This decision is likely to affect the level of controls testing required. When determining how much controls testing is necessary auditor should base any

(29)

Notes 4 Session 4

decisions on a combination of professional audit judgement and statistical modelling. The factors affecting the auditor’s judgement include:

• the frequency of a control : for example, if the auditor plans to place reliance on monthly reconciliations, the control could be tested 12 times in an accounting period. Where controls are frequent they may need to be tested more often;

• degree of reliance on the control : If the auditor intends to place significant reliance on the control or the auditor has assessed that the control mitigates a particular risk in another part of the system, the auditor will need to carry out more testing;

• evidence: the amount of testing may be affected by the evidence available that the control operated as intended. If direct evidence is not available, the auditor may need to carry out more controls testing. As explained earlier many computer controls do not produce direct evidence that they operated and therefore, the auditor may need to gain indirect evidence of its operation;

• continuous operation of the control : the auditor will need to determine if the control operated during the whole accounting period. This will require evidence and possible visits to the client at various intervals during the year; and

• importance of the control : the audit should consider the importance or sensitiveness of the transactions the controls are applied to. If the transactions are particularly sensitive the auditor may need to carry out more in depth controls testing.

Where the auditor plans to test controls which are performed on many transaction the auditor will need to determine a sample size. The sample sizes will be largely dependant upon statistical calculations and any audit assurance models the SAI has adopted. The sample sizes will also be dependent upon the auditor’s judgement. Any judgement decisions will be affected by the factors described above.

(30)

Notes 4 Session 4

Application auditability and the use of computer assisted audit techniques (CAATs)

Introduction

This chapter looks at what the auditor would do when reviewing a client’s financial systems to ascertain whether these requirements have been satisfied.

This chapter only briefly covers computer assisted audit techniques (CAATs)., and concentrates on using generalised audit software to support the financial audit process using sampling and data interrogation procedures.

Audit trail and auditability

What is meant by an audit trail and why is one important?

When confronted with a client’s financial system the auditor should remember the aim of the audit. Where the aim is to provide an opinion on the financial statements the auditor will need to determine how to get the evidence necessary to provide that opinion.

The audit evidence will usually be gained from carrying out a combination of compliance and substantive testing procedures. It is likely that financial auditors will have to carry out a certain minimum amount of substantive testing procedures, even when a client’s internal controls are deemed to be strong.

Without an audit trail it will be difficult, if not impossible to carry out direct substantive testing of individual transactions. In the absence of an audit trail the most probable outcome would be that the auditor will not be able to provide an unqualified opinion on the client’s accounts. The auditor would probably have to state that the client was unable to provide satisfactory evidence to support the accounts (what is usually known as a limitation in the scope of the audit opinion).

In a few situations, where there is not adequate audit trail the auditor may be able recreate one using external sources of evidence, e.g. information from suppliers,

(31)

Notes 4 Session 4

clients, or sponsoring departments. However, such an approach is unlikely to be followed as the evidence may not be complete and process will probably required the input of considerable audit resources.

Substantive procedures normally include testing individual transactions against the audit assertions (completeness, measurement, occurrence etc).

The audit trail can go in two directions. Firstly there is the audit trail which starts at the financial statements and works back to the individual transactions which are summarised in the balances in those statements.

Secondly, there is the audit trail which starts at source documents and works it way up to individual account balances which are then grouped together and summarised in the financial statements.

Financial statements to individual transactions

The first audit trail, i.e. from the financial statements back to the transactions is required to satisfy the Occurrence, Measurement, Rights and Obligations , Existence, Valuation, Regularity and Disclosure objectives. To obtain substantive assurance for each of these objectives, the auditor will need to select and test individual transactions which make up the summarised figures in the financial statements. This requires the auditor to be able to take summarised balances from the accounts, traces the summarised balances back to the trail balance and then be able to obtain a list of transactions which make up the trail balance figures. The auditor can then select transactions from that list and check back to supporting documentation, e.g. invoices, purchase orders, staff pay files etc. This will allow the auditor to determine if a transaction results in a misstatement in one or more of the audit objectives.

Individual transactions to the financial statements

The audit trail which starts at source documentation, and end with the financial statements is necessary to provide assurance for the completeness objective.

(32)

Notes 4 Session 4

Completeness is defined as: there being no unrecorded assets, liabilities or transactions which should be recorded”. To gain evidence to satisfy this objective, the financial auditor will need to select transactions at their source and trace them through to their inclusion in the financial statements. For example an auditor would select a sample of transactions from the client’s cash receipts book and confirm that all the receipts have been recorded in the accounts.

What should the auditor ask the client?

To find out whether there is an audit trail the auditor will need to ask questions which will allow him/her to check the following:

• whether the transactions records for the accounting period are available for audit inspection. The auditor should determine if the client has an archiving policy and whether it would affect the accounting records. Some clients have been known to only store that last few months detailed transaction data on the computer. Any transactions more than a few months old are summarised into brought forwards figure. Other clients may not store the documentation supporting individual transactions as they believe it takes up too much space; The latest development in the retention and storage of transaction documentation is

document image processing (DIP). This involves the client scanning documents using flatbed scanners and storing the resultant image on a CD ROM. The auditor should decide whether such evidence is acceptable. This decision should take account of any legal precedents, local auditing guidelines and the auditor’s judgement. Provided that the controls surrounding the scanning process and the storage of CD ROMS are acceptable, DIP systems are probably an acceptable form of audit evidence.

• whether it is possible for the auditor to obtain a transaction listing for each account code. The listing may be on screen. on a hard copy printout or on magnetic disk or tape (for CAATs). The auditor can use the transaction listing to select individual transactions for testing; and

(33)

Notes 4 Session 4

• whether it is possible, for each transaction, to trace an individual item, from source documentation to a particular account code on the financial system. The link between individual transaction and the accounts is normally achieved by a unique identification number or code.

For example, when reviewing fixed assets accounting system the auditor may need to select physical assets from the client’s premises and have a means of tracing them back to the fixed assets register. This would normally be achieved by the client assigning each asset a unique fixed asset registration number.

Similarly, if the auditor was reviewing a purchasing system the auditor may look to see if each purchase order is assigned a purchase order number.

The auditor should also assess the auditability of the system, i.e. even though an audit trail may exist, how easy is it to follow and what facilities does the client have to make reviewing transaction data easier.

This may range from having access to an on-line enquiry terminal, to the client storing documents in a systematic manner e.g. storing invoices in numerical order.

The auditability can affect the auditors’ resource requirements calculations. If it is time consuming to extract and examine each transaction then more staff resources will be required.

Computer assisted audit techniques (CAATs)

Computer assisted audit techniques are used to improve the speed and efficiency of audits. They can also be used to carry out selective testing of transactions which the auditor believes to be of higher risk.

CAATs for sampling and reperformance

Whenever an IT auditor is tasked with reviewing an application, one of the questions invariably asked by the financial auditor is “can I use CAATs to extract samples or interrogate certain account codes ?”

(34)

Notes 4 Session 4

To be able to use CAATs for sample extraction the auditor will need to determine:

1. if the client’s system can produce a listing of transaction for specified account codes. Financial data is typically stored in a financial database (increasingly in a relational database such as Oracle, Ingress or Sybase). The client may already have pre-defined reporting facilities, e.g. standard reports, which can provide the information the auditor requires. However, it is more likely that the auditor will need to specify exactly what data records, and fields are required and then have the client extract the data using the systems report writing facilities . For example, this may involve the client writing out an extraction query using structured query language (SQL);

2. if the client is able to write a report which extracts the required transaction details the auditor will then need to confirm that the data is complete. This is normally achieved by reconciling the report to the client’s ledgers;

3. once the auditor has conformed that the report contains the required information and that it reconciles to the client’s ledgers, the auditor will then need to remove the report from the client’s system to the auditor’s own computer. This may involve transferring the file over a physical link (e.g. via a personal computer’s parallel port) or via magnetic tape or disk. The auditor should ensure that there are facilities to deal with the data transfer, e.g. if the client can produce the data on a half inch tape, the tape is only of use if the auditor has the equipment to read the tape;

4. the way in which data is stored may also be important. For example, many off-the-shelf audit interrogation packages can accept data in a number of forms e.g. ASCII, EBCDIC, comma separated, fixed length records etc. Data storage details can be quite technical and are covered in the INTOSAI CAATs module.

(35)

Notes 4 Session 4

Other types of CAAT

The client’s applications may have in-built CAATs or audit facilities such as integrated test facilities (ITFs) or embedded audit modules (EAMs).

Where such facilities exist the auditor should document what they do, and make an assessment of whether they can be relied upon to provide audit evidence.

Application security

Relationship between general security controls and application security controls This chapter considers the physical controls within the finance and the logical access controls designed into individual applications.

The physical and logical access issues considered previously concentrated on what was done in the IT department. This chapter looks at what is done within the users’ own departments, e.g. finance , personnel etc. Hence, when attempting to find out about the security controls within an application the IT auditor would need to ask the application administrator within the user department.

Many of the principles in this chapter remain the same as those covered previously and there is a degree of repetition, for example the principle of restricting user access according to their needs.

Purpose of applications security controls Application security controls are designed to:

• ensure the integrity of transaction and standing data by allowing only authorised users to access specific application functions and data;

• enforce separation of duties by allocating different privilege levels and menus to individual users; and

(36)

Notes 4 Session 4

• make users accountable for their actions by logging their identifiers against their actions.

As mentioned before, security is all about identifying and authenticating users, protection resources (in this case the financial data and the organisations money), logging and accountability.

The security administrator within the IT department should have ensured that these controls exist at the operating system level. The application administrator would ensure that there are appropriate access and security controls within their applications. For example, the security administrator in the IT department would ensure that only authorised employees would be able to log on to the network. S(he) would also ensure that only authorised members of the finance department would have access to the directory which held the accounting application.

The application administrator would then deal with the next level of security and ensure that only finance department staff can access the application. In addition, once finance staff have gained access the administrator could then ensure that they are only allowed access to certain modules, e.g. the sales module, the purchasing module or the fixed assets module, provided that they have been granted permission to use those modules. The application administrator may even be able to control what a finance user can do within each module.

Commonly encountered application access controls

The extent and number of controls will vary from client to client . The auditor should use judgement when assessing the application controls and should be careful when putting forwards recommendations for improvements. It may be difficult and or costly to bolt on any controls recommended by the auditor.

In addition many clients use off the shelf financial applications with standard access control mechanisms. Where this is the case the auditor should find out what

(37)

Notes 4 Session 4

access controls are available and determine whether the client has configured them properly to derive maximum benefit.

Application log-on screens and password controls

When users select the accounting system option from the operating system menu, they “load” the application. The application immediately asks them who they are and what are their passwords. Users are normally required to enter a short ID code followed by their password. This process identifies and authenticates the finance system’s users.

As with logon procedures at the operating system level the auditor should review the application’s ID and password policies and procedures and controls. For example the policies on password composition, the use of unique ID codes, password ageing policies, allowing users to change their passwords.

The log-on controls built into applications may vary considerably from very good to non existent. Even where an application has good basic password controls, much will depend on how the client implements the access controls. For example the application may have the ability to force users to change their passwords, however if the client does not activate this option it is of no benefit.

Unfortunately the log-on controls within many financial applications are weak. The application may only be able to handle passwords of up to 5 characters in length, there are frequently no password ageing controls, no automated controls over password composition, no facilities for users to change their passwords (i.e. passwords are allocated by the application administrator).

The auditor should check that the application administrator has adopted and makes use of manual procedures to keep password access files and log-on procedures up to date. For example, maintaining contact with the personnel and the IT department to identify starters, leavers and those staff who move from one part of the business to another (e.g. to and from finance). This should enable the application

(38)

Notes 4 Session 4

administrator to ensure that only current, authorised users are able to log-on to their application.

Application administrators may not be as conversant with good security practices as the security administrators in the IT department. For example, they are more likely to forget to disable the guest accounts which are used when a system is set up by a supplier or visitor accounts for temporary staff.

Resource protection

Once users have navigated the log-on screen and successfully gained access to the application they may then be confronted with a menu screen which contains the names of the application modules.

Financial applications may have facilities to control access to function within modules. A common way of doing this is to either:

1. for each authorised module user, specify which permissions are granted or disallowed.

2. for each job function e.g. sales clerk, sales supervisor, cashier, etc, specify what functions are permitted. Individual users are then assigned one or more job functions. This method is preferred because it makes the job of the system administrator easier.

Logging and accountability

This is the third area of application security which an IT auditor should review. The basic principle is that users should be held accountable for their actions. Where users can be held to account, they are less likely to make errors (if the error can be traced back to you, it is likely that more care will be taken) or carry out unauthorised activities.

In the past, when manual systems were dominant, user identification and accountability could be achieved by users physically approving (signing, initialling or stamping )financial stationery, e.g. purchase orders, goods received notes, cheques

(39)

Notes 4 Session 4

etc. In an increasing number of today’s modern computerised accounting system the number of manual forms and paper based financial stationery has declined. There has been an increase in both direct computer input and the use of electronic documents (e.g. EDI).

Where transactions are processed by a computer, applications may produce an electronic audit log. Logs are used to record transaction details such as:

• what transaction was entered (details of account codes, amounts); • who entered the transaction;

• when was the transaction entered; and

• amendment details ( who changed what, when?)

Where an application can produce a transaction audit log the auditor should review what the log has been set up to record.

A large amount of information can be written to an application audit log. The auditor should assess whether too much or too little information is logged. If too much is logged the system’s performance is likely to deteriorate. If too little is recorded it may not be possible to find out enough about users and transactions when errors, fraud or other unauthorised activities occur.

The audit log acts as a detective control and is only of use if it cannot be altered or deleted by somebody trying to cover up details of their unauthorised activities. To ensure that the log is safe from unauthorised editing or deletion, the auditor should review access permissions to the log. Normal users should not be able to create edit or delete the audit log. The log may be protected by placing it in a protected directory. Access to the directory would be restricted to a few users (e.g. the application administrator).

Note : directory permissions are likely to be controlled by the IT department rather than the application administrator.

(40)

Notes 4 Session 4

Another way of protecting the audit log would be to write it to a WORM drive.

Some applications may be able to encrypt the audit log. This would make it difficult for an unauthorised user to identify which parts of the log need to be amended to hide their activities.

Input, processing and output controls Introduction

This considers the controls which are applied, either manually by application users, or automatically by a computer system, to an organisation’s financial transactions.

The overall objective of a system of control over any accounting system is to ensure the completeness, accuracy and validity of the accounting records. It should be borne in mind that the objectives of control will be exactly the same for all accounting systems, from simple manual operations to complex real-time database systems. However, the methods of achieving the desired degree of control will vary according to individual circumstances.

Irrespective of whether a financial system is manual or computer-based system, the validity of entries is best ensured by some method of authorisation.

The auditor can place greater reliance on the authorisation of the output from a system than can be placed on the authorisation of input, as this achieves the authorisation of what is actually processed rather than the authorisation of what should be processed.

The financial statements are the product of an accounting system and the main control objectives will be satisfied if procedures are in place to exercise control over the completeness and accuracy of: