RIAM:Overview: The Assertion Linked Systems Based Audit (ALSBA)

Introduction

The keys to the method are structure and focus. With RIAM, Internal Audit applies a Systems and Assertion Based approach to answer targetted 'questions' about an audit area. Questions focus the review, while assertions define the criterion for the answer.

Many systems based approaches merely measure the compliance of an organisation's staff with a particular system. The ALSBA method is a substantial enhancement to this commonly used approach. The RIAM auditor analyses compliance of the system process with a strategic and/or tactical purpose, compliance of practice with procedure and the awareness and readiness presented by potential (risks and opportunities) in the system itself.

The avoidance of checklists makes the auditor adaptable by permanently adopting a 'learning' posture. More importantly, the process is universal in that the same logic structure can be applied from the strategic level through to transactional compliance level, and from 'hard' financial processes to 'soft' subjective process.

Very large organisations present some particular challenges for the systems based audit, including coordination of teams across multiple jurisdictions, locations and organisation units. Here we present an overview to the technical aspects of the RIAM SBA, in Conduct of the Very Large Audit we explore the method in detail in both the large audit and small audit context.

What Are Assertions?

The figure on the preceding diagramme summarises the Assertion Linked Systems Based Audit analytic structure. The process starts with the five areas of Internal Audit's "Scope of work" within which Assertions are defined. Support for the selected Assertions is classified into management's 10 Control Classes (areas for management action). The systems built by management to support the Assertions within the Control Classes will have identifiable "Control Attributes" identical to those used in our Control Implementation Service, and are classifiable according to the "Type" - preventive, detective or corrective.

The concept of Assertions is the core of a RIAM Systems Based Audit. Assertions are truths that we wish to express about a system. They formulated as statements of "fact" about a system. Examples of typical Compliance Assertions for financial aspects of a Grants Scheme might be:

That:

a. Grant expenditure is bona fide (ie that acquittals are for actual grants and for services appropriate to grant activity);

b. Grant data reported/processed is:

• Attributed to the proper period,
• Accurately calculated,
• Correctly and appropriately accumulated,
• Accurately recorded,
• Correctly disclosed,
• Properly authorised with respect to transactions (ie grantee approved costs and the Commission is satisfied that the amount is for an appropriate expense),
• Providing benefits to which grantees are eligible,

c. The relevant management directions and legislation are observed:

• Payments are in accordance with legislation, and
• Approvals for grants are in accordance with the legislation (ie properly vetted by the Grant Committee and approval is given by the Board); and

d. The assets of the organisation are efficiently, effectively and otherwise appropriately protected and applied (ie having an appropriate process of grant approval that assures projects are of an appropriate standard, and that Commission resources are used efficiently).

For What are Assertions Used?

When we say a given system is operating satisfactorily we mean that our review has tested the truth of a set of assertions and we have found that they have been sustained. Thus testing the assertions is the purpose of the audit.

Assertions are the focus and underlay the structure of the RIAM analytic method. All review activities, findings, discussions and recommendations must be able to be tied back to the review's assertions.

The result is that both the auditor and the auditee have a precise understanding of the level of comfort a given review offers.

Assertions have another huge advantage for the auditor: They allow us to frame focus questions about a system in "yes" or "no" form, which are answered by proving or disproving the assertions. For example, the question "Is system XYZ operating effectively?", is, by its nature, subjective. My meaning of the word 'effective' may be radically different from your understanding of theat same word. If we say, "effective in this context means accurate and timely" then we both know that neither of us meant "authorised and consistent", or "fair and equitable".

Thus by combining a focus question, with the assertions that define a "yes" answer, we as auditors, can give management and the governance committee what they want: certainty. We do not need to hedge for the unknown - because we have stated clearly our context specific meaning.

Thus we say that assertions are the definitions of the audit project focus question.

For a detailed discussion of assertions and example assertion sets in various kinds of systems see:

• PHASE 2: ASSERTIONS

How do we Establish Assertions?

A reviews assertions are agreed with the auditee management before a review commences. In many cases, such as financial balances audits and quality audits, we are able to recommend appropriate assertions. In other reviews, particularly those specifically requested by management, the managers will have a clear idea of particular "Questions" they wish answered by the review.

The establishment of "Questions" is the first step in selecting audit assertions.

During the entrance interview phase of the audit management identifies a number of questions about the target system they wish to have answered. The auditor then proposes a series of Assertions, the sustaining of which will constitute an affirmative answer, and the suppressing of which will constitute a negative answer. These assertions are agreed with management.

What is the Assertion Linked Systems Based Approach?

The Objectives

The objectives of the reviews are summarised as:

• Document the procedures in operation within the section so far as they relate to the target activities;
• Collect sufficient data and analyse that data to support assertions that address management's critical success factors represented by questions they request audit to answer;
• Identify risk and efficiency exposures to the organisation and the critical success factors of management;
• Recommend relevant and practicable changes in the systems and procedures to management where these exposures are present; and
• Form an opinion as to the overall reliability of the systems in place and as modified.

Meeting The Objectives

The structure of the approach, diagrammed above, that meets the audit objectives has four phases. Here we summarise those phases. A more detailed discussion of these phases mapped into the context of both small team and large multi-location, team audits is explored in:

• The Four Phases of the RIAM Systems Based Audit

PHASE 1: FAMILIARISATION, SCOPE AND PLANNING Define View of the Audit Area, Establish Risks, Threats & Benefits expected by Management. Identify the objectives and purposes of the section being reviewed, and the review being conducted; document critical success factors. Entrance interviews are held with senior management during which management's concerns and directions are communicated as well as the Critical Success Factors of the audit and the section being audited. Certain objectives, such as legislative compliance, are always assumed to be present; Identify the functions in place to realise the objectives, critical success factors and purposes. A series of initial interviews are conducted with relevant middle and line management and staff to: Introduce the review and reassure staff as to the assisting rather than policing nature of the review, Identify the operations and organisation structure adopted to meet the objectives, purposes and critical success factors. Set Focus Questions, Audit Scope, Boundary & Assertions Establish focus questions and their associated answering assertions , the satisfaction of which will represent a "pass" result. The assertions represent the criteria for evaluation; This topic is explored in more detail in: ALSBA - Phase 1. FAMILIARISATION, SCOPE & PLANNING in the Very Large Audit

PHASE 2: DOCUMENTATION AND SYSTEMS ANALYSIS Systems Description Build functional description of the area under review, focussing on the ten Control Classes or other appropriate classification of management action areas. Build a cyclic description of control systems, examining both time based cycles and data flows. Investigate the control systems in place to implement the functions. Tasks include: Document the procedures in operation so far as they relate to the scope and boundary of the Audit task, Compare actual procedures to legislation, policies, guidelines and documented procedures noting exceptions; Examine management information and reporting systems in place to monitor the operations; Threat Causing Assertion Failure & Controls Addressing Threats Evaluate the systems against the assertions to be supported, noting key controls in the systems, and which assertions they affect, to determine: Potential strengths and weaknesses of the designed systems; Preliminary ranking of risk and exposures including efficiency exposures. More detail is available on this topic in: PHASE 2: DOCUMENTING & ANALYSING THE SYSTEM OF INTERNAL CONTROL PHASE 2: ASSERTIONS

PHASE 3: TESTING AND RESULTS ANALYSIS Test Systems Design a testing program and Test the system and its transactions and/or data for: Compliance of operations with specified system (strengths); Occurrence of the identified weaknesses, risks or exposures; Evaluate Results Analyse the results of systems analysis and compliance testing stages to accept or refute the established assertions and operating compliance.

PHASE 4: REPORTING AND FOLLOW UP Design Corrections Conclude and report in which we: Identify risk and efficiency exposures to the Organisation; Recommend changes in the systems and procedures to the Organisation's management where these exposures are present; Form an opinion as to the overall reliability of the systems in place and as modified; Report to both management and the Audit Committee after and during each task; The control system's ability to support the assertions and therefore the key controls identified are analysed at three levels: Preventive Controls/Treatments Including direct controls such as authorisation and certification of forms, indirect controls such as training, maintenance of up-to-date reference material, section administration and organisation; Detective Controls/Treatments Such as supervisor review, batch control totals, edit checks and periodic system reconciliations; Corrective Controls/Treatments Such as routing an error back through the same control system that originally processed and detected the error and response to exception reports. Gain Management Ownership Conduct exit interviews, produce the final report and review action plans as required. Although steps presented here suggest a linear sequence of steps, the correct approach involves regular, on-going to management during the conduct of the review. Interim reports, either formal or informal should be provided during the review. The key factor is that there should be NO SURPRISES for management at the end of the review. This facilitates ownership and acceptance of the findings, recommendations and the audit generally. Classify Findings, Facilitate Action Plans and Update the Organisation Risk Model The final stage of the Review is to formalise the findings and recommendations by classifying their effects on the risk evaluation of the organisation and feed these back into the risk model. The risk model both provides an ongoing measure of the organisation's risk level, and eventually feeds back into the planning process for the identification of either further action or necessary reviews.

Establishing the framework

The key principles of the framework include:

• Interviews to scope and focus the review and involvement of Management and Staff throughout the process;

• Ensuring agreement as to the purpose, focus, scope, boundary, approach and findings of the review;

• Assertions as criteria for evaluation.

• Application of Risk Analysis, not just at the Planning stage, but also the Threat Analysis stage when assessing Systems Design, and the Reporting Stage when finalising recommendations. The Audit Risk is the risk that the audit will provide a wrong opinion. This is a function of:
  • The Inherent Risk in the organisation
    • the risk that an error is likely to occur;
  • The Control Risk
    • the risk that the control system will not prevent, detect or correct the error; and
  • The Detection Risk
    • the risk that our procedures will not identify the existence of a material error.

The ALSBA uses Assertion focussed Risk and Threat analytic procedures to minimise this risk.

• Risk and Threat analysis aims to minimise the cost of reviews by keeping procedures tuned to the real exposures, and when combined with assertions, raises the certainty that our systems opinion is correct.

• Use of a variety of report and presentation styles to best communicate information; and

• The Internal Auditor MUST become part of the management & systems improvement process, not a disinterested, occasional observer.

• Analysis of control systems performance in meeting objectives.

• Clear discussion and specific recommendations to provide improvements.

What is Threat Testing?

Threat testing is an approach to assertion testing used as an alternative to a Desired Control Model. RIAM supports both concepts.

The key benefits of threat testing are:

• Controls analysis is kept current to the ACTUAL systems in place rather than an out-of-date control model;

• The audit process recognises and supports improvement and change in systems - essential for environments where Total Quality Management is operating;

• By evaluating the sources of possible problems, the process RESULTS in the development of Desired Control Models;

• Management is involved in the assessment of risks of systems failure;

This is a brief outline of the Threat Testing process :

• Each assertion is examined in turn. For each assertion a list of causes for failure of an assertion is prepared based on experience, statistical sampling, management advice, consultant advice, and checklists, etc. These causes are called threats. To each threat a probability of occurrence may be assigned if desired (perhaps based on historic samples).

• Each threat is then applied to the control system model (developed during the systems documentation phase) to investigate the probability of the system preventing the threat (ie. mitigating the risk). This probability is expressed as a probability of system failure.

• The risk of the threat occurring multiplied by the risk of system failure (Control Risk) is probability of the assertion not being sustained in operation.

The sum of all such threat related probabilities is the total risk of assertion failure in the system.

How Do We Document Systems?

Working Papers

RIAM working papers are designed to form a "tree" or pyramid with the apex being the opinion of the systems in operation, and the base being the detailed "views" or models of the organisation's systems and the testing results verifying aspects of the system's operations.

REF	CONTENTS
1	Final Audit Report and Other Relevant Files
2	Supervisor, Manager & Partner Reviews and Follow Up
3	Engagement Letters, Contract and Contacts
4	Action Plan, Client Follow Up and Correspondence
5	Matters for Manager & Partner Attention
6	Matters for Review Next Audit
7	Planning Documents and Audit Program
8	Work & Time recording Schedule
9	Background and Organisation Details
10	Organisation Objectives, Operating & Financial Policies, and Performance Measures
11	Strength & Weakness Schedule
12	Control System Documentation and Conclusion (Control Questionnaires, flowcharts, checklists and narratives)
13	Records of Interview
14	Legislation and Management Directives - Compliance (Including Important Contracts and Agreements)
15	Analysis and Tests of Transactions, Processes and Account Balances
16	Other Background Data and Notes

 The Index for The Standard RIAM Audit File

The foregoing index shows that the files are self contained units including not only plans and tests, but also:

• date records of client contacts;
• relevant legislation and directions;
• full internal and external cross references;
• systems documentation; and
• organisation background and structures.

Section 12 of the file contains the detailed analysis of the systems under review:

PHASE	ACTION	WHO	REF
1	Conclusion	12.
2	Objectives (Purpose) of the Control System	12.
3	Framework of Analysis (Assertions to be supported)	12.
4	Key Controls	12.
5	Overview of the Control System (Principal Flows)	12.
6	Control System Flowcharts/Documentation	12.
7	Files & Records in the System	12.
8	Cycles in the System	12.
9	Transactions and Value	12.
10	Documents in the System	12.
11	Segregation of Duties	12.
12	Other	12.

 Index for Section 12 of the Standard Audit File - Control System Documentation

The continuation of the "tree" structured analysis is evident in the above index. Each subsection contains further structured working papers, the details of which can be found in the volume "Standard Forms & Papers" of this series.

Methods of Analysis

The working papers require that systems design is analysed BEFORE any testing is performed. While prewritten test programs can be used, the full benefit of the method is received when the systems analysis is performed using the various systems models:

• Segregation of Duties Chart
• Client Provider Analysis
• Key Quantities (transaction values and volumes)
• Cyclic Events
• Annotated Data Flows, Narrations and/or Document Flows
• Key Controls structured by their "data flow focus":
  • Inputs
  • Processes
  • Outputs
  • Storage

And evaluated within the assertion/control attribute structure outlined earlier.

The Working Paper's documentation of the system culminates in the Assertion Matrix and Control Strength & Weaknesses Chart. The Systems Analysis of section 12 of the file is summarised in these charts.

Within the assertion structured systems model are many subsystems. These are documented throughout the documentation "tree". Each subsystem should be documented in the way that best suits our analytic needs. For transaction flows this might be some type of annotated data flow, for a delegations analysis it might be an organisation chart, and for a risk analysis it might be a Fitzgerald Matrix, etc.

There are a number of techniques available to the auditor for use in documenting systems of internal control, such as:

• Narration
• Process and Document Flows
• Annotated Data Flows
• Organisation Charts
• Segregation of Duties Chart
• Assertion Matrix
• Lancaster Modelling
• Algorithm Pseudo-programming
• Simulation

Irrespective of which method is chosen, documentation should include:

• the origin of every document and record in the system
• all processing that takes place on the document
• the disposition of every document and record in the system
• a description of internal controls operating within the system

What Are Some of the Types of Reviews Conducted Within the ALSBA?

Management Assurance services utilising the ALSBA cover the full range of Internal Audit work including:

Internal Audit Unit Performance Review; Efficiency and Effectiveness Reviews; Compliance and Integrity Reviews; Strategic and Tactical Planning Reviews; Financial Audits; Systems Analysis and Design Review; Quality Audit (TQM); Computer Controls Implementation; Methodology Design and Development Review; Control Systems Design;

Training Review; EDP Reviews (15 different types); Corporate Design and Planning Reviews; Risk Management Review; Change Control; Occupational Health & Safety; Inventory Management; Maintenance Systems; Process Control; Fraud Control; and Quality Management System Integration.

How Do We Report?

Ultimately the product produced and of greatest significance to management is the report. Our reporting is standardised to ensure consistency of structure, coverage, presentation, language and quality.

The significant features of our reports include:

• Standardised structure;
• Systems documentation and flow charts;
• Every finding is presented with: "Observation", "Risks and Implications", "Recommendations", and "Management Comment" sub sections;
• Clear, specific and relevant recommendations, not vague references to the need to "review" an area or "correct a problem";
• Clearly argued risks and implications of each finding. An observation is analysed by:
  • The assertions affected,
  • Risks and exposures from the observation,
  • Arguments in favour of the breach and audit's comment on that argument;Inclusion of and focus on Action Plans; and
  • Linking of findings to a clearly stated premise for the finding's importance: The Assertions affected.

Although the Report structure is one of the aspects of RIAM specifically tailored to the client, most adopt a close variation of one standard structure. RIAM includes five distinct report structures to assist clients identify their reporting needs.

The report is presented under the following headings/sections:

1. Executive Summary
   Provides a summary of the purpose, objectives, assertions, approach, scope, the overall opinion, key findings and issues arising.
   
2. Objectives and Approach
   Addresses the "How" and "Why" of the review, and defines the assertions on which the conclusions and findings are based.
   
3. Scope and Boundary
   Clearly defines the matters covered by the review, and most importantly the matters excluded from the review.
   
4. Brief Description of the System Reviewed
   Covers the Purpose of the Section/Systems, The People and Organisation Structure, the Principal Activities of the Section/Systems, Documents and Records (both manual and computer) and the Reports Produced from and to the Section/Systems.
   
5. Checklist of Findings, Recommendations and Action Plans
   Presents in Landscape form a summary of the findings and recommendations in section 6 under the headings: "Findings" and "Recommendations". Tables include boxes for Action Plans to be referenced or detailed. This section assists in monitoring and following up responses to audit recommendations by the Audit Committee.
   
6. Detailed Findings and Recommendations
   The findings and recommendations have a standard structure:
   
   • Observation
     • The observed facts, relevant legislation, directions and industry relevant information.
   • Implications and Risks
     • Assertions suppressed or supported.
     • Principal risks and exposures.
     • Arguments in favour of, or reasons for, the breach and audit's comment.
     • Summation of audit's conclusion as to risk or exposure.
   • Recommendations
     • Numbered, clear, specific and relevant recommendations for action.
     • Where alternatives are identified either by audit or the client they are presented and evaluated.
   • Management Comment
     • Management's response to the issues raised and action taken. After discussion and exit interviews the vast body of your recommendations should be accepted by management. If not, you have not done your job correctly!

BackLinks

• Contents
• Internal Audit Method
• RIAM:Conduct of the Very Large Audit
• RIAM:Overview: Control Implementation Services (CIS)
• RIAM:Overview: The Five Arms of RIAM - At a Glance
• RIAM:Overview of the Method
• RIAM:VLA:FAMILIARISATION, SCOPE & PLANNING
• RIAM:VLA:The Four Phases of the RALSBA