View source for RIAM:VLA:AUDIT SAMPLING AND AUDIT TESTING

(TO BE FORMATTED)

=PHASE 3: AUDIT SAMPLING AND AUDIT TESTING=

==Introduction==

	The testing phase follows the evaluation phase where the focus was on the control framework with the aim of gauging some assurance about the system operation and achievement of objectives by analysing risk.  But with documentation providing only limited verification of its operation, the conclusions provided to this point are not final, hence the need for further testing - to supply the auditor with the necessary proof to support the audit opinion.

The approach taken to designing the tests will depend on the nature of the audit problem.  There is, however, one guiding principle which needs to be borne in mind when framing the tests.  Does the direction of the test achieve the desired audit objectives?

The objectives of each verification test must be clearly defined prior to developing a comprehensive audit program.  These objectives and the nature of the policy, system or activity under review will determine the verification techniques required. A wide range of techniques can be selected for developing the audit program, including:

¨	transaction testing
¨	personal observation and enquiry
¨	report and data analysis
¨	independent or third party confirmation or interviews
¨	comparison and analysis of costs or data for similar activities

It is expected that auditors will, where circumstances warrant it, utilise statistical sampling techniques in the testing phase of their audits.


==Why sample ? - When to and when not to sample==

	Sampling is the application of audit procedures to less than 100% of the population.
	This involves selecting a proportion of the population, using characteristics of that portion to draw inferences about the entire population.
	There are two basic approaches -
			Statistical 
			Judgemental ( or Non-Statistical )

	Consider the objectives that are trying to be achieved by using sampling
			Compliance testing
			Substantive testing

	Factors in deciding when to use or not to use sampling include -
	objectives
			data or information involved
			access to computer facilities
			cost/benefit
		other methods of testing to achieve the desired results


==Types of samples, their features, use and benefits==


	Attributes Sampling

	used for compliance testing

	estimates the rate of deviations in the population

	gives statistical result that can draw inferences to the population


	Variables Sampling

	used for substantive testing

	estimates the $ amount of error in a population

	easier to use


	Mean per Unit Sampling

	Probability-proportional-to-size

	Difference estimation sampling

	Stop - go Sampling



 

==Key Terms in Sampling==


Sampling - the process of applying audit procedures to less than 100% of the population

Population - is the data set from which the sample is taken to assist in reaching a conclusion. The individual items that make up a population are known as Sampling Units

Sampling Risk - is the risk that the auditors conclusion based on a sample is different to that if the whole population was used

Non Sampling Risk - is the risk not specifically caused by sampling, ie the risk that incorrect procedures are being used

Confidence level or reliability -  The percentage of times that one would expect the sample to adequately represent the population.  Thus, a confidence level of 90% should result in samples that adequately represent the population 90% of the time.  Confidence level is related to audit risk because the auditor is accepting a risk of 10% (100%-90%) that the sample will not represent the population.  In sampling for variables (substantive testing), the primary concern is the risk of incorrect acceptance or the risk that the sample supports the conclusion that the assertion tested is not materially misstated when it actually is materially misstated.  In sampling for attributes (tests of controls), the primary concern is the risk of assessing control risk too low, or the risk that the assessed level of control risk based on the sample is less than the true operating effectiveness of the internal control structure policy to procedure.  These wrong conclusions are also called type II or beta errors.

Precision of confidence interval - An interval around the sample statistic (for example, the mean) within which one expects the true value of the population to fall.  Precision is based upon tolerable misstatement determined by materiality considerations.  In sampling for attributes (tests of controls), precision is determined by subtracting the expected deviation rate from the tolerable rate.  In sampling for variables (substantive testing), precision is determined by considering tolerable misstatement in conjunction with the confidence level (an effectiveness issue) as well as the risk of incorrect rejection.  The risk of incorrect rejection is the risk that the sample indicates that the assertion tested is materially misstated when in fact it is not misstated (termed a type I or alpha error).  It relates to efficiency issues because the auditor will likely continue auditing until the balance is finally supported.  A table is typically consulted to determine the appropriate precision for various risk levels.  A rule of thumb is often used is to set precision at 50% of tolerable misstatement.

Alpha (Type I) error is the rejection of a correct hypothesis.  The risk is incorrect rejection of an assertion and the risk of assessing control risk too high both relate to alpha error.  These risks are aspects of sampling risk that involve efficiency issues.

Beta (Type II) error is the failure to reject an incorrect hypothesis.  The risk of incorrect acceptance of an assertion and the risk of assessing control risk too low both relate to beta error.  These risks are aspects of sampling risk that involve effectiveness issues.

Sampling without replacement means not returning a sample item to the population to prevent its being selected more than once.  Audit sampling is customarily done without replacement.
Sampling with replacement means returning a sample item to the population so that it has a chance to be chosen more than once.



Standard deviation is a measure of the degree of compactness of the values in a population.  This measure is used by the auditor to help determine appropriate sample sizes.  The first formula given below is for the population standard deviation (ó).  It is the square root of the quotient of the sum of the squared deviations from the mean (µ), divided by the number of items in the population (N).  The sample standard deviation is found using the second formula given below.  The sample standard deviation is s, the mean of the sample is s, the mean of the sample is x, and the sample size is n.


sigma ~=~ SQRT {{ SIGMA ( CHI SUB i~-~ mu ) SUP 2}over{N}}





s ~=~ SQRT {{ SIGMA ( CHI SUB i~-~ x ) SUP 2}over{n~-~1}}






Hypothesis testing involves a predetermined rule for evaluating the auditee's assertion.  Strictly speaking, the auditor either rejects the hypothesis or is unable to do so.  In auditing literature, however, a hypothesis that cannot be rejected is often said to be "accepted".  This usage is followed even though a sample is never a sufficient basis for concluding that the hypothesis is in fact true.  In hypothesis testing, the auditor determines the acceptable risk of incorrect rejection and the acceptable range (the nonrejection region) about the auditee's value.  The auditor "accepts" (is unable to reject) this value if the sample value is within the auditor's prespecified materiality limits about the auditee's assertion.  The auditor rejects the auditee's figure if the sample precision interval is sufficiently outside the precision limits.  The sample tests the null hypothesis that there is a null or zero difference between the true value and the assertion.  The null hypothesis will consist of an equality (Ho = a given value) if a two-tailed test is involved (relatively large or small values will be rejected).  If a one-tailed test is involved (relatively large or small values will be rejected).  If a one-tailed test is involved (extreme values on one side can be ignored), the null hypothesis will be an inequality (Ho  or  a given value).

Judgment (nonstatistical) sampling uses the auditor's subjective judgment to determine the sample size (number of items examined) and sample selection (which items to examine).  This subjectivity is not always a weakness.  The auditor, based on other audit work, may be able to test the most material items and to emphasise the types subject to high control risk. The auditor's working relationship with managers means that in many audits (particularly those of a non-financial nature) small scale judgement sampling may be sufficient. A judgementally selected sample can not be projected onto the full 'audit' population from which the sample is drawn, but may be sufficient when the purpose of the testing does not require mathematically rigorous proofs, extrapolation to the full population, accurate estimates of error rates or where management has waived the need for scientific testing.
Probability (random) sampling provides an objective method of determining sample size and selecting the items to be examined.  Unlike judgment sampling, it also provides a means of quantitatively assessing precision (how closely the sample represents the population) and reliability (confidence level, the percentage of times the sample will reflect the population).

Tolerable rate is the maximum rate of deviations from a prescribed internal control structure policy or procedure that the auditor is will to accept without changing his/her assessment of control risk for the assertions related to the policy or procedure.

==Judgmental and Statistical Sampling==

	Both types of sampling have the in common the reliance on judgement for planning, executing the plan and evaluating the results
	Both methods are subject to sampling and non-sampling risk
	The critical difference between the two, is that the Laws of Probability are used to control sampling risk in statistical sampling


 

==Judgmental V. Statistical Sampling==

	STATISTICAL SAMPLING

ADVANTAGES	DISADVANTAGES
	allows sample size to be set at a minimum	 requires the use of a random sample, which may be costly and time consuming
	objective method of determining sample size, sample risk and evaluating the sample	 may require additional training of staff
	allows more control over sample risk	 many audit areas are not extensive enough to warrant its full use
	easy to use is access to computers is available	
	allows for specific levels of reliability (confidence) and degree of precision (materiality)	

	JUDGEMENTAL SAMPLING

ADVANTAGES	DISADVANTAGES
	allows for the auditor to sue his judgement, ie. high risk items	 cannot quantitatively measure sampling risk
	may be more cost effective than statistical sampling	 cannot draw statistical inferences from the sample results
	 presents the risk of either under-auditing or over-auditing
	 may be inappropriate for inexperienced staff

 

==Calculating Sample Sizes==

===Dollar Unit Sampling ( DUS )  Assumes No Errors in Pop Sample===
 
	This may be known as probability-proportional to size (PPS), cumulative monetary amount (CMA) and many others

	DUS relies on an attribute sampling approach to express deviations in dollar amounts rather than as a deviation rate

	The formula for determining a sample size in DUS is -

Equation 1
n~=~{BV~*~RF} OVER {TE}






Equation 2
n~=~{RF} OVER {[TE/BV]}







Equation 3
I~=~{BV} OVER {n}







n= Sample Size
BV = book value of items tested
RF = reliability factor
TE = tolerable error
I= Skip Internal

DUS Reliability Factors
Reliability
Required	RF
99%	4.605
95%	2.996
90%	2.300

 
	EXAMPLE:

Assume:
BV =	5,000,000
N =	2,000
TE =	$250,000
CL =	95% (confidence level  RF = 2.996)
RF =	2.996
(A)
n~=~{BV~*~RF} OVER {TE}
{$5M~*~3.0} OVER {$250,000}~(rounding)


=


= 60

(B)
Skip~Internal[[User:Bishopj|Bishopj]]I~=~{BV} OVER {n}~=~{$5M} OVER {60}~=~$83,333







(C)	If no errors in Sample, Auditor is 95% confident that Overstatement does not exceed $250,000.


Assume:	
Z =	1.96 (95% confident)
N =	1000 (population size)
A =	± 2% (Prec)
P =	5% (est. error rate)

(1)
n SUB{( e)}~=~{1.96 SUP 2~*~.05~*~(1-.05)} OVER {.022}





= 456


(2)
n SUB {(f)}~=~{n SUB {(e)}} OVER {1~+~(n SUP {(e)}/n)}~=~{456} OVER {1~+~(456/1000)}~=~ UNDERLINE {313}






 

===Attribute Sampling===

	generally used for compliance testing 
	it is used to estimate the rate of deviations form prescribed control procedures in a population
	The formula has two parts -


The first formula is:

n SUB {( e )}~=~ {{ Z SUP 2(p)(1~-~ p)}over{A SUP 2}}






where

n(e) 	=  First estimate of sample size
Z	=  Standard deviation factor
p	=  Occurrence rate
A	=  Desired precision.
N	=  Population size


The second formula uses the first estimate of sample size and adjusts it to fit the population:


n SUB{ (f)} ~=~ {{ n SUB {(e)}}over{1~+~(n SUB {(e)}/N)}}






where

n(f)	=  Final sample size
n(e)	=  First estimate of sample size
N	=  Population. 


 

10.7.3	Variable Sampling



Equation 1: Calculate the Standard Deviation of the Test Sample
s ~=~ SQRT {{ SIGMA ( chi SUP 2)~-~ ( SIGMA chi ) SUP 2/n }over{n~-~1}}






Equation 2: Estimate the required sample,assuming infinite popoulation
n SUB {(e)} ~=~ ({Zs} OVER {A}) SUP 2






Equation 3: Adjust sample size for finite population
n SUB {(f)} ~=~ {n SUB {(e)}} OVER {1~+~(n SUB {(e)}/N)}






Equation 4: Calculate the sampling error (error in sample per unit sampled)
A ~=~ +-~ {[Z~{s} OVER SQRT n}~( SQRT {1~-~n OVER N})~]






Equation 5: Estimate the total population error
A SUP P ~=~ +-~(A~*~N) 






	Where
 

s	=  Standard deviation of the sample	N   =   Population Size
	=  Sum of					Z   =   Standard Dev. Factor
x	=  Value of each sample item		A   =   Precision (Sampling Error) in $
n	=  Sample size

	Generally this approach is used for substantive testing
	It is used to estimate the total $ of a population or the total $ amount of errors in a population

 
(1)	VS example - Sample Size

[Pop Size] N				=	5000
[Pop Value]				=	$500,000
[Designed Confidence] C		=	90%
 [Desired Std. Dev. fact] Z	=	1.645
Std. Dev. of Sample		=	$40  from 200 items selected at random
Desired Prec.				=	4%
 Desired Prec. of Pop.		=	$20,000
 Desired Prec./Unit A		=	± $4

Thus
n SUB {(e)}~=~({Z SUB {(s)}} OVER {A}) SUP 2~=~({1.645~*~40} OVER {4}) SUP 2~=~271

n SUB {(f)}~=~{n SUB {(e)}} OVER {1~+~(n SUB {(e)}/N)}~=~{271} OVER {(1~+~{271} OVER {5000})}~=~ UNDERLINE {257}

Sample size = 257  select another 57 items

(2)	VS example - Precision of Estimates

n(e)	=	257		N = 5,000
BV	=	$27,000	Z = 1.645 (90%)
AV	=	$23,130
S	=	$40

(2.1)	 Aug. Unit	=	AU	=	23,130/257	= $90
Est.Inv.Val.	=	$90 * 5000				= $450,000
(2.2)

THEREFORE ~A~=~Z~{s OVER SQRT n}~( SQRT {1~-~n OVER N})

=~1.645~{40 OVER SQRT {257}}~(~ SQRT {1~-~{257} OVER {5000}})

=~$ UNDERLINE 4~~( +- )~  ~unit~ precision


(2.3)	± 4 * 5000 = ± $20,000 = Pop. Precis.
(2.4)	 Est Inv. Val ranges from $430,000 to $470,000
 

 

===Discovery Sampling===

	This method is used when the auditor is examining a population for fraud or gross errors are expected

	This method involves setting two parameters

		critical rate of deviation ( max errors allowed / population )
		probability
	In this case it is necessary to use discovery sampling tables.



==Effect of risk analysis on the extent of substantive testing==


	Risk of incorrect rejection:
		is the risk that, the sample tells the auditor that the balance is materially misstated , when in fact it is not misstated

	The risk will effect the efficiency of the audit , as it generally results in more testing having to be performed.

	Risk of incorrect acceptance
		is the risk that, the sample tells the auditor the balance is not misstated, when in fact the balance is misstated

	This may have a serious effect on the audit as it may result in the wrong opinion being issued

	Both of the above risks have an inverse effect on sample sizes, that is, a lower risk level will result in a higher sample size



==Substantive Test Risk Matrix==

Evidence indicates relevant account balance should be:	Relevant Account balance is in fact:

	Fairly Stated	Not Fairly Stated
Accepted	Correct
Decision	Risk of incorrect Acceptance
Rejected	Risk of incorrect Rejection	Correct
Decision

 

==The effect of stratification on sample sizes==



	Is the process of dividing populations into sub-populations. Thus allowing the auditor to direct his efforts towards items considered to contain the greater monetary error.



	The principal advantage of stratified sampling is that it produces sub-populations that are individually more homogeneous, thus decreasing the sample size required to accomplish the audit objectives.



	This serves to reduce the variability of the sampling units within each stratum



==Interrelationships between concepts such as sample size, confidence, expected error rates and precision==




Attribute sampling:

	Increase in required confidence -> Increase in sample size
	Increase in expected error rate -> Increase in sample size
	Increase in precision -> Decrease in sample size.


Variable sampling:

	Increase in required confidence -> Increase in sample size
	Increase in standard deviation -> Increase in sample size
	Increase in precision -> Decrease in sample size.



==Methods of making selections==



	Random number sampling:

		offers every item in the population an equal chance of selection
		involves using random number tables or computer generated random numbers
		is facilitated when items in the population are consecutively numbered
		although the same number may be selected twice , practically this method uses no replacement sampling , therefore it may result in a larger sample. Thus it is considered to be a conservative approach.


	Interval sampling:

		simply means " selecting items at intervals "
		used when random number sampling is inappropriate
		simple to use
	
	Stratified sampling

		involves arranging the population to provide greater sampling efficiency
		the population will be separated into two or more strata
		samples are then taken from each strata
		stratification ,allows for smaller sample sizes and controls distortion

	Cluster sampling

		used when documents or records are dispersed or scattered and other methods are to time consuming or costly
		as its name suggests this method simply involves the selection of clusters instead of individual items.


==SUMMARY==

In dealing with audit sampling, the auditor should keep these ten commandments in mind:

1.	Know the principles of scientific sampling, but use them only when they best fit the audit objectives.

2.	Know the population, and base audit opinions only on the population sampled.

3.	Let every item in the population have an equal chance of being selected.

4.	Do not let personal bias affect the sample.

5.	Do not permit patterns in the population to affect the randomness of the sample.

6.	Do not draw conclusions about the entire population from the purposive or directed (judgement) sample, even though it does have its place.

7.	Base estimates of maximum error rates on what is reasonable in the real world; try to determine at what point alarms would automatically go off.

8.	Stratify wherever it would appear to reduce variability in the sample.

9.	Do not set needlessly high reliability goals (confidence level and precision).  Controls, supervision, feedback, self-correcting devices, and management awareness and surveillance should all be considered in seeking to reduce the extent of the audit tests.

10.	Do not stop with statistical results; know why the variances occurred.


==Backlinks==
* [[RIAM:VLA:The Four Phases of the RALSBA| Back To The Four Phases of RALSBA]]
* [[RIAM:Conduct of the Very Large Audit| Back To Conduct of the Very Large Audit (Main)]]
* [[Internal Audit Method| Back To The RIAM (Main)]]