Sampling and Data-Driven Audit Techniques

Learning objectives

By the end of this chapter you will be able to:

• Explain audit sampling and identify when it is suitable and when it is not.
• Select an appropriate sampling approach based on the audit objective and the nature of the population.
• Determine practical sample sizes using key drivers such as risk, tolerable deviation/misstatement, expected deviation/misstatement, and the level of confidence required.
• Evaluate sample results, allow for sampling risk, and conclude appropriately on the population.
• Explain how data-driven audit techniques complement sampling and when full-population testing of specific attributes may be possible.
• Design effective follow-up procedures for exceptions identified through analytics and link outcomes to audit responses.

Overview & key concepts

Auditors rarely test every transaction or balance in full. Instead, they gather sufficient, appropriate evidence by focusing work where the risk of error is higher and by using methods that make testing efficient. Sampling supports this by allowing the auditor to test a subset of items drawn from a defined population and use the results to support a conclusion about that population. Data-driven techniques (often referred to as audit analytics) extend this approach by scanning or analysing larger datasets to identify patterns, outliers, and exceptions that deserve targeted attention.

Sampling and analytics are not simple substitutes. Analytics can improve planning and direct work towards higher-risk items, while sampling provides a disciplined way to select items, test them, and evaluate results. Even where analytics can test an entire dataset for a tightly defined rule, professional judgement, data validation, and corroborative procedures remain essential.

Audit sampling

What audit sampling is

In many audits, examining every item would add cost without producing proportionate assurance. Instead, the auditor selects a subset of items from a clearly defined population and applies the planned audit tests to those items. The aim is to obtain evidence that supports a conclusion about the population as a whole—while recognising that conclusions drawn from a subset carry uncertainty, which must be managed through careful selection and evaluation.

Sampling is typically used to:

• assess whether controls operated consistently (tests of controls), or
• estimate misstatement in a balance or class of transactions (substantive testing).

Sampling is not suitable where the audit objective requires testing every item, such as when:

• the population is very small and full testing is feasible,
• a small number of items are individually significant and must be examined in full, or
• completeness is the primary concern and the population listing itself may be incomplete or unreliable.

When sampling is appropriate—and when it is not

Sampling is usually appropriate when:

• the population is large and contains many similar items,
• the population can be defined and obtained reliably,
• the auditor expects the sample to be capable of supporting a conclusion about the population, and
• full-population testing would be inefficient without materially improving the conclusion.

Sampling may be inappropriate when:

• the population is small enough to test fully at reasonable cost,
• the risk is concentrated in a small number of items (suggesting 100% testing of those items),
• the population is incomplete or unreliable (sampling from a flawed list produces weak evidence), or
• the audit objective cannot be met by partial testing (for example, where each item is critical).

Population and sampling unit

Population

The population is the complete set of items from which the sample is selected. It must match the audit objective precisely. A well-designed sample does not help if it is drawn from the wrong population.

Examples of populations:

• all purchase orders raised during the year above a stated threshold,
• all sales invoices recorded in a particular month,
• all manual journal entries posted after period-end close.

Sampling unit

The sampling unit is the individual item selected for testing (for example, one purchase order, one invoice, one journal entry). Each sampling unit must be identifiable, available for inspection, and appropriate for the test being performed.

A frequent practical issue is ensuring that the sampling unit aligns with the control or assertion under test. For example, if testing a purchase approval control, the sampling unit would normally be a purchase order (or requisition) that should contain evidence of approval.

Sampling risk and non-sampling risk

Sampling risk

Sampling risk is the possibility that the sample results do not reflect what would be found if every item in the population were tested. The risk cuts both ways:

• a sample can givefalse comfort, suggesting a control is operating well (or a balance is reasonable) when the wider population is not, or
• a sample can trigger afalse alarm, suggesting a widespread issue when the population is actually acceptable.

The auditor reduces sampling risk by choosing a suitable method, using a sample size consistent with the level of assurance needed, and interpreting exceptions in context rather than in isolation.

Non-sampling risk

Non-sampling risk arises from causes unrelated to the representativeness or size of the sample. Examples include:

• using an inappropriate procedure,
• misunderstanding how a control operates,
• misinterpreting evidence (for example, accepting an approval that is not from an authorised person),
• failing to investigate exceptions properly.

Non-sampling risk is reduced through good planning, supervision, training, professional scepticism, and review.

Tolerable and expected deviation or misstatement

Tolerable deviation rate (controls testing)

When testing controls, auditors consider a tolerable deviation rate—the maximum rate of control failure that could exist in the population while still allowing reliance on that control for audit purposes.

Expected deviation rate (controls testing)

The expected deviation rate is the rate of failure anticipated before testing, based on prior results, process changes, or other knowledge. Higher expected deviation generally leads to larger sample sizes and may reduce the value of controls reliance.

Tolerable misstatement and expected misstatement (substantive testing)

For substantive sampling, the focus is on monetary error:

• Tolerable misstatement: the maximum monetary error in the population that can be accepted without changing the planned audit response.
• Expected misstatement: the likely monetary error in the population anticipated before testing.

Stratification and selection methods

Stratification

Stratification means splitting a population into subgroups that share a common feature (often value or risk). This improves efficiency and audit focus. A typical approach is:

• test all items above a set value threshold (100% testing of the high-value stratum), and
• apply sampling to the remaining items.

Stratification is especially useful where risk is not evenly spread across the population.

Selection methods

Common selection methods include:

• Random selection: each sampling unit has a known, non-zero chance of selection. This supports defensible conclusions and helps reduce selection bias.
• Systematic selection: selecting everynth item after a random start. This is efficient, but the ordering of the population list must not create patterns that distort selection.
• Haphazard selection: selecting items without a structured technique while attempting to avoid bias.

Important caution on haphazard selection:
Haphazard selection does not allow a measurable level of sampling risk and is more vulnerable to unconscious bias (for example, avoiding complex items or selecting “easy” documents). It may be acceptable for limited, low-risk work where the conclusion does not depend on extrapolating to the population. Where conclusions about the population matter, random or systematic selection is generally preferable because it is more defensible and better controlled.

Data-driven techniques in auditing

Data-driven techniques use data extracts, queries, and analytical routines to examine datasets for features relevant to audit objectives. They can support planning, risk assessment, controls testing, and substantive procedures.

Common uses include:

• identifying duplicate payments,
• spotting transactions just below authorisation thresholds,
• highlighting unusual posting times (e.g. weekends, late-night entries),
• analysing trends by supplier, department, or product line,
• isolating manual entries with higher-risk characteristics.

Tight boundaries on “replacing” sampling

Analytics can sometimes enable full-population testing of specific attributes under a clearly defined rule (for example, identifying all payments that match a duplicate-payment rule). However, the auditor still needs to:

• validate the completeness and accuracy of the dataset,
• confirm that the rule logic is appropriate and does what it claims to do, and
• investigate flagged items using corroborative evidence (documents, authorisations, bank records, and explanations).

Analytics rarely removes the need for professional judgement or further testing. It most often improves the audit by increasing coverage, directing attention to higher-risk areas, and providing stronger insight for designing and evaluating other procedures.

Core theory and frameworks

Planning a sampling test

A practical planning framework is to link the test to the claim being made, the data available, the selection approach, and the evaluation and response.

1. State the audit purpose clearly
   • Example: “Assess whether purchase orders above £5,000 show evidence of approval by an authorised manager.”
2. Define and obtain the population
   • Confirm the period covered, inclusion criteria, and completeness of the listing used for selection.
3. Define the sampling unit
   • Ensure the unit selected contains the evidence needed for the test (e.g. the purchase order record with approval evidence).
4. Decide whether sampling is suitable
   • Consider whether full testing is feasible, whether risk is concentrated, and whether the listing is reliable.
5. Set tolerable and expected deviation/misstatement
   • These thresholds drive the planned work and how results will be interpreted.
6. Determine the sample size using practical drivers
7. Sample size tends to increase when:
   • assessed risk is higher,
   • tolerable deviation/misstatement is lower,
   • expected deviation/misstatement is higher,
   • greater reliance is planned on the control (or tighter assurance is needed), and
   • ahigher level of confidence / lower sampling riskis required.
8. Select an appropriate selection method
   • Choose a method that supports the planned conclusion and can be documented clearly.

Evaluating sample results

For tests of controls (deviation-based)

Evaluation involves more than comparing an observed rate to a tolerable rate. The auditor must also consider sampling risk—i.e. whether, allowing for the uncertainty that comes from testing only a subset, the population deviation rate could be above tolerable.

A practical approach is:

• calculate thesample deviation rate,
• consider thepattern and natureof deviations (isolated vs systematic; linked to a time period, site, user, or transaction type),
• allow for anallowance for sampling risk(plain-language concept: the population failure rate may be higher than what the sample shows), and
• decide whether it remains reasonable to rely on the control, or whether the audit response should be changed.

In statistical sampling, this allowance is often expressed through an “upper” estimate of the population deviation rate at a chosen confidence level. In non-statistical sampling, the auditor does not calculate an upper rate but should still make a cautious judgement that reflects the possibility that the true population rate is higher than observed.

If deviations are identified, the auditor should also consider:

• whether the deviation indicates non-operation of the control or missing evidence,
• whether the control is designed so that evidence should always be retained (if so, missing evidence is treated as a deviation),
• whether additional testing is needed (for example, expanding the sample or testing a focused period).

For substantive tests (monetary misstatement)

Evaluation typically involves:

• measuring misstatements found in the sample,
• projecting misstatement to the population where appropriate,
• comparing estimated population misstatement to tolerable misstatement (with a sensible allowance for sampling risk), and
• determining whether further work or adjustments are required.

Data-driven testing approach

A structured approach to data-driven testing is:

1. Clarify the audit question
   • Define the risk and what constitutes an exception.
2. Obtain and understand the data
   • Identify key fields and how they are generated.
3. Validate the dataset
   • Reconcile totals to the ledger, confirm the time period, and check for completeness/duplication issues.
4. Run targeted routines
   • Use rules that link directly to the audit objective.
5. Investigate exceptions
   • Determine whether the exception is a true issue, a timing/processing difference, or a false positive.
6. Conclude and link to the audit response
   • Decide how results affect risk assessment, control reliance, and further procedures.

Worked example

Narrative scenario

ABC Ltd is a manufacturing company with annual revenue of £900,000. The audit team plans to test the operating effectiveness of a control requiring manager approval for purchase orders above £5,000.

• Population: all purchase orders above £5,000 during the year
• Population size: 1,200 purchase orders
• Sample selected: 60 purchase orders
• Deviations found: 3 purchase orders with missing evidence of approval
• Tolerable deviation rate: 2%
• Expected deviation rate: 1%

ABC Ltd also uses data-driven techniques within the purchase-to-pay cycle. The analytics routines flag:

• 12 duplicate payment candidates, and
• 7 payments processed at weekends.

Required

1. Calculate the sample deviation rate and compare it to the tolerable deviation rate.
2. Evaluate the impact of the deviations on audit conclusions, allowing for sampling risk and deviation patterns.
3. Describe follow-up actions for the analytics exceptions.
4. Explain how sampling and analytics can be integrated in the audit process.

Solution

1) Sample deviation rate and comparison

Sample deviation rate = deviations ÷ sample size
= 3 ÷ 60
= 0.05 = 5%

Comparison to tolerable deviation rate:

• Sample deviation rate:5%
• Tolerable deviation rate:2%

The observed deviation rate is above the tolerable level.

2) Impact on audit conclusions (including sampling risk and deviation pattern)

Initial implication:
A 5% observed deviation rate indicates that the control is not operating consistently in the sample. Since the observed rate already exceeds tolerable, this strongly suggests the control is not reliable enough for the planned level of reliance.

Allowance for sampling risk (plain language):
Because only 60 items were tested, the true deviation rate in the full population could be higher or lower than 5%. The auditor should not assume the population rate equals the sample rate. A cautious conclusion is required, particularly when tolerable deviation is low (2%).

Nature and pattern of deviations:
Before finalising the response, the auditor should consider:

• Root cause:Are approvals genuinely not happening, or is approval occurring but evidence is not being retained? If the control requires evidence to be retained, missing evidence is treated as a control failure either way.
• Clustering:Do the three deviations relate to the same month, location, approver, or supplier category? Clustering may indicate a specific breakdown that can be targeted.
• Control design link:If approvals are missing, what downstream risks arise (unauthorised purchases, incorrect supplier selection, price manipulation, or fraudulent payments)?

Conclusion for audit planning:
On the basis of the sample results, reliance on this control should be reduced or removed for the affected population. The audit response would normally include:

• increasing substantive procedures over purchases, payables, and supplier payments,
• considering whether other related controls compensate (for example, three-way match controls),
• performing additional focused testing if clustering suggests a specific period or area of weakness (e.g. expand testing in the suspect month or site), and
• considering whether the results indicate a broader control environment issue requiring a wider response.

3) Follow-up actions for analytics exceptions

A. Duplicate payment candidates (12 items)
Follow-up aims to confirm whether each is a true duplicate, an allowable repeat payment, or a false positive:

• agree key fields (supplier, invoice reference, date, amount, bank details) to source documents,
• inspect invoices and credit notes to determine whether the “duplicate” is a correction or reversal,
• review payment run reports and authorisation for the payment batches,
• trace payments to bank statements to confirm number and timing of payments,
• check whether duplicates arise from vendor master file issues (e.g. duplicate supplier records),
• where true duplicates are confirmed, assess recovery actions and consider the control implications.

B. Weekend payments (7 items)
Weekend processing can be legitimate or higher risk. Procedures may include:

• determine whether payments were automated or manually initiated,
• inspect who initiated and approved the payments and whether approval complied with policy,
• review supporting documentation and business rationale for weekend processing,
• assess whether weekend items correlate with other risk indicators (unusual suppliers, last-minute bank detail changes, manual overrides),
• expand testing if items cluster around specific users, suppliers, or periods.

4) Integration of sampling and analytics

An integrated approach can work as follows:

• Useanalytics in planningto identify higher-risk features (duplicates, threshold behaviour, unusual timing).
• Performtargeted testingof flagged items to understand whether issues exist and why.
• Usesamplingto support a conclusion on broader control operation or population characteristics.
• Refine each approach using outcomes from the other:
  • if sampling suggests weakness, analytics can be used to identify where the weakness is concentrated,
  • if analytics shows concentrated risk (e.g. a month or user), the population can be stratified and sampling focused on that stratum.

Interpretation of the results

The observed deviation rate of 5% suggests the approval control is not operating consistently enough for reliance at a tolerable deviation rate of 2%. The auditor should adjust the audit response by reducing reliance on the control and increasing substantive work, while also investigating the root cause of failures and whether deviations cluster around specific conditions. Analytics flags provide targeted starting points for follow-up but require corroboration before any conclusion is drawn. Used together, sampling and analytics provide broader visibility and more focused testing, improving the quality of audit evidence and the precision of the audit response.

Common pitfalls and misunderstandings

• Treating control evaluation as purely mechanical:observed deviation rates must be interpreted with an allowance for sampling risk and an understanding of deviation patterns.
• Confusing sampling risk with non-sampling risk:sampling risk is about representativeness; non-sampling risk is about performing or interpreting work incorrectly.
• Defining the wrong population:the sample cannot support the objective if the population does not match what is being tested.
• Using a biased listing for systematic selection:ordering by value, supplier, or approver can distort results.
• Over-relying on haphazard selection:higher risk of unconscious bias and no measurable sampling risk; weaker defensibility where population conclusions are required.
• Skipping data validation in analytics:results are unreliable if the dataset is incomplete, duplicated, or not reconciled to the ledger.
• Assuming exceptions are errors:analytics flags indicate risk, not proof; each exception needs investigation.
• Weak linkage to the audit response:findings must change the planned nature, timing, or extent of further procedures where necessary.
• Poor documentation:unclear rationale, selection method, and evaluation undermines the strength of the conclusion.

Summary and further reading

Sampling supports efficient audit evidence by enabling testing of a subset of items drawn from a defined population. Effective sampling depends on clear objectives, correct population definition, appropriate selection, and careful evaluation that allows for sampling risk. Data-driven techniques strengthen audit work by scanning datasets to reveal trends and exceptions, improving planning and directing attention to higher-risk items. Even where analytics can test an entire dataset for a defined rule, the auditor must validate the data, confirm logic, and investigate exceptions using corroborative evidence.

For further reading, refer to professional auditing guidance on audit evidence and sampling, and practitioner resources on audit analytics, data validation, and exception follow-up in transaction cycles.

FAQ

What is the difference between sampling risk and non-sampling risk?

Sampling risk is the possibility that the sample does not reflect the population and therefore leads to a different conclusion than full testing would. Non-sampling risk arises from issues such as using the wrong procedure, misunderstanding the control, or misreading evidence. Sampling risk is reduced through suitable selection and sample sizes aligned to the confidence required; non-sampling risk is reduced through good planning, supervision, training, scepticism, and review.

How do you determine an appropriate sample size for an audit test?

Sample size is driven by assessed risk, tolerable deviation/misstatement, expected deviation/misstatement, and the level of confidence required (lower acceptable sampling risk generally means a larger sample). Practical constraints matter, but they do not override the need for evidence strong enough to support the conclusion.

When can data-driven techniques be used instead of traditional sampling?

Data-driven techniques can sometimes test an entire dataset for a narrowly defined attribute under a clear rule (for example, a duplicate-payment rule). However, the auditor must validate the dataset, confirm rule logic, and investigate results with corroborative evidence. In most cases, analytics complements sampling and other procedures rather than replacing them.

What are common pitfalls in systematic sampling?

The main risk is unintended bias from how the population list is ordered. Selecting every nth item can over- or under-represent certain transaction types if the ordering aligns with value, supplier, approver, or processing batches. A random start helps, but ordering must still be assessed.

How does stratification improve sampling?

Stratification improves efficiency by separating higher-risk or higher-value items from the rest. Testing all items in a high-risk stratum and sampling the remainder provides stronger evidence where it matters most while reducing unnecessary work on low-risk items.

Summary (Recap)

This chapter explains how sampling supports audit evidence by enabling disciplined testing of a subset of a defined population. It shows how to define populations and sampling units, distinguish sampling risk from non-sampling risk, set tolerable and expected deviation/misstatement, and evaluate results with an allowance for sampling risk. It also explains how data-driven techniques can scan datasets for trends and exceptions, how to validate data and rule logic, and how to investigate exceptions using corroborative evidence. A worked example demonstrates calculating a deviation rate, drawing a cautious conclusion on control reliance, and designing follow-up procedures for analytics flags.

Glossary

Audit sampling
Selecting a subset of items from a defined population and testing them to support a conclusion about the population as a whole.

Population
The complete set of items relevant to an audit objective from which a sample is selected.

Sampling unit
The individual item that can be selected for testing (for example, one purchase order, one invoice, one journal entry).

Sampling risk
The possibility that the sample results do not reflect the population, leading to a different conclusion than full testing would produce.

Non-sampling risk
The risk of an incorrect conclusion for reasons unrelated to sampling, such as applying an inappropriate procedure or misinterpreting evidence.

Tolerable deviation rate
The maximum rate of control failures that could exist in the population while still allowing reliance on that control.

Expected deviation rate
The control failure rate anticipated before testing, based on prior results and knowledge of the process.

Tolerable misstatement
The maximum monetary error in a population that can be accepted without changing the planned audit response.

Expected misstatement
The monetary error anticipated before testing based on prior experience and risk assessment.

Stratification
Dividing a population into subgroups (often by value or risk) to improve the efficiency and focus of testing.

Random selection
A selection method where each sampling unit has a known, non-zero chance of being chosen.

Systematic selection
Selecting items at a fixed interval after a random start, requiring careful consideration of how the population list is ordered.

Haphazard selection
Selecting items without a structured method while trying to avoid bias; less defensible for population-wide conclusions due to higher bias risk and no measurable sampling risk.

Data-driven audit techniques (analytics)
Using data extracts and analytical routines to scan datasets for patterns, outliers, and exceptions relevant to audit objectives.

Exception
An item highlighted by an analytical routine as unusual or higher risk, requiring investigation and corroboration before conclusions are drawn.