Where next with assurance: a review of ICAEW’s consultation paper

Your reading for the weekend is Where next with assurance?, a consultation paper from ICAEW which aims to take their series on assurance, “The Journey” to the next stage. It draws on feedback from ICAEW members who are developing assurance engagements in practice, and brings together strands from the corporate reporting and assurance debate around the world.

I have mixed feelings about this paper, so I hope that many of you will read it and respond to the consultation questions ask.  It is a valuable contribution to the assurance debate, but I can’t give my unqualified assent to the five key views expressed.

Addressing each of those “We think that…” statements in turn:

“Rather than focusing on the annual report – or any other single report of an organisation – we should think about the right way to use assurance to meet the needs of the organisation itself”

I don’t believe that the needs of the organisation which is the subject of the assurance are the only, or principal driver of the need for assurance.  In my view, assurance – and the role of the chartered accountant – are relevant to the public interest, to better business and to a better society; the first question we should ask about assurance is whether it is meeting those needs.

Moreover, if we were trying only, or mainly, to address the needs of organisations themselves, many of the issues assurance providers face with determining whether engagements are appropriate for users’ needs, who should be permitted to use and rely on reports and what consequences this has for the assurance provider’s liability would disappear.   So to position assurance as focussed primarily on addressing the needs of an organisation, not its stakeholders, skirts around some of the biggest practical obstacles to the development of an assurance market in the UK.

“The role of the board in determining the need for assurance, internally and externally, is vital to understanding the future of assurance”

No-one could disagree with this statement.  But I’m concerned that this view risks polarising the relationship between executive directors and the other parties who might be interested in assurance.  ICAEW skirts close to painting a picture of a world where directors know best and the very valid concerns of other stakeholders, including the public and those who feel excluded from the debate on trust in business, are automatically accorded lower value.

I believe that one of the most valuable roles of a chartered accountant is to facilitate engagement, and therefore relationship, between executive directors and other stakeholders, so that we can achieve a consensus on where better, more useable, information – and more assurance – is needed.

The paper correctly observes that the modern assurance market is undeveloped “…apart from in a few specific and regulated areas.”  It fails, though, to explore the reasons why a thriving assurance market has developed only where assurance is required by regulation.  This question is of fundamental, structural important to the market and we will not get very far if we ignore it.

The challenge facing assurance providers is determining whether there is a market, and for what, in the absence of regulation.

“Getting the right assurance in the right place is essential. This means asking the right questions about risks and information flows, and in a complex organisation it means keeping track of the situation with an assurance map”

Again, I’m sure no-one could argue with this.  However I don’t agree that “…the first step is working out where there are risks associated with information flows.”   Recent corporate scandals have demonstrated that where boards do not start from the perspective of strategy (and the risks that threaten achievement of strategy), they do not accord an appropriate level of importance to strategic non-financial information flows, and therefore these don’t make it onto the assurance map to start with.

I’d say that VW would be one example of this – their business model had an inherent tension between the need to comply with environmental regulation and the preference of their customers for the high performance that can be achieved unconstrained by regulation.  Unless that strategic tension is acknowledged, the related information flows are unlikely to be regarded as high risk.  Tesco is another such example where the challenge for the Board was recognising the extent to which the company’s profitability depended on pushing compliance with regulation to the absolute limits.

Essentially, once a company has fallen into the trap of not making meaningful disclosure of the most strategic information, then any assurance map that starts by asking what risks relate to the information which is gathered and disclosed will already have some significant omissions.

“Assurance can be provided over risk disclosures or forward-looking information, even if the question asked is different from ‘is this true and fair?’”

The paper sets out the four characteristics of “useful” forward-looking information – it is: understandable, relevant, reliable, and comparable.   The paper then proposes that “An assurance provider can carry out an engagement to provide an opinion on whether information that cannot be assessed yet for accuracy has the four characteristics for usefulness.”  I would agree that those are characteristics of a good basis of preparation of forecast information, but in my view, any assurance opinion on that forecast information would s be expressed as “properly prepared” in accordance with the basis of preparation, rather than “very useful”.

Of course a good basis of preparation does result in useful information, but not necessarily for every user – indeed existing assurance practice, including case law relevant to the financial statement audit, recognises that the needs of a homogenous population of users may differ even from that of any individual user who is a member of that population.  If we imply that it is simple to determine what is “useful” to a potentially very large range of users, we again skirt round one of the most difficult issues in the development of an assurance market.

Needless to say, the answer to the question “what would be useful here?” may well be one of the unknown unknowns as I think we could argue it was in the financial crisis. Is it “useful” for any set of forecasts to anticipate the zombie apocalypse? I suspect all will argue that it is not, until afterwards, when they’ll ask where the auditors were.

“Assurance can add value to narrative information using current principles and techniques, and the skilled judgement of preparers and assurance providers”

The previous section of the paper concludes with the words: “An assurance engagement on these subjects might consider whether the information is useful, or whether the process has been implemented as described, rather than asking ‘is this true and fair?!”  This is, I think, a misleading question, since it suggests that the financial statement auditor is consciously considering whether the financial statements are “true” and “fair”, as if those were separable testable characteristics.  In reality, in my view, the phrase “true and fair” has passed into regulatory rhetoric, as having an understood meaning as a whole (compliance with GAAP) that cannot be analysed down to its constituent parts.

The relevant question when exploring the assurance market is how long it might take for non-financial assurance phraseology to pass into the equivalent assurance rhetoric.  I would argue that, in the context of ISAE 3402 “fairly presents”, “suitably designed” and “operating effectively” have crossed the rhetorical Rubicon.  But I think it may be a long time before “fair, balanced and understandable” has acquired a similar standing as a phrase, the meaning of which is understood and shared, without reference to its constituent parts.

In conclusion

I think it needs to be clearer up-front that this paper is intended to be a provocation, a thought piece, rather than a technical analysis.  I find the idea of a continuum of assurance that embraces internal and external assurance helpful but I think the paper could be clearer about the fact that its references to assurance address the role of the chartered accountant in not only providing independent external assurance, but also in developing innovative interfaces which allow the value of internal assurance to be unlocked for stakeholders.


Assurance: the completeness conundrum

In December 2015, ICAEW’s Audit and Assurance Faculty published the third “milestone” of its journey towards assuring the whole of the annual report, entitled The journey milestone 3: Assuring the appropriateness of business information. The paper sets out to answer the question: how can an assurance provider be confident that disclosures made by a business give a fair picture of what is going on in that business with the greater part of the paper given over to the question of how the assurance provider should address the question of completeness.

In practice, assurance providers don’t get asked for opinions over completeness alone.  Preparers and users of information are most often looking for an opinion over the general “rightness” of some disclosed information and that “rightness” is frequently expressed in terms of “proper preparation” or “fair presentation” in accordance with a disclosed basis,

The first challenge, then, for the assurance provider is to decide to what extent an assertion from management and an opinion from an assurance provider couched in the terms “is properly prepared” or “fairly presents” inherently includes an assertion from management and an opinion from the assurance provider that the disclosed information is complete.

This is further complicated when the information disclosed is not intended to be 100% of the set of data, but a selection of data above or below a particular threshold, or with particular characteristics, for example, “the most significant items”. This would be the case for key performance indicators (KPIs).

In my view, for any data set which management present as a sub-set of a finite universe of data – e.g. top 5 contracts by revenue in the period, broadcasts with more than 1 million viewers – the assertion that it is properly prepared inherently implies that it is complete.  Certainly, in the case of finite populations, management can demonstrably begin with the complete universe of, say, contracts or broadcasts, and then select the complete set of the five with highest revenue or the ones that had more than a threshold number of viewers, respectively.

For such a dataset to be properly prepared must, surely, mean that it is the complete sub-set.  To provide assurance over its proper preparation, without testing completeness – even if disclosing as a limitation in the scope of work that no testing was carried out on completeness – doesn’t seem to be an option.

In the case of infinite populations, such as KPIs, it’s impossible for management to demonstrate that they selected from the full set.  Instead, they are demonstrating that they applied a fair process to the setting of boundaries to create a finite subset.

For some assurance engagements over KPIs, completeness is intentionally taken out of the question.  An example would be assurance over NHS quality accounts, where the indicators within the scope of the assurance are mandated by an external body.  Management are not asserting that those indicators are a complete sub-set meeting a definition (e.g. “key”); the assurance provider’s opinion on proper preparation cannot be interpreted as an opinion on the completeness of the indicators selected by the external body.  Indeed, the user of the assurance is left to form their own view as to whether the right indicators have been brought into scope.

It would not, however, be possible for management themselves to select a sub-set of indicators to disclose and ask the assurance provider for an opinion on proper preparation, excluding from scope the question of completeness.  Without any explicit or implied assertion from management that boundaries have been set so as to mark out a complete sub-set of some kind, the data is being presented as, apparently, selected at random.  In that scenario, what could be the rational purpose for its disclosure or for assurance?

The paper doesn’t address these initial acceptance questions, though in practice, in my view the question “is it possible to form an objective opinion as to whether this set of information is, in a meaningful sense, complete” is critical to the decision to accept an assurance engagement over non-financial information.

In the guidance that is provided, the paper rightly goes beyond the question of completeness into relevance. It notes that, in the case of KPIs, first management and subsequently the assurance provider have to consider whether the set chosen is sufficient (i.e. complete) and necessary (i.e. all the selections meet the definition of “key” performance indicators.)

I’m reminded of the FRC’s Guidance on the Strategic report which states “The number of items disclosed as a result of the requirements to disclose principal risks or KPIs will generally be relatively small; they should not, for example, result in a comprehensive list of all performance measures used within the business or of all risks and uncertainties that may affect the entity.”  Clearly the FRC thinks that, in the context of the strategic report, over-disclosure is as much of an issue as incompleteness.

The greater part of the paper focusses on how the assurance provider gathers evidence over completeness.  For readers like me, with a background in financial statement audit, it might have been helpful to break this section down more clearly in two parts corresponding to controls testing and substantive testing.

The paper recommends mapping out what management are trying to achieve by way of completeness (referred to in the paper as “control objectives”) to their process for achieving it.  The assurance provider considers whether each stage of the process will achieve what is intended (design), and whether there is evidence that the process was adhered to (operation).

Finally, you look at the actual output – the sub-set selected.  Does it make sense?  In the case of KPIs for example, we could create an expectation for the extent to which we’d expect this set of data to be comparable with that of the organisation’s peers, or other datasets prepared by the same organisation for different users, and then test that by actual comparison.  We could also test the extent to which the data is, in practice, used in decision-making by management and Those Charged With Governance.

The paper doesn’t address how the assurance provider will deal with problematic findings.  What if management’s process for achieving completeness – boundary-setting – is not adequate to start with?  Or hasn’t been adhered to in full in the period?  Should the assurance provider decline the engagement, or is it possible to do sufficient testing of the actual output to compensate for the weakness in design and/or operation of the process?

On the other hand, if the process is strong, is testing the output of any additional value? In practice, if management have evidently engaged appropriately with internal and external stakeholders, is the assurance provider ever in a position to argue that the resulting data selected is not complete and/or relevant?

I think a good next step would be to collect examples of how these issues are resolved in practice.  In the meantime, this paper will be useful for financial statement auditors who are grappling with the challenges of non-financial assurance, and finding the issue of completeness, outwith a system of double-entry book-keeping, thinly addressed in professional standards and guidance.

It will also help preparers to assess how comfortable they are with completeness in their data collection and reporting processes for non-financial data and their assertions over it.  And finally, it might lead everyone interested in company reporting to reflect on the extent to which “fair, balanced and understandable” inherently implies “complete”.

%d bloggers like this: