The Audit Methodology: Steps of an Audit continued... - Yellowbook-CPE.com (2024)

Now we are getting down to the nitty gritty – the audit methodology! We have worked through the first 7 steps of performing an audit in recent articles: Steps 1-3, Steps 4-6 , and Step 7. Now let’s look at steps 8 & 9.

The 14 Steps of Performing an Audit

1. 1. Receive vague audit assignment
   2. Gather information about audit subject
   3. Determine audit criteria
   4. Break the universe into pieces
   5. Identify inherent risks
   6. Refine audit objective and sub-objectives
   7. Identify controls and assess control risk
   8. Choose methodologies
   9. Budget each methodology
   10. Formalize the audit program
   11. Perform & document audit methodologies
   12. Conclude
   13. Draft findings
   14. Finalize report

How to judge an audit methodology

Go ahead. Admit it. You are copying other auditors’ procedures to perform your audit. You are glancing over last year’s audit program and using that as a basis for your current audit methodologies. Although that isn’t ideal, I understand why you are doing that. But lecturing you on why you should write customized audit methodologies is not the subject of this newsletter. For more on the dangers of SALY (Same As Last Year) auditing, please see an ancient article of mine, here.

The subject of this article is judging whether a procedure or, as I call it here, “AUDIT METHODOLOGY” is worthy of doing. Just because an auditor performed it in years past, does not mean it was a solid idea! So, in this article, I hope to help you develop a more discerning eye when you do try to “customize” an old audit program for your use now.

Here is what makes an audit methodology worthy of copying. A worthy audit methodology is:

• POINTED: It answers the objective directly.
• STRONG: It creates convincing, strong evidence.
• CHEAP: It is worth the time you spend on it.
• EASY: Evidence is easy to access and work with.

Let’s cover each of these ideas in turn.

Pointed

When I was a college student, I spent many hours shooting darts with friends in the student union. We would get a pitcher of Spaten beer and compete with anyone who would take the challenge. After a few beers, we felt ready to take on the world dart champion. Every dart hit the bullseye. But after a few too many beers, we started missing the target entirely!

These same friends also liked playing with guns (I did go to school in Texas). Yet, no beer was involved in this activity! One friend had a shotgun that would tear your shoulder off if you didn’t hold it just right. I hated shooting that, but it would do serious damage to a target. The shot was scattered all over the target. Very few pieces of shot actually hit the bullseye, but you could hardly tell because the paper holding the bullseye was all torn up.

I liken using someone else’s audit program to shooting the target with a shotgun. You perform a lot of extra, unnecessary procedures, just to ensure that some of them hit the bullseye.

It is much more efficient and, in my opinion, fun to use a dart to hit the bullseye. If you liken the holes in the target to audit effort, a dart is so much quicker and simpler – assuming you aren’t drinking beer.

And, obviously, you can’t hit a bullseye that is vague or moving. A bullseye in auditing is the audit objective. The bullseye must be clearly defined and stable. Remember, a good audit objective has a finite subject matter, a firm criteria, it is stated as a question and it may include a performance aspect.

Strong

The second attribute of a good audit methodology is that it gathers strong evidence. Three facets affect the strength of evidence: the evidence type, the source of the evidence, and the amount of evidence that is gathered

Type: Evidence can be physical, documentary, or testimonial. The GAO does a good job describing the types of evidence in the Yellow Book:

8.104 In terms of its form and how it is collected, evidence may be categorized as physical, documentary, or testimonial. Physical evidence is obtained by auditors’ direct inspection or observation of people, property, or events. Such evidence may be documented in summary memos, photographs, videos, drawings, charts, maps, or physical samples. Documentary evidence is already existing information, such as letters, contracts, accounting records, invoices, spreadsheets, database extracts, electronically stored information, and management information on performance. Testimonial evidence is obtained through inquiries, interviews, focus groups, public forums, or questionnaires. Auditors frequently use analytical processes, including computations, comparisons, separation of information into components, and rational arguments, to analyze any evidence gathered to determine whether it is sufficient and appropriate. Evidence may be obtained by observation, inquiry, or inspection. Each type of evidence has its own strengths and weaknesses.

Each of these types of evidence has an inherent weakness or two. The inherent weakness of testimonial evidence is that the person testifying can be lying or simply telling you what you want to hear. Or maybe he simply doesn’t know what he is talking about.

The inherent weakness of documentary evidence is that it can be forged. Yes, you can easily create just about any document you want. If you have the guts to visit a few questionable websites, Google “fake receipts” and “fake diplomas.”Once on these sites, you can create a taxicab receipt or graduate from Harvard; take your pick.

Physical evidence can also be tricky. Crazy Eddie, the famous fraudster, was very pleased that his auditor told him in advance the dates and locations of the auditor’s planned inventory counts. That way Crazy Eddie could move the TVs and stereos that he had stored in NYC to Boston and then on to Philadelphia just ahead of the auditor. The unsuspecting auditor counted the same inventory multiple times and concluded that Crazy Eddie had plenty of merchandise to sell.

Physical evidence can also be garnered by observing a process. But what is the inherent weakness here? Well, how do you act when an authoritative person observes you? During an observation, the auditee can easily “stop hitting the bottle” and act right. After the auditor leaves, he will return to his normal, non-compliant behavior.

Source: From whom you get the evidence matters – a lot. Please do not source every bit of your evidence from auditees. They are not exactly objective!

For instance, I once worked with a regulatory agency audit shop that was responsible for determining whether extracted gas tanks were remediated. In less fancy terms, if you own a gas station and you close shop, you must dig the gas tank out of the ground, properly dispose of it, and clean up the soil.

And remediation isn’t cheap. Owners of the stations had fooled the auditors many times. Some owners would do anything not to spend that money and get the auditors off their backs. They would fake moving invoices and engineering studies; they would even change the map of the property to make it look like the tank was buried yards away from where it truly was.

Because evidence coming from the owner of the gas station could not be trusted, the auditor got in the habit of asking neighbors whether they had seen any activity at the station. If a dishwasher having a smoke behind a neighboring restaurant reported that he had seen a tank dug out of the ground, the auditor felt more confident in the evidence provided by the owner of the station.

The GAO also counsels us on the need to source your evidence from unbiased sources.

8.104f. Evidence obtained from a knowledgeable, credible, and unbiased third party is generally more reliable than evidence obtained from management of the audited entity or others who have a direct interest in the audited entity.

Quantity/Scope: My second husband and I wanted to get married soon after we met. But one of my girlfriends slowed us down. He was so giving, so kind, so balanced; she simply couldn’t believe he was real. So we dated for a year before we married. I was able to gather a good amount of convincing evidence about how great he was as I watched him with his family and observed him during the holidays, sick days, and mundane workdays. He did turn out to be for “real,” by the way, and, after a year, I could confidently walk up the aisle.

Maybe not all auditors are as cautious as I am in their personal lives, but they should be in their professional lives. Evidence is stronger if we have a lot of it and it covers a wide timeframe. For instance, a sample of 10 items from just one month of the year is nowhere near as convincing as a sample of 120 harvested from all 12 months of the year. Here is some wisdom from the GAO’s Yellow Book:

8.101a. The greater the audit risk, the greater the quantity and quality of evidence required.

8.109 When assessing the overall sufficiency and appropriateness of evidence, auditors should evaluate the expected significance of evidence to the audit objectives, findings, and conclusions; available corroborating evidence; and the level of audit risk. If auditors conclude that evidence is not sufficient or appropriate, they should not use such evidence as support for findings and conclusions.

How much evidence is enough?

Sometimes new auditors ask me, “How much evidence is enough?” They want to know whether three pieces of evidence will be enough. And I have to break the bad news to them that there is no rule of thumb for how many pieces of evidence are necessary.

Many auditors say they have a “warm fuzzy feeling” about their evidence when they can imagine standing in front of an adversarial governing body and confidently report that something has or has not happened.

So, let’s say you are going to tell a university that its head coach has been using his university-issued credit card to buy furniture and electronics for his lake house. The governing body will not like that news, especially if the coach has been winning games. You can expect that the board will challenge your conclusion. Will you need one, two, seven pieces of evidence? You will have to make that judgment. And your judgment will differ from mine.

Remember the promise we make to the users of our audit report if we are following Yellow Bookstandards:

9.03 We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Sufficient means you got enough evidence (a measure of quantity) and appropriate means it relates to the subject matter (a measure of quality).

Cheap

We know we need enough evidence to be convincing, but each piece of evidence we gather will cost us. Testing a nice, large sample will take a long, long time, and most auditors are constrained by audit resource budgets.

In public accounting, any procedure that takes longer than four hours to perform is questioned because time is money in that arena. And while internal auditors and government auditors aren’t always under time pressure, it is still silly and wasteful to spend time performing an audit methodology that takes a whole week when one that only takes half-a-day would be just as convincing

Easy

Some information is hard to get. Sometimes you need additional training to work with it; sometimes you need to make extraordinary effort to get to it.

What if I were asked to conclude whether medical claims were medically necessary? Any judgment I would make would be a complete guess because I would need to consult with a trained medical professional to make that conclusion. (I was at a doctor’s office recently looking at diagrams of a human nervous system. The diagrams approached the subject from several different angles, and I got lost very quickly because I was trained as an accountant, not a doctor.) And that adds complexity to my work and isn’t easy.

Or what if the evidence I needed was far away and I had to travel to get it? Once I traveled all over Texas to physically verify the existence of office equipment. It wasn’t my idea, O.K.?!?

Example Audit Methodologies

Let’s apply what we have learned. Let’s look at three different audit methodologies and determine their worthiness.

Let’s say that our audit objective is pretty straightforward, like, “Are expenditures allowable?” Are the following audit methodologies:

• POINTED? Does it answer the objective directly?
• STRONG?Does it create convincing, strong evidence?
• CHEAP? Is worth the time you spend on it?
• EASY? Is the evidence easy to access and work with?

Audit Methodology #1

Scan general ledger accounts that have material balances for large or unusual expenses.

• POINTED: No. Not all general ledger accounts relate to expenses. Some are for assets and liabilities and equity, etc.
• STRONG: The scan is based on the general ledger, which is a database created by the client. The general ledger is not an objective source of evidence.
• CHEAP: It could be, or it could not be. If we see plenty of questionable transactions, we will have to investigate them, which will take time.
• EASY: Yes, a cursory scan of accounts is pretty easy. What we do with the large and unusual expenses next could make the follow-up procedure crazy hard.

Audit Methodology #2

Select a sample of 20 expenditures and determine that the expenditures were properly coded in the general ledger and in the report.

• POINTED: Nope. Whether expenditures are properly coded or not has nothing to do with whether they are allowable or not. They could be properly coded yet unallowable.
• STRONG: This methodology doesn’t specify the source of the expenditure data for our sample. Is it from a general ledger report? Or is it from receipts? Or a credit card bill from the credit card company? (I like the credit card bill for a source, but it won’t capture all expenditures. The upside of a credit card bill is that it is from an objective source!)
• CHEAP: This is a cheap procedure but not worth the time that is spent on it because it is not pointed.
• EASY: Yes, the evidence should be easy to obtain.

Audit Methodology #4

Select a sample of 10 expenditures and confirm that the expenditures are allowable per the list of allowable expenditures.

• POINTED: Finally, we have a methodology that actually relates to the objective!
• STRONG: The sample size is pretty small, unless the population of all expenditures is also small. And we have no idea where we are pulling the sample. Hopefully, it is an objective source.
• CHEAP: Yes, on the surface it seems like it would be a pretty quick procedure.
• EASY: This depends on where we get the sample. But I would assume it to be pretty easy because we have firm criteria (the list of allowable expenditures) against which we can compare the expenditures.

Use these four attributes to determine whether the audit methodologies you plan to use will get you where you want to go.

Linking Objectives to Related Methodologies

One way to verify that your audit methodologies will get you where you want to go is to link the master objective to the sub-objectives and the related methodologies. I like to lay it all out visually. The audit structure will look something like this:

And you would assign hours to each of the other two objectives – audit methodology by audit methodology – in a similar manner.

Please keep in mind that nearly 40% of the audit budget has been consumed just to get to this point!

So, use these four attributes to determine whether the audit methodologies you plan to use will get you where you want to go. If not, it is time to design a new methodology that is pointed, strong, cheap, and easy.

Please share your thoughts with me at Leita@yellowbook-cpe.com

Next time, audit programs!

Steps12 & 14 are not specifically discussed in a blog.