/
Streamlined Audit/Evaluation Report Writing Streamlined Audit/Evaluation Report Writing

Streamlined Audit/Evaluation Report Writing - PowerPoint Presentation

olivia-moreira
olivia-moreira . @olivia-moreira
Follow
347 views
Uploaded On 2018-12-05

Streamlined Audit/Evaluation Report Writing - PPT Presentation

Presentation to 2014 FAEC Conference Andy Patchan AIG for FRBCFPB 1 2014 by Andrew Patchan Jr United States of America All rights reserved No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author ID: 736536

findings report patchan part report findings part patchan means 2014 andrew electronic reserved author permission written mechanical form reproduced

Share:

Link:

Embed:

Download Presentation from below link

Download Presentation The PPT/PDF document "Streamlined Audit/Evaluation Report Writ..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.


Presentation Transcript

Slide1

Streamlined Audit/Evaluation Report Writing

Presentation to 2014 FAEC ConferenceAndy Patchan, AIG for FRB/CFPB

1

© 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.Slide2

My Background

10 years at GAO, learned report writing from the ground up, with lots of supervisor revisions and little guidance2 years as an attorney at DOJ, where I gained experience in drafting legal opinions and case decisions

13 years at GSA OIG, from audit manager to AIGA, where I began identifying ways to streamline report writing/development and helped develop report writing courses2 years as SES at

Department of Education OIG, where I experienced other processes for report writing/development, but still similar challengesAt the FRB/CFPB OIG since 2008

2

© 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.Slide3

The Challenge of Report Writing

Focus of audits/evaluations is on the fieldwork – identifying and developing findingsReport Drafting and fieldwork require different skill sets Lots of revisions by Managers

3

© 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.Slide4

Problems in Audit Report Writing

No existing methodology for auditors to draft reports – same tools as 1980s. They have to look at past reports to try and understand how to draft the findings, conclusions and recommendations. Reviewers provide

comments at meetings but less experienced auditors are unsure how to proceed

Provided a methodology for auditors to follow in drafting findings, linking findings into a report message, and flowing the message into conclusions and recommendations

Long Timeframes to Develop the Initial Draft

Decreased timeframes 50% in initial pilot projects; discussion draft in 1 month

Multiple Rounds of Reviewer Comments and

Revisions

Only 2 rounds of reviewer comments in initial pilot projects

Frustration

and low morale on the part of auditors drafting the

report

On initial pilot projects, auditors took ownership of draft report, and were proud and excited about their draft

Long Timeframes

for Completion of

the Audit and

draft

report (12 months or more)About 6 months total for initial pilot projects

4

© 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.Slide5

Focus of Streamlined Report Writing

Not instruction on how to write better grammar or use of Chicago style manual

Not discussion of auditing and how to identify findings

Focus on a Methodology for organizing findings into a coherent, unified, and persuasive messageTools

to more expeditiously develop a draft report and reduce frustration and management revisions

5

© 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.Slide6

Key Areas for Focus

Linking Findings to Audit Objectives Fully Developing Findings Before Drafting

Message Development Conference

Report Conference Report Message and Charge Paragraphs

Conclusions and Recommendations

Finalize the above before Beginning Drafting

© 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.

6Slide7

Link Between Audit Objectives, Findings, and Report Message

Audit Objectives define the main questions posed on your audit, and the boundaries for your areas of inquiry (audit scope), methodology, data to be collected, conclusions, and recommendationsAudit objectives can address compliance, program effectiveness, adequacy of controls, economy/cost-effectiveness, etc

.The report must answer the questions posed by the audit objectivesAll findings must tie-in to the audit objectives.

7

© 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.Slide8

Examples of Objectives(1) Determine the extent to which the Pell grant program is properly controlled and administered to prevent fraud and abuse

.

(2) Assess the adequacy of the process and controls for effectively reviewing the Bonus Program applications to ensure eligibility requirements are met.

(3) Assess how the information technology system will meet select requirements within planned timeframes.

8

© 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.Slide9

Link Between Audit Objectives and Findings (cont.)

How does a differently worded objective change the area of inquiry for potential findings and potential report message?

Example Objective 1: Determine whether adequate procurement controls are in place to prevent fraud, waste, and abuse. Example Objective 2: Determine whether adequate procurement controls are in place to prevent fraud, waste, and abuse and assess whether additional costs are incurred.

9

© 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.Slide10

Linking Findings to the Objectives (cont.)

Objective 1: Determine whether adequate procurement controls are in place to to prevent fraud, waste, and abuse.

Finding 1:Criteria: Federal Acquisition Regulations require an independent government cost estimate before soliciting offers from contractors for space alterations, to help evaluate proposed prices.

Condition: Building managers are not always developing an independent government cost estimate (IGE).Effect: The contracting officer may not be able to negotiate lower prices

Cause: Building managers were not aware of existing regulations and were not exercising proper oversight to ensure compliance.

10

© 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.Slide11

Linking Findings to the Objectives (cont.)

Objective 2: Determine whether adequate procurement controls are in place AND assess whether additional costs are incurred.

Finding 1: Criteria: Federal Acquisition Regulations require an IGE before soliciting offers from contractors for space alterations, to help evaluate proposed prices

. Condition: Building managers are not always developing an IGE

. Effect: We reviewed 10 contracts with an estimate and found in 9 cases, the contract award cost met the IGE. In 10 contracts where there was no IGE, the contract cost was on average 20% higher than IGEs for other similar work. In addition, in some of the latter 10 contract files, notes by the contracting officers indicated they were unsure whether prices were reasonable and how to negotiate further.

Cause: Building managers were not aware of existing regulations and were not exercising proper oversight to ensure compliance.

11

© 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.Slide12

Link Between Audit Objectives

and Findings (cont.)Thus, two slightly different objectives but with VERY DIFFERENT MESSAGES. The objectives define not only your areas of inquiry, scope, and methodology, but also the message of your findings and report.

using the 1st example objective, the reader is expecting a message of whether controls are operational, and instances of where they’re lacking.

using the 2nd example objective, the reader is expecting a message of whether the agency is incurring excess costs, and by how much, and that corrective actions are needed as soon as possible.

12

© 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.Slide13

Fully Develop Findings Before Drafting

Audit Findings Are the Building Blocks for Your Audit ReportMuch like constructing a house, if we don’t finalize the basic floorplan, our construction efforts will have to be redone, at significant effort and expense, as the floorplan is changed

Not fully developing the findings during fieldwork results in redrafting over and over again as you attempt to develop the finding while you’re writing

13

© 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.Slide14

Fully Develop Findings Before Drafting (cont.)

Establish the 4 elements of each finding

Sample Finding Objective: Determine the Effectiveness of the Stock Program in Filling Client-agencies’ Requirements for Supplies

14

© 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.Slide15

Fully Develop Findings Before Drafting (cont.)

Audit Objective: Determine the Effectiveness of the Stock Program in Filling Client-Agencies’

Requirements for SuppliesCriteria: To optimize the Stock Program operations, inventory

managers need to ensure adequate stock is ordered to

fill customer orders (Inventory Mgt. Handbook)Condition: We analyzed 100 stock items and found some items had insufficient stock available to meet customer orders, while others had lower customer demand and excess stock.

Effect

: Where insufficient stock, customer orders cannot be filled.

Cause

: The Stock Program computer system inadequately

projects

amounts of stock to be ordered

15

© 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.Slide16

Fully Develop Findings Before Drafting (cont.)

Analyze each element to determine what additional information is needed. What further questions need to be asked?Ask yourself “so what;” do the elements separately, and together, tell an important message?What is the message?

16

© 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.Slide17

Fully Develop Findings Before Drafting (cont.)

Your findings must convey why the deficiency is important and needs to be corrected.It is not enough to just indicate that procedures vary from what is required (not persuasive to the reader).

Keep in mind the cause, and what part that plays in the message of the finding.

17

© 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.Slide18

Fully Develop Findings Before Drafting (cont.)

For example, the following points convey the importance of a finding:Key goals of the program are not being achievedExcess funds are being used unnecessarily

Key controls are lacking/can be improved

18

© 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.Slide19

Fully Developing Findings Before Drafting (cont.)

Example of Fully Developed, convincing Finding:Criteria: To optimize Stock Program operations, inventory managers need to ensure that adequate stock is ordered to fill customer orders (Inventory Management Handbook)

Condition: We analyzed 100 stock orders and found amounts ordered are inconsistent with customer demand in 80% of the cases. In 60 cases, insufficient stock was ordered. In 20 cases, excess stock was ordered. Yearly stock orders under the program represent $600 million.

19

© 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.Slide20

Fully Developing Findings Before Drafting (cont.)

Effect: Where insufficient stock was ordered, customer orders could not be filled, causing customers to go elsewhere and a risk of losing them as continuing clients. Where stock was in excess, holding costs increased an average of 10%, and depending on the item, such as paint or glue, could result in spoilage and waste. Yearly holding costs are $50 million.

20

© 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.Slide21

Fully Developing Findings Before Drafting (cont.)

Cause: The Stock Program’s computer system contributed to the problems in that the system’s recommended actions to the inventory manager are not based on actual customer demand and vendor lead-time data. Instead it relies on historical customer data and computer models from the 1970s, and inventory managers lack experience to make adjustments.

21

© 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.Slide22

Message Development Conference

Now that you have developed the 4 elements of

each finding – you need to present your findings to OIG managementTypical attendees: AIC, audit staff, audit manager, senior manager/regional IG/AIGA, Deputy IG (depending on organization setup and size)Audit team presents findings and associated elements

Attendees raise any questions about the impact, significance, or facts about the finding or its elements to ensure sufficiencyLength of conference – 1 to 2 hours

22

© 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.Slide23

But what follows the Message Development Conference?

How do we ensure the AIC and audit team know how to go forth in drafting?What is the starting point for the audit team in drafting?How does the audit team divide up report writing responsibilities?

© 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.

23Slide24

Reach consensus among the audit team on the report message and logic flow of the findings

Answer audit team questions on draftingHelp the audit team to “hit the ground running” as the start the drafting

Attendees: AIC, audit team, audit manager, senior audit manager, Regional IG/AIGA (depending on the organizational setup)

Length of time – ½ to 1 day, depending on scope of audit© 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.

24

Report Conference

Can Bridge

the GapSlide25

Determining the Report Message and Charge Paragraph

Primary task is to “synthesize” the findings into the report charge paragraph, to create your overall messageOnly select elements of each finding may be presented, such as effect or condition

Some findings may be “nested,” such as where the cause of the finding also functions as a finding with its own elements

25

© 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.Slide26

Determining the Report Message and Charge Paragraph (cont.)

This part of report development is the most tedious:The team is trying to craft the raw message of the reportHave to use trial and error to figure out the best fit

Need to try fitting elements of findings with others and combine the flow of the findings to achieve the best logical flow and message

26

© 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.Slide27

Determining the Report Message and Charge Paragraph

As you proceed in this work, what message or story line jumps out at you?Which are the strongest findings/elements and which are the weakest?What are the linkages between the findings

Which points depend on others to make the case (which are primary versus supporting points)?

27

© 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.Slide28

Synthesizing the Main Points of the Findings Into a Report Message and

Charge Paragraph (cont.)Let’s Revisit the Previous Fully Developed

Finding:“Inventory Orders Are Not Adequately Managed to Minimize Out-of-Stock and Overstock Conditions” (condition-based with effect introduced)

In 60 of the cases sampled, stock ordered was insufficient to meet customer orders - may lose the customer - Handbook 999.99 requires adequate stock to fill customer orders (condition, effect, and criteria)

In 20 of the cases, stock was excessive, which could increase program holding costs by several million dollars (condition and another effect)Stock Program information systems inadequately determined amount of stock to be ordered; the systems don’t use actual data, and the computer logic is from the 1970s (cause)

28

© 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.Slide29

Synthesizing the Main Points of the Findings Into a Report Message and Charge Paragraph (cont.)

Let’s assume another team member has identified an additional finding about problems with a new computer system under development to replace the old 1970s inventory management system and improve the

ordering of adequate stock. You’ll need to integrate this additional finding into your Charge Paragraph and in conjunction with your previous finding about problems with the stock program.

29

© 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.Slide30

Synthesizing the Main Points of the Findings Into a Report Message and Charge Paragraph (cont.)

How do you then craft these points together in one report charge paragraph? Logical ordering of the points?

Flowing story? Unified message?

30

© 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.Slide31

Synthesizing the Main Points of the Findings Into a Report Message and Charge Paragraph (cont.)

Example of Report Charge Paragraph with Combined Report Message

“Inventory Orders Are Not Adequately Managed to Minimize Out-of-Stock and Overstock Conditions”In

80 of the cases, ordered amounts was not sufficient to meet customer ordersIn 20 of the cases, excess stock was ordered, increasing holding costs by several million dollars.

The 1970s stock program computer system contributes to this problem by relying on older methods for determining amount of stock to be ordered (

cause links to the additional finding

)

Officials are developing a new replacement information system to better order optimal stock amounts, but the system is already undergoing design before all requirements have been detailed.

Making additional changes after the system is designed could result in costly retrofits.

31

© 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.Slide32

DRAFTING REPORT SECTIONS FROM THE COMBINED REPORT MESSAGE AND CHARGE PARAGRAPH

Once you have determined the main points of each finding and ordered/flowed them in a report message/charge paragraph, you have accomplished Phase 1 of Report DraftingPhase 2 is the actual “pen to paper” (finger to keyboard) of fleshing out the details of each point of your report message/charge paragraph

It cannot be over-emphasized how important completion of Phase 1 and the prior steps we discussed is prior to detailed report drafting, or else you will waste time drafting multiple different versions trying to pitch findings in different ways, and relating different findings to each other in various combinations.

If Phase 1 is completed well, Phase 2 can actually be the easier part of the drafting process.

32

© 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.Slide33

DRAFTING REPORT SECTIONS FROM THE COMBINED REPORT MESSAGE AND CHARGE PARAGRAPH (cont.)

In Phase 1 you determine how the pieces fit to complete the puzzle.In Phase 2 you craft the details for one piece (section or subsection) at a time. Each piece tells a specific part of the story.But the ordering and content of the sections and subsections need to flow from one another and each need to tell a convincing story or argue a specific point in a persuasive manner.

33

© 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.Slide34

DRAFTING REPORT SECTIONS FROM THE COMBINED REPORT MESSAGE AND CHARGE PARAGRAPH (cont.)

Have to decide how

to structure/map out the flow of the main points of the findings into sections and subsections?Which are separate sections, or subsections of other sections?

Do some points get paired with other points in the same section?Which points are stronger than others?

How do the weaker points fit in?

34

© 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.Slide35

LINKING THE FINDINGS TO THE CONCLUSIONS AND SETTING UP THE RECOMMENDATIONS

Conclusions are a very different part of the report; they bring together the message and tell the reader what it all means – the bottom line Not just a summary of the main points

You need to stand back from the details in the draft Determine the “bottom line” message(s) at a higher level than the details of the findings

Need to flow reader directly into recommendations The reader should expect them.

35

© 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.Slide36

Final Thoughts

“The time to begin writing an article is when you have finished it to your satisfaction. By that time you begin to clearly and logically perceive what it is you really want to say” (Mark Twain)“I learned that the only way you are going to get anywhere in life is to work hard at it. Whether you’re a musician, a writer, an athlete or a businessman, there is no getting around it. If you do, you’ll win - - if you don’t you won’t” (Bruce Jenner)

36

© 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.Slide37

Questions?

Andrew.patchan@frb.gov

© 2014 by Andrew Patchan Jr., United States of America. All rights reserved. No part of this guide may be reproduced in any form or by any electronic or mechanical means without written permission from the author.

37