Chapter 8: Who Checks the Checkers? Independent Appraisal of EROM

Book: Enterprise Risk and Opportunity Management: Concepts and Step-by-Step Examples for Pioneering Scientific and Technical Organizations
Author: Allan S. Benjamin
ISBN: 9781119288428

After seven chapters of building the EROM machinery, Benjamin turns to a question every federal agency and large technical enterprise eventually faces: who verifies that the machinery actually works? Chapter 8 is about independent appraisal. Not the kind where someone skims a slide deck before a quarterly review. The kind that gives executives, legislators, stockholders, and the enterprise itself real assurance that risk and opportunity decisions are informed, integrated, and defensible.

Why independence matters

TRIO enterprises juggle risks that would make a banker’s spreadsheet curl. OMB Circular A-123 (2016) pushed this further by tying enterprise risk management directly to internal control validation. Auditors, internal or external, are expected to evaluate both. Management and auditors may read the same risk differently based on their roles. The circular says the agency risk function should coordinate with auditors so independence stays intact while risk information still flows where it needs to go.

For federal agencies, independent evaluation gives the executive and legislative branches confidence that significant risks and opportunities are recognized and handled. For commercial enterprises, it gives stockholders and creditors the same comfort. Inside the organization, it tells decision makers at every level that their choices rest on solid analysis, not wishful thinking.

What the guidance actually says

Benjamin pulls from three sources.

OMB A-123 treats independent audits as essential for the annual internal control assurance report required under FMFIA. Continuous monitoring of controls tied to significant risks should feed that assessment.

Department of Energy (2014) routes independent evaluation through financial statement audits, quality assurance, and peer review. DOE also makes a point worth repeating: controls for intolerable risks should be tested more often than controls for marginal ones. Their example risk categories (human resources, contractor oversight, procurement, budget execution, safeguards and security) read like a checklist any federal science agency would recognize.

UK Institute of Internal Auditors (2009) draws a bright line between what internal audit should do and what management owns. Core audit roles include assuring that risk management processes work, risks are evaluated correctly, and key risks get reported and reviewed. Legitimate audit roles with safeguards cover facilitation, coaching, and framework development. Off limits for auditors: setting risk appetite, imposing processes, taking risk response decisions, or owning accountability. Internal audit can consult, but it cannot manage risks. That is management’s job.

What an appraisal must cover

An independent evaluation of Benjamin’s EROM approach cannot stop at internal controls. It has to trace the full chain from team structure through risk acceptance. The appraiser needs to verify:

• How the EROM team and subteams are organized
• How the objectives hierarchy was built and how objectives connect
• How risk tolerances and opportunity appetites were derived from stakeholder parity statements
• How scenarios, leading indicators, roll-ups, and drivers were identified and evaluated
• How mitigations, opportunity actions, and controls were selected and optimized
• Whether implementation plans are viable
• Whether residual aggregated risks and opportunities are acceptable

That is a lot. Which is why Benjamin provides Table 8.1, a 56-question evaluation template organized into twelve categories.

The 56 questions in plain terms

Rather than reproduce all 56 rows, here is the logic. Each category asks whether the EROM team did the work correctly, documented the rationale, and closed gaps.

Team structure (items 1-5): Are scopes defined? Do teams have the right skills? Is communication regular? Is there a shared database with appropriate access controls? Does leadership visibly support the effort?

Objectives hierarchy (6-9): Were sources interpreted correctly? Are all important objectives included? Are interfaces between objectives identified and explained?

Risk tolerances and opportunity appetites (10-12): Did stakeholders get queried? Were parity statements converted into watch and response boundaries correctly? Is the rationale documented?

Scenario identification (13-19): Were all important risks and opportunities captured, including those introduced by pursuing opportunities? Are cross-cutting scenarios handled consistently? Did anyone ask whether new objectives could advance the mission?

Leading indicators (20-30): Are indicators identified, including those that surface unknown and underappreciated risks? Are correlations to objective success transparent? Do trigger values align with tolerance boundaries? Are trends evaluated consistently across units?

Roll-ups (31-36): Is there a systematic bottom-to-top aggregation? Do roll-ups account for interfaces, redundancies, and workarounds? For commercial enterprises, do quantitative and qualitative monetary roll-ups agree?

Drivers (37-40): Do drivers reflect hardware, software, human response, controls, assumptions, and organizational factors, not just technical elements? Does each driver actually move aggregate risk or opportunity across tolerance boundaries?

Mitigations, actions, and controls (41-48): Are existing controls characterized? Are flaws found? Do alternatives address all drivers? Are new or modified controls practicable and do they protect key assumptions?

Optimization and planning (49-52): Were sensitivity analyses run? Did they produce near-optimal asset distributions and control selections? Is there an implementation plan?

Risk acceptance (53-56): Is cumulative risk acceptable now? Could more mitigations improve the picture? Are leading indicators being monitored? What is the recommendation going forward?

Each row has columns for evaluation result, recommendation, and resolution status. It is an audit workbook, not a philosophy lecture.

What surprised me

Most risk books mention “assurance” in passing. Benjamin hands you the questionnaire. The IIA role boundaries are also sharper than I expected. Auditors assure and evaluate. They do not set appetite or implement responses. That separation keeps the appraisal honest.

The DOE testing-frequency guidance is practical too. If you treat every control the same, you either over-test low-risk areas or under-test the ones that could sink a program.

The bottom line

Independent appraisal is not a box to check before the annual assurance letter goes out. It is the mechanism that connects everything in Chapters 1 through 7 to the internal control and risk acceptance decisions executives actually sign. If you are building an EROM program, start with Table 8.1. Run it before your auditor does. The gaps you find yourself are cheaper than the ones they find for you.

Previous: Chapter 7 Risk Acceptance Examples · Next: Chapter 9 Strategic Integration