Can We Prove Our Security Controls Work?

If a critical security control failed today, what evidence would tell leadership—and how quickly could the organization act?

A security dashboard can show completed training, deployed technology, and closed remediation tickets. Those measures tell us work occurred. Leaders still need to know whether that work protects the operations they depend on.

My position is straightforward: confidence in a security control should rest on evidence of its performance in the environment where it matters. A policy establishes an expectation. A configuration shows how a safeguard was set up. A test shows what happened under defined conditions. Each contributes something different to the decision.

The leadership responsibility is to understand what the evidence supports, where its limits lie, and who must act when those limits leave the business exposed.

Start with the business outcome

"Backups are successful" is an incomplete assurance statement. Can the organization restore the systems, data, and dependencies required to resume a critical service within the time the business can tolerate?

"MFA is enabled" raises another question. Which access paths, users, privileged identities, and exceptions are included? An enrollment percentage can conceal an unprotected route into an important application.

Before requesting more metrics, identify the consequence a control is intended to prevent or limit. Interrupted patient care, unauthorized payment, disclosure of sensitive information, and loss of production create different priorities. The business owner should define the operational consequence and acceptable performance. Technology and security teams should demonstrate the safeguards supporting that outcome.

NIST's Cybersecurity Framework 2.0 provides a common language for cybersecurity outcomes across sectors. It does not prescribe how organizations must achieve them. Leadership must translate those outcomes into requirements suited to its operations. ^[1]

Ask what the evidence actually demonstrates

NIST SP 800-53A Revision 5 provides a methodology for assessing security and privacy controls, with procedures that organizations can tailor to their needs and risk tolerance. Its assessment approach helps move the discussion from whether a control exists to whether it achieves its intended objective. ^[2]

For executive oversight, I would distinguish four questions:

Practical leadership questions for assessing confidence in a control. This is an editorial aid, not a NIST assessment model.

These are practical review questions, rather than a separate NIST maturity model.

A screenshot from one server cannot establish fleet-wide coverage. A successful test in a demonstration environment cannot establish production performance. A passing result from last year may remain useful, but its relevance depends on what has changed.

A successful backup can still leave a hospital unable to operate

Consider this illustrative healthcare scenario.

A hospital's overnight backup report shows successful completion. During a controlled recovery exercise, the team restores the clinical application database, but an identity service and an interface dependency are unavailable. Clinical staff cannot complete the workflow needed to use the restored information.

The backup process performed its assigned task. The hospital's recovery objective was not met.

The exercise should produce more than a technical ticket. Clinical operations should establish which workflows must resume and in what order. Technology teams should map the dependencies and demonstrate recovery. Security should evaluate how the recovery environment is protected, including the assumptions made about compromised credentials or infrastructure.

Leadership needs the measured interruption, the operational consequences, any interim arrangements, and the owner responsible for closing the gap. A scheduled retest should establish whether the corrective action worked.

This same reasoning applies to a manufacturer restoring production scheduling or a financial institution recovering payment services.

Make control testing part of change

A control can become less effective after an application migration, an acquisition, a new access route, or a supplier integration. Testing needs to follow the material change.

I would expect control owners to identify what makes existing evidence insufficient. A new privileged integration may require access testing. A revised recovery architecture may require a restoration exercise. A change in log collection may require verification that meaningful events still reach the team expected to respond.

Test safely. Agree on scope, authorization, expected results, and stop conditions before execution. Use controlled environments and synthetic data where appropriate, and coordinate production testing with the operational owner.

For a high-consequence control, seek a level of review independent of the people who implemented it. The depth and frequency should reflect the risk and the decision being supported.

AI adds another reason to test the full workflow

An AI application can produce an acceptable response while its surrounding workflow exposes data or grants excessive authority. Testing model output alone leaves those questions unresolved.

Consider an AI assistant authorized to summarize supplier documents. What happens when a document contains instructions to retrieve unrelated information or initiate an action outside the approved task? A useful assessment examines the application's permissions, data access, approval steps, logging, and ability to stop further activity.

The test should distinguish a model generating text from an application taking action through connected tools. It should also verify whether the human reviewer has enough information and authority to challenge the proposed action.

NIST's AI Risk Management Framework calls for evaluation in conditions similar to deployment, documentation of measurements, and monitoring of production behavior. Its MEASURE function also addresses security, resilience, and privacy evaluation. ^[3]

For leadership, that means asking for evidence tied to the deployed use case. A vendor benchmark can inform evaluation, but the organization still needs to understand what happens with its data, integrations, users, and operating conditions.

Report uncertainty before it becomes an incident

A credible executive report should make untested coverage visible.

"Not tested," "partially covered," and "failed retest" describe different conditions. Combining them into a green completion percentage deprives leadership of information needed to allocate resources or restrict an activity.

For a small number of critical services, request a concise record of:

• The business outcome and accountable owner.
• The control assessed, scope, assessment date, and result.
• Any material exclusions or dependencies.
• The operational consequence of an unresolved gap.
• The corrective action, due date, and verification plan.
• The executive decision needed while the exposure remains.

The control owner should produce the evidence. The relevant business owner should evaluate the operational consequence. Risk acceptance should follow established authority, with input from security, privacy, compliance, legal, and other functions as the issue requires.

A due date is a management commitment. It does not reduce the exposure by itself.

Continue the Exploration

This discussion connects the program responsibilities in Enterprise Information Security Program™, the operational practices in Cybersecurity Operations Manual™, and the evidence and review disciplines in Security Compliance & Assurance Manual™. Together, these volumes help leaders connect security expectations to day-to-day performance and defensible oversight.