Identify, validate, mitigate: Getting fraud control testing right

For organizations seeking to mature their fraud risk management capabilities, a structured program for regular evaluation of anti-fraud controls can be a highly effective tool. It may not generate the excitement of a new technology investment, but it can provide something equally valuable: objective evidence of what is and isn’t working.  

Control testing isn’t a groundbreaking concept. It’s rooted in established governance models such as the U.S. government’s Fraud Risk Framework, which recommends evaluating anti-fraud control suitability and effectiveness. Controls can range from specific tools and processes (such as two-factor authentication and document scanning for counterfeit detection) to broader policies and operations.   

Organizations can be reluctant to implement a control-testing program because they don’t see the value in it or aren’t sure how to do it effectively. But without control testing, risk managers often struggle to convert risk assessment findings into action, leaving flagged risks and their proposed mitigation strategies to die on the vine. Often that’s because risk identification is theoretical or lacks evidentiary data, or because executive and implementation teams find it hard to manage and prioritize a heavy stream of incoming risk assessment findings.  

Control testing helps resolve these issues by providing evidentiary data that shifts a theoretical risk into a demonstrable control deficiency or strength. That data in turn strengthens the risk assessment narrative, enabling better risk scoring and prioritization. Stronger narratives, especially ones supported by tangible proof of control deficiencies, can open the eyes of stakeholders and decision-makers and push the most urgent risks toward the top of their to-do lists.  

Probing an organization’s fraud controls through adversarial red-teaming and other means yields unparalleled technical understanding of the threats anti-fraud teams are grappling with. To quote the Nobel Prize-winning physicist Richard Feynman: “What I cannot create, I do not understand.” Control testing applies that maxim by helping teams reconstruct tactics and scenarios and—to invoke another maxim—“think like a fraudster.” This knowledge generates more-detailed mitigation strategies and clearer requirements for implementation teams. 

A three-step framework 

A practical approach to control testing starts with embedding it in the broader fraud risk assessment process. In this framework, anti-fraud control evaluation is integrated into a three-phase workflow, where control testing serves as the validation mechanism linking identification and mitigation. Here’s what this model might look like in practice. 

1. Identify 
Knowing what to test begins with knowing where to look. Proper control testing often requires an initial fraud risk assessment in which risks, controls, and potential control weaknesses are identified and catalogued.  

Consider a hypothetical scenario in which an anti-fraud team depends on a third-party vendor’s tool for anomaly detection. The vendor touts the tool’s security features, but functionally it remains a black box. A fraud risk assessment identifies the vendor dependency as a risk and surfaces indicators that some of the tool’s controls—in this case, its anomaly-detection features—may not be performing as intended. An increase in fraud losses, recurring attack patterns, and other operational observations suggest that certain controls warrant closer examination.  

At this stage, the organization has merely identified areas of concern and hasn’t yet proven whether those weaknesses truly exist. Nor do they know where existing weaknesses might reside or how threat actors are exploiting them. Bringing these concerns to the vendor is a logical next step, but the organization determines that it also needs to conduct its own validation. This is where control testing comes in. 

2. Validate 
Control-testing exercises help pinpoint precisely where technical weaknesses exist. Testing can be as simple as determining whether a control works, or it can be more adversarial, producing hard evidence of how a control can be evaded. In either case, testing data strengthens both risk assessment findings and recommendations. A “validated risk” can even be factored into the risk-scoring calculation, which in turn enables strategic prioritization. 

In our hypothetical scenario, the organization conducts a series of internal tests to validate the vendor tool’s specific deficiencies. An adversarial red-teaming exercise uncovers one vulnerability: A relatively unsophisticated bypass technique can elude the tool’s anomaly-detection controls. Armed with this knowledge, the anti-fraud team’s position on the identified vulnerability shifts from “we think” to “we know.” The team uses that testing data to refine and reinforce escalation to the vendor. 

3. Mitigate 
The mitigation phase often presents the most stubborn obstacles because turning validated risks into action usually requires external resources. That could be an engineering team with limited capacity and unfamiliarity with anti-fraud solutions.  

Control testing helps overcome these obstacles. It can offer clear guidance through detailed, technically precise recommendations and validate implementation outcomes to provide confidence that claimed fixes actually work. By reducing frustrating back-and-forth iterations of potential solutions, these two benefits often combine to create a third: speed. 

In our hypothetical scenario, the organization’s anti-fraud team decides to share its control-testing data with the vendor immediately. The vendor acknowledges the deficiency and responds quickly by updating one of its algorithmic models. While the vendor claims that the fix has eliminated the vulnerability, the anti-fraud team decides to validate it themselves. By working collaboratively with the vendor while also holding it accountable, the team ultimately confirms that the vendor’s fix has resolved the issue and mitigated the risk. 

A flexible model rooted in best practices 

Control-testing programs can be scaled up or down to meet resource availability, developed iteratively over time, and implemented by organizations of any size. Consistent methodologies, regular cadences, and documented standard operating procedures are essential regardless of implementation approach. 

For example, organizations conducting adversarial red-teaming should establish rules of engagement, create secure sandbox environments, and seek proper authorization. In some public sector contexts, that may require coordination with law enforcement. Thorough legal review is important, too, as some vendor contracts contain language that restricts certain types of testing. 

Even the best controls can deteriorate or become obsolete over time. For fraud prevention teams, this points to a simple truth: Any organization deploying anti-fraud controls also needs to implement fraud control testing. The identify-validate-mitigate framework offers a clear path for doing so successfully.