FINRA to Require Systematic Verification for AI
In their 2026 Regulatory Oversight Report, FINRA emphasized Rule 3110 (Supervision) and how it pertains to the adoption of AI. The rule states that a member firm must have a reasonably designed supervisory system tailored to its business. The Report goes on to emphasize the need for validation and human-in-the-loop review of model outputs on a regular basis.
Human-in-the-loop
Hallucination rates attributed to AI tools have been reported north of 50% in some instances, even from tools at the application layer published by companies who have all the incentives to get things right.
The practical answer most firms reach for is more human eyes on more output. But if you end up throwing more human resources to review unprecedented volumes of sloppy work produced by artificial intelligence, any efficiency or even cost savings get thrown out the window.
One report focused on agentic coding reveals that while AI adoption caused the number of accepted codebase contributions (merged PRs for those that are familiar) to increase by 98%, review time for these contributions increased 91%. When you factor in mistakes that get missed by human review, it begs the question if AI adoption has driven any efficiency whatsoever.
Generation-Verification Gap
The other instinct is to throw more AI at the problem, which is also problematic and technically proven to be ineffective. AI researchers have noted the systematic discrepancy between the ability of generative models to produce plausible content and the reliability with which such content can be verified for correctness, consistency, authenticity, and alignment with external ground truth. What's referred to as the 'Generation-Verification Gap' is widely recognized as a core limitation not only in LLMs but also in applied AI across all domains.
So we've established three things:
- There is a regulatory need to verify generative AI outputs.
- Human verification erases the efficiency and cost savings that AI is supposed to deliver.
- Using AI to check AI is proven unreliable.
So what else can we do?
Introducing Tie's Deterministic Verification Layer
Over the last few months, our team has been building a deterministic verification layer for knowledge work that solves both problems. Tie breaks down human- and AI-generated content into units that can be tested individually and in aggregate with deterministic code. The solution automates the verification requirement without falling victim to the GV gap and keeping the human in the loop efficiently and voluntarily.
More to come on this topic.
