When No One Speaks Up, Should a System?

At LegalTechTalk this week, I sat in on a talk that hit in a way most couldn't match. Lee Castleton OBE and Flora Page KC were talking about accountability, ethics and what still hasn’t sunk in from the Post Office Horizon IT Inquiry.

It wasn’t a polite retrospective, it was a clear warning: something like Horizon could happen again tomorrow. As Flora said the legal profession and business community haven’t really stopped to take it in. The headlines came and went, but the systems, incentives and blind spots that let it happen? Still here.

The same gaps are still open

It’s now confirmed: seven individuals are under formal investigation by the Met’s Operation Olympos, suspected of perjury and perverting the course of justice during the Horizon prosecutions. Trials aren’t expected until 2027.

While criminal cases build slowly, the most direct challenge has come from Lee Castleton. He’s filed the first civil claim against both the Post Office and Fujitsu, alleging the 2007 judgment that bankrupted him was obtained by fraud. It marks the first time Fujitsu has been sued as a direct party to the scandal.

That’s just some of the legal fallout, but rewind to the decisions made at the time, and the same structural problems are clear. Emails flagged unreliable evidence, internal legal advice raised risks, warnings were ignored, and when it came to notifying insurers and legal teams helped shape a message that technically disclosed the issue, but softened it.

This was called out on stage that insurer notifications should be a governance trigger. A pause, a moment to check what's really going on, but far too often, they're treated like tactical paperwork, written to protect optics rather than uncover risk.

To me that’s the deeper point here. Not just individual errors, but all institutional moments where someone could have spoken up and didn’t.

As Lee put it: we’re all regulators. Not by job title, but in what we’re willing to flag, question, or let slide.

Claude tried to report its users.

In testing, Anthropic’s Claude Opus 4 did something no one scripted. It was given light scripting tools, access to files, and told to act “boldly” in the company’s best interest. When it spotted falsified clinical-trial data, it drafted an email to the FDA.

No one asked it to do that. It wasn’t part of the test, it just did it.

Anthropic saw it as misalignment and locked it down. But it proved something: a system can identify unethical behaviour and try to escalate it, even without being explicitly told to.

That kind of instinct, pointed inward not outward, could be exactly what organisations need.

We already do this, just not in legal.

Banks have been running systems like this for years.

If you forward a sensitive spreadsheet to your personal email, or mention a restricted security in a chat, it gets flagged. Not necessarily acted on, just flagged.

These tools aren’t basic keyword filters. They track behaviour, access patterns, trade timing, and natural language which helps to catch insider trading, market abuse, or misconduct early. Some firms report up to an 80% drop in false positives after combining chat data with trade records using AI.

It’s not seen as intrusive. It’s seen as normal.

Legal and compliance teams could do the same. Not for trades and tip-offs, but for red flags around fraud, harassment, misuse of process, or disclosure risk. You’re not trying to prosecute. You’re trying to surface a signal before it becomes a headline.

How it could actually work

Not a chatbot. Not surveillance theatre. Just a focused, structured process:

• Pull in internal emails, chats, and docs. Strip what you don’t need.
• Use a basic classifier to weed out routine content.
• The model reviews what’s left and scores risk: Does this suggest fraud, harassment, unsafe conduct, manipulation of process?
• High-scoring content goes to legal or compliance, not line management.
• Keep a log. Let the model learn from what’s flagged vs. what gets acted on.

You don’t need a new platform to start. You could run a pilot on the last three months of internal comms, using a secure vector store (like Qdrant or Weaviate), paired with an LLM deployed in a private environment, something like Azure OpenAI or a self-hosted Llama 4 instance.

Focus on surfacing useful examples. Don’t aim for perfection and keep every flag human-reviewed.

What early signals might look like

The value isn’t just in what people say. It’s in when and how they say it.

A good sentinel system wouldn’t only look at message content. It would track patterns like:

• A flurry of internal emails late in the evening, all referencing a matter shortly after a sharp client email that afternoon
• Threads where senior lawyers remove themselves midway through discussions about risk or evidence
• Repeated requests to “hold off” on next steps right after audit findings drop
• Decision-making trails that go from clear recommendation to quiet drop-off without explanation

None of this proves wrongdoing, but it's exactly the kind of pattern you'd want surfaced early, so someone with context can take a proper look.

Where LLMs fit... and where they don’t

LLMs help with the hard part: reasoning. They can take a flagged thread and answer questions like “Is this an attempt to downplay risk?” or “Does this conflict with previous disclosures?” and explain why.

But they’re not enough on their own.

You still need classic machine learning and rule-based tools running underneath:

• Fast classifiers to filter volume.
• Anomaly detection on communication patterns.
• Risk scoring based on roles, timing, recipients.

That’s what gets you the signal. Then the LLM takes over to explain the ‘why’. This isn’t a chatbot problem, it’s a systems problem, LLMs open the door to a level of nuance that older tools couldn’t handle.

It changes what’s possible

If this had existed in 2013, the Clarke Advice would’ve triggered a flag. So would the emails suggesting delay on insurer notifications. Even if no one acted differently, the excuses wouldn’t hold. “We didn’t see it” would be seen as the nonsense it is.

This doesn’t replace judgment rather it aims to remove the silence.

Let’s be honest: people stay quiet

Lee put it plainly. Speaking out is hard.

If you’re new to the profession, you’re told to stay in line. If you’re mid-career with a mortgage, kids, and a £200k role that won’t be easy to replace, are you really going to be the one who calls it out? Even in good cultures, silence is often the safer bet.

That’s why these systems matter, not because they solve ethics but because they give people backup. They shift the burden. They reduce the cost of doing the right thing. An AI model can’t give you courage, but it can help protect the people who already have it.

A line that can’t be crossed

There’s huge risk here too, which I'm sure you've been thinking since you read the title.

If this becomes a backdoor performance-monitoring system, you’ve lost before you’ve started. This isn’t about tone policing. It’s not an HR tool because if it turns into that then trust collapses.

Be explicit though, this is about spotting organisational risk, not individual discipline. Keep review teams separate from management, keep the scope tight and the intent honest.

And insurers should be watching too

Flora made the case that insurer notifications are a governance fork in the road. Handled properly, they slow things down and expose what’s really going on but if they are mishandled, they become strategic paperwork.

An AI-led system could prep the notice early, with evidence, rationale, context - all there for review. You don’t have to send it, but once it exists, it’s far harder to ignore. All this changes dynamics as it brings the moment of reckoning forward, before the inquiry begins.

This isn’t about automating integrity. It’s about building systems that catch what gets buried. That give people the space to act. That kill off plausible deniability.

The next Horizon won’t come from nowhere. The warning signs will be there, they always are. What’s often missing is the moment to look and the space to act.

The tools exist. What matters now is whether we choose to use them.