Rethinking the Foundations: Four Emerging AI Architectures That Could Reshape Legal Tech

Legal teams have spent the past couple of years experimenting with generative AI. Most of what’s being used in legal AI today like summarising agreements, extracting obligations, flagging unusual clauses, all still depend on Transformer-based models. They’ve proved super useful, but come with familiar constraints: short memory, occasional hallucinations, and a tendency to generate surface-level answers that still need close human review.

While most of the legal world is trying to work within those limits, researchers at leading AI labs are quietly working on something else. Not better prompts or bigger context windows, but entirely new architectures that aim to replace Transformers altogether. These aren’t features you’ll find in next quarter’s release notes, they are a deeper rethink of what AI is and where it could go.

Here’s a look at four developments that are beginning to shape those conversations and what they might mean for legal tech if they gain traction.

1. Sub-Quadratic Architectures and the End of Context Limits

Transformers struggle with memory. As inputs grow longer, they become expensive to process. That’s why even the best models today still lose track of what was said earlier in the conversation, or why uploading a disclosure bundle often means splitting it into manageable chunks.

A new wave of models is trying to solve that by making memory part of the system, rather than an afterthought. They don’t try to retain everything, just what’s likely to matter. That change alone allows for far longer context windows, without requiring a retrieval workaround or constant summarising.

For legal tech, this could bring:

• Persistent AI assistants that stay with a matter over time, not just within a session
• More natural workflows that don’t rely on engineered context injection
• Tools that remember prior decisions, strategy changes, or client-specific preferences

Memory limits have been a core constraint. If they go away, a lot of tooling will need to be rethought.

2. JEPA and Conceptual Reasoning Beyond Text

Most LLMs reason by generating words. It works well enough for surface-level tasks, but breaks down in more complex scenarios, the kind that involve interpreting a clause, weighing outcomes, or holding multiple concepts in tension.

JEPA, an architecture being explored by Yann LeCun and others, doesn’t rely on constant narration. It allows models to process ideas internally without having to phrase them out loud, only producing text once something is ready to be expressed. It’s closer to how people think through problems quietly before responding.

So if this approach matures, it could end up bringing support for:

• Clause analysis tools that stop midway if the logic doesn’t hold, rather than bluffing their way through
• Models that can map multiple interpretations of a contract term before settling on one, without rambling through them in output
• Legal research systems that reason conceptually across jurisdictions, rather than looking for the closest matching text

It’s less about flashier output and more about what happens before anything is actually said.

3. Absolute Zero and Self-Evolving Reasoning

Fine-tuning legal models is expensive, it requires annotated examples, not just of what the answer is but how to arrive at it and why it matters. Most firms aren’t sitting on that kind of data, because it's time consuming and has no immediate ROI.

Absolute Zero is a training method that skips the human-labelled data entirely. It lets the model generate its own problems and learn by solving them, the same way we see with kids, and this kind of self-play has worked well in games and now it’s being adapted for reasoning tasks.

What this could mean in practice:

• In-house copilots that refine their approach to risk, compliance, or matter triage based on how they’re corrected in daily use
• Domain-specific tools for pensions, tax, or environmental law, built without needing a labelled dataset upfront, where
• AI systems that improve at drafting fallback clauses or pushing negotiation boundaries through simulated deal termsIt’s still early, and there are risks around reliability, but it could change the cost-benefit equation for building domain-specific legal tools.

4. World Models and Multimodal Understanding

Legal work rarely comes in one format. You’ve got contracts, charts, call transcripts, scanned exhibits, and tables which are often all part of the same matter. Most AI tools still treat those as separate silos.

World models aim to break that down. They take inspiration from how people process input, not by modality, but by the meaning. Whether a piece of information arrives as text, image, or sound, the model then figures out how it fits.

A few places this could be used:

• Analyse scanned deeds, tabular data, and clause logic in one unified pass without format conversions
• Combine email trails, attachments, and call summaries to flag inconsistencies or regulatory exposure
• Build workflows where AI doesn’t just parse documents but builds a real mental model of the matter, like the parties, positions and risk points

All this is already being explored in labs, and the question is how quickly it becomes accessible, and affordable for real-world legal workflows.

None of these approaches are ready to deploy in production legal systems. They’re still in the early stages, some are theoretical, others are only just starting to show results in adjacent domains like robotics, image generation, or open-ended reasoning tasks.

Obviously, there’s also no guarantee they’ll translate cleanly to legal contexts. The standards for accuracy, interpretability, and accountability in law are different. A model that works in a consumer setting may well not meet the bar in a regulatory one (as we've seen with ChatGPT case law...).

That said, these ideas aren’t just blue-sky research, they point to what’s likely coming and what current tools are really missing. If models start to retain long-term memory, or reason without spelling every thought out in text, or combine modalities in one unified workflow, it changes how legal tools should be designed. It affects what kinds of oversight are needed. It shapes what becomes possible, and what becomes unnecessary.

It’s also important to say that none of this makes current LLMs obsolete. Much like traditional machine learning still plays a valuable role in fraud detection, classification, and document tagging, LLMs will continue to underpin many of the tools legal teams rely on. They remain well-suited for drafting, summarisation, and structured generation tasks where the text is the product.

To me what’s more likely is a blend of approaches. A persistent memory layer here, a reasoning engine there, a language model acting as the interface. Each model type doing what it’s good at, integrated into a single workflow.

The point isn't to chase every breakthrough. It’s to stay oriented. To design tooling that doesn’t fall apart the moment something more capable comes along and to avoid getting locked into assumptions that may no longer hold next year.

Source for all of this: