At CPIT in Brno on Why AI Deployments in Law Fail for Organisational Reasons
Why Law Firms Choose the Wrong Battle
When we talk about AI in law firms, the conversation usually turns to which language model is better, whose API is faster, which platform integrates with legacy systems. Jakub Jan Fiala calls this the wrong fight.
"Current language models are capable enough for a wide range of legal tasks," he said at the CPIT conference. "The failures we see aren't because the technology doesn't work. They're because the organisation hasn't defined what it's trying to do, or hasn't built the processes to turn outputs into value."
He came to Brno not to argue about models, but to name the patterns he has seen across law firms and internal legal teams—patterns that repeat so reliably they form a structure. That structure is a taxonomy of four classes of failure.
The Four Ways AI Deployments Unravel
Correspondence Failure: The firm acquires a tool before it has defined what problem it's solving. No baseline. No task definition. No metric for success. The tool sits unused or delivers outputs nobody knows how to evaluate.
Process Failure: The firm confirms the tool works in a test case, but never turns it into a safe, repeatable working procedure. Each time someone uses it, they improvise. Risk accumulates. Trust erodes.
Interaction Failure: The tool is available, but people can't, won't, or have no reason to use it in their daily work. A clever solution that nobody opens.
Expectation Failure: The firm and its clients expect different things from the tool. What is measurable value to one party registers as disappointment to the other.
These categories grew out of work by Kalle Lyytinen and Rudy Hirschheim on information systems failures (1987), but Fiala populated them with real observations from legal practice—situations where firms have spent money on AI, confirmed it works in principle, and still failed to capture value.
What Makes Legal Work Different
The law is not software engineering, and the costs of failure are not the same. An error in a contract costs money. An error in an internal tool often costs time. More importantly: in law, the output must remain traceable to the reasoning behind it. The client must be able to understand where a recommendation came from. And the lawyer, not the tool, carries the liability.
This changes everything about how AI should be deployed.
"Time savings in law are always relative to the cost of verifying the output," Fiala explained. "You save thirty minutes with a tool, but spend an hour checking what it produced. The real value isn't in the speed of the model—it's in a process that lets you trust the output without rewriting it."
That trust doesn't come from the model. It comes from an organisation that has thought through the task, prepared the data, built the procedure, and chosen a metric that everyone agrees on.
The Three Elements That Make It Real
In his closing remarks, Fiala distilled the insight into three interdependent parts:
- A suitable task. Narrow, defined, measurable. Not "improve contract review" but "flag indemnity clauses outside market standard for this client in this jurisdiction."
- A verifiable procedure. Not "throw it at the model and see what happens," but "model runs on this data, outputs go to this review step, flagged items follow this escalation path."
- An economic reason. The firm knows what it costs to do this work manually, what it will cost with the tool, and what the difference is worth.
If one of these three is missing, the firm has no way of knowing whether the tool is helping or not.
Why This Matters
AgiLawyer has worked with law firms and internal legal teams long enough to recognise these patterns. We've seen firms move too fast, acquiring tools before they understood what they wanted to automate. We've seen them succeed technically but fail organisationally—the tool works, but nobody uses it. We've seen them measure the wrong things and mistake noise for signal.
The conversation around AI in law needs to shift. It's not about finding a better model or a faster API. It's about building organisations that can actually make use of what the models already do well.
That means starting with the task, not the tool. Starting with a clear baseline, a repeatable process, and agreement on what success looks like. It means treating AI as an organisational decision, not a technology decision.
The CPIT conference in Brno brought together technologists, lawyers, and policy makers grappling with exactly this question. The work is hard. But the structure Fiala presented—and the evidence behind it—provides a map.
