AI Analysis
for Legal Intake
Post-acquisition, a legal compliance firm struggled with bottlenecks. Our AI-driven automation boosted their intake by a staggering 88%
Our client is a successful healthcare-compliance law practice specializing in representing insurance carriers, governmental entities and self-insured businesses. Having carved out a rock-solid track record with over twenty years of practice, they had their sights set on scaling up which came with a unique set of challenges.
Key Takeaways
The Document Deluge: Navigating Legal Intake Challenges
To grow their practice, our client took stock of the bottlenecks across the various teams within the organization and started with the most limiting: the intake phase for new case files. Not only the greatest inhibitor to their growth, this phase was crucial in nature as it sets the stage for the entire downstream case their attorneys would work.
This step in their process involved gathering and analyzing enormous piles of unstructured documents for essential data critical to representing their clients' interests. The output of this stage impacts team assignment, informs on how complex the case is and therefore how many hours need to be billed. But it also crucially mitigates the risk of additional discovery later in the process, an event that could throw a wrench in any legal strategy attorneys will carve out once they've begun their analysis.
Paperwork Paralysis
Our client faced a formidable challenge with the flood of documents their intake operation teams needed to review. These documents, which come from various sources such as healthcare providers, insurers, and pharmacies are often a mess—unsorted, out of order, scattered and deceptively incomplete—which made finding the key details a slow and painful task. The average case source material for our client could easily span fifty separate digital files totalling over 400 pages of source material- not a trivial heap!
The documents are seldom straightforward to parse through, arriving in various formats like excel spreadsheets, PDF and MS-Word document. Some are straightforward and standardized, well-formatted letters from governmental agencies and others come in the form of handwritten healthcare provider forms- both essential to discovery.
Expertise Crunch
Our client's practice is about as nuanced as it is niche, requiring a deep understanding of the intersection of healthcare and compliance law in order to analyze documents for discovery. The analysis team is comprised of trained clinicians with years of tangible experience working in healthcare provider settings who transitioned into the practice of compliance. Having lived through the healthcare provider-patient experience for years they had developed an intuition for effective patient care and its implications on the legalities of insuring against the perils of workplace related injury and illness.
This isn't a readily available talent pool that can just be tapped into and training new personnel is optimisitcally a many-month endeavour. It was becoming clear that the prospect of training an AI model using the firm's prior work may yield a tool that could have a dramatic impact on their ability to handle client cases at scale.
The Devil’s In The Details
In working alongside and observing how our client’s process was designed it became clear that an attorney’s strategy for a case was only as successful as the result of the clinician analysis intake process. Specifically, once all the documents had been accounted for, analyzed and discovered- the clinicians would produce an executive summary which the attorney used to guide their approach.
Our client’s procedural playbook prescribed a very specific format for this summary that each analyst was trained to emulate. it was designed to maximize the amount of context and value attorneys could derive from it without being bogged down in non-essential details. This format is a proprietary trade secret and something the leadership had been perfecting since the firm’s inception.
Building a plane while flying it
Having cultivated the firm’s processes through decades of practice, the leadership knew that their process was a winning one. They also identified that they had to grow to reach their milestones but training junior clinical analysts to embrace and follow procedure was an uphill battle. Given the rather loose nature of this discovery process and the unique composition of each individual case, there wasn’t an obvious way to ensure the process was always followed. Reviewing each individual case simply wasn’t feasible either. Our client needed a way to standardize a process in a largely inconsistent playing field, a seeming insurmountable task.
Training an AI model to think like a clinical analyst
After living in our client’s world for a while and taking the time to grasp their unique growth challenge, we honed in an a solution that we felt could hit all of the key objectives of this scaling process head-on. Our recommendation was an AI model fine-tuned to work, think and produce just like our client’s most senior and seasoned clinical analysts.
Then, instead of beginning their intake process by digging themselves out of a heap of documents, a specialized AI model would analyze all of the documents in a fraction of the time it would take a human. It would split, sort, and index everything so that when a human analyst actually picks up the case file they have a solid overview of the contents to ground their understanding.
Our AI analyst would also write the first draft of the case summary using our client’s very specific proprietary formula so that when the human analyst gets to it, their job would become one of review instead of cold composition. This new workflow also offered the promise that each executive summary would be structured in the precise way that our client’s winning formula prescribed, setting each case up for success.
Let’s take a look at how we got from proposed concept to a completely transformed strategy primed for growth.
Teaching a new dog old tricks
While frontier AI models are great at analyzing any document and producing some kind of analysis of it, their general purpose nature puts them at a disadvantage for specialized analysis: exactly the kind our client needed. So we knew that using an off-the-shelf open-source model like LLama or DeepSeek was not likely to deliver value. But if we can show a model what a great analysis does and doesn’t look like, we can fine-tune it to specialize and perform like the ideal analyst. We tested this by asking our client to gather a golden sample of the best analysis produced on some of their most complex cases by their most skilled analysts. We also requested examples of the opposite, analysis that missed the mark. By using these datasets we performed a fine-tuning method that closely resembles unlikelihood training to produce a model primed for this specific use-case.
Separating signal from noise
No matter how agile and efficient our AI model performed, the quality of its analysis depended on its ability to understand the information it was processing. In light of the fact that a lot of the documents arrived out of order, in a variety of formats and incomplete, we needed a way to clean and assimilate all of it before our AI analyst went to work on it. To do this we employed a vision model that could read images, spreadsheets, emails and an assortment of other file formats and transcribe the contents in a manner that our LLM analyst could understand.
Once this phase is complete, we ran the entire set through our fine-tuned expert model to produce the index. The output of this stage is exactly that: a high-level outline of everything contained in the case-file. A glance at this output tells us what is and just as crucially what isn’t in the file and needs to be requested to complete the analysis.
Trust, But Verify
Just like when reviewing a colleague’s work to provide constructive feedback and looking for errors and omissions, reviewing the output of an AI model is no different. Given that our client’s analysts would begin with a first draft of the analysis, we wanted to make this function as seamless as possible. To do so we had our model produce citations so that the analyst can immediately jump to the context in the source material to understand how it arrived at its conclusion. This capability protected against the risk of the model hallucinating or misunderstanding something in the source material.
Preserving privacy and trust
Given the sensitive nature of the materials being analyzed by our model we had to carefully consider how our solution would remain compliant with the HIPAA and SOC2 policies governing how protected health information (PHI) is to be handled. It became clear that using an externally hosted LLM by AI providers like OpenAI, Anthropic or Groq was not an option as that would involve sending this data to untrusted third parties. In order to remain compliant we needed to self-host a model within our client’s intranet to minimize the risk of a data breach. Thanks to the vibrant and diverse landscape of open source models like Meta’s Llama and Qwen, we found no shortage of capable and cost effective models that fit the bill.
Leaps of faith
A prevalent challenge we encountered at each step of the process was one that we anticipated but not to the degree in which it presented. Specifically, we did not anticipate just how little faith our client’s analysts had in the possibility that an AI solution could deliver an output that would actually free up their time. The prevailing sentiment among them (and even some of the leadership) was that the AI would simply output slop that would make their job less efficient. It took a lot of trust building and running small experiments on sample data to build up trust that a model could learn from the golden cases we compiled.
Part of this trust-building exercise necessitated getting these analysts even more involved in the process and demonstrating that their role was not being subsumed bur rather transformed. No AI solution we could deliver had any chance of performing as well as their analysts unless they were part of the process. We had to demonstrate that the AI analyst was there to free up their time for more critical functions, not to replace them.
Measuring what matters
After several iterations of generating the clinical case analysis, having our client’s subject matter experts review it, gathering feedback and iterating some more, we had established a baseline performance that we felt was ready for production use. Our client began processing a subset of their new file intake with a team of clinical analysts utilizing our AI solution. They also maintained a control group of analysts using their normal workflow to draw a material comparison between the two. We were careful to control for variables like case complexity, skill level of the humans in the loop and the volume of cases that each group handled. With this setup we ran a month-long pilot to determine how effective a new process could be in removing impediments to growth, if at all.
The impact was immediate and transformative. First, let’s start with the quantitative:
Average analyst time spent overall per case
Cases fully analyzed before attorney review
The qualitative improvements are were equally as enticing.
From document drudgery to process optimization
Freed up from the seemingly never ending avalanche of intake documents, our client’s senior analysts were quickly and naturally evolving into a new role. By reviewing a number of the initial AI outputs they were able to identify ways to improve both the document collection and analysis, as well as the presentation of the final output. This function of influencing the behaviour of the AI became a force multiplier because a small percentage improvement in its output is compounded across every single case the firm works.
Consistency at scale
By having the AI write every case brief using our client’s proprietary, tested and true formula, we were able to maximize the probability that each case file an attorney saw would begin from solid footing. Teaching the AI how the best briefs were to be written effectively meant teaching every new analyst what great looks like. This also minimized the amount of training needed to grow the intake analysis team because the tooling would now guarantee a higher quality outcome.
Improved Case Preparation
The AI-driven process closed discovery gaps in 72% of cases before attorney review, up from 46%. This means attorneys now receive more complete case files from the outset, enabling them to develop stronger legal strategies without the disruption of missing information later. It’s a foundational advantage that enhances the quality of legal work downstream.
Compliant and Secure Operations
Designed to meet HIPAA and SOC2 standards, the AI solution ensures secure handling of sensitive health data within the firm’s intranet. This compliance not only mitigates risks but also reinforces client trust, a critical factor in the healthcare compliance industry where data security is non-negotiable.
Collaborative AI Integration
By involving analysts in the AI’s development and positioning it as a supportive tool rather than a replacement, the firm fostered a culture of trust and teamwork between human expertise and technology. This collaborative approach eased adoption and maximized the AI’s effectiveness, aligning it with the firm’s established practices.
Transparent Analysis for Verification
The AI’s inclusion of citations in its outputs allows analysts to quickly verify its analysis against source material. This transparency builds confidence in the AI’s work, streamlines the review process, and ensures accuracy—empowering analysts to focus on refinement rather than rework.
Closing thoughts
Deploying AI in a nuanced industry like healthcare compliance proved to be an iterative process and was made possible by relying on expert-level human reasoning at every step of the process. By taking the time to walk in the shoes of practitioners in the field and gain an understanding of the people who perform the work daily, we were able to gain perspective on what it might take to automate a process that previously only a person with years of experience could perform.
It wasn’t just our client that gained valuable experience and knowledge by going through this process, there were a few key takeaways for us also. First, the overall success of the project was made possible by high quality labelled datasets that we could use as ground truth for what is and isn’t a good result. Second, taking the time to work with and listen to the expert practitioners gave us the perspective to assemble a working mental model of the task we were trying to automate and served as the base framework from which we built the components of the final solution. Without both the quality data and the expert understanding, no progress in an AI solution is possible.
Empowered by this new process, what was once the most time consuming and limiting factor on our client's growth was entirely transformed into a powerhouse of scalability and continual process improvement. Our client's analysts went from being bogged down and overwhelmed by manual document drudgery to defining a process and framework that could be applied throughout the organization at every level.
🇨🇦 Built in Toronto