How to Map Automation Levels by Litigation Portfolio
A practical framework for choosing full automation, assisted review, or mandatory human validation across litigation portfolios.
Assess your portfolio
In this article8 sections
- Why automation levels should be mapped by portfolio
- The decision matrix for litigation automation levels
- How to score each sub-portfolio before automating
- When to use hybrid workflows in high-volume litigation
- How to calculate operational risk and AI triage cost
- How to run a pilot with different automation levels
- How to configure automation without losing legal control
- Common mistakes when choosing an automation level
Why automation levels should be mapped by portfolio
Choosing a single automation policy for every legal case is rarely safe or efficient. Litigation portfolios combine different courts, procedural events, deadlines, financial exposures, data quality levels, and business priorities. The right objective is not maximum automation. It is the highest safe level of automation for each type of work.
A litigation automation framework should classify work into three operating levels: full automation, assisted automation, and mandatory human review. Full automation handles predictable, low-risk events with clear downstream actions. Assisted automation prepares a result for a person to validate. Human review remains the default when an error could cause a material deadline, financial, regulatory, or reputational consequence.
This distinction is especially important for legal teams in Brazil that monitor publications across many courts and electronic diaries. The National Justice Council's Justice in Numbers publication illustrates the scale and diversity of the Brazilian court system, while day-to-day legal operations must still turn fragmented updates into specific actions.
Pocket Jurídico's experience monitoring more than 490 official diaries and courts supports a practical principle: the automation decision should be made at the sub-portfolio level, not only at the department or software level. A retail collection portfolio, a mass employment portfolio, and a strategic commercial dispute portfolio should not receive identical treatment.
The decision matrix for litigation automation levels
- ✓Full automation fits events with low ambiguity, repeatable classification, stable business rules, and a low cost of false negatives. Examples include routing a routine publication to a predefined team, tagging a known case type, creating a noncritical follow-up task, or synchronizing a validated status with an internal system.
- ✓Assisted automation is appropriate when software can reliably reduce reading and data-entry work but a professional should confirm the legal meaning or next action. Examples include summarizing a decision, extracting a possible deadline, identifying an appeal opportunity, or proposing an owner for a newly published event.
- ✓Mandatory human review is necessary when the event may start or alter a procedural deadline, change the company's legal position, involve an unusual order, affect a high-value dispute, or contain incomplete or conflicting information. Automation can still prepare the material, but it should not be the final control.
- ✓Type of process is only one variable. Add the cost of a missed event, the cost of an unnecessary escalation, the available deadline window, the reliability of the source feed, the complexity of the required action, and whether the result can be audited later.
- ✓Use a conservative default when evidence is limited. A portfolio can move from human review to assisted automation after measured performance, but reversing a missed deadline is usually impossible.
- ✓Do not confuse an accurate summary with an accurate legal conclusion. An AI-generated summary can accelerate triage, while the decision to waive, contest, appeal, pay, or negotiate may still require a qualified professional.
How to score each sub-portfolio before automating
- 1
Define the unit of analysis
Separate the portfolio by process type, business owner, court or jurisdiction, procedural stage, and required action. A useful unit might be mass employment claims in first instance, tax enforcement cases, or strategic commercial litigation, rather than the entire case inventory.
- 2
Score the cost of a false negative
A false negative occurs when the system fails to identify or appropriately escalate a relevant publication. Score the consequence from one to five, considering deadline loss, financial exposure, injunction risk, compliance impact, and damage to the client relationship.
- 3
Score the cost of a false positive
A false positive occurs when an irrelevant or low-priority event is escalated as important. Measure the professional hours consumed, unnecessary task creation, duplicate work, and alert fatigue. In high-volume portfolios, false positives can quietly eliminate the benefit of automation.
- 4
Measure the critical deadline window
Record how much usable time remains after a publication is received, validated, assigned, and reviewed. A short window favors human control and early delivery, while a stable and generous window may support assisted automation.
- 5
Assess feed and data coverage
Check whether the monitoring service captures the relevant diary, court, instance, case identifier, parties, documents, and historical movements. A sophisticated rule cannot compensate for missing source coverage or an unvalidated process match.
- 6
Assign the initial automation level
Use the combined score to designate full automation, assisted automation, or mandatory review. Document the reason, owner, fallback procedure, and conditions for changing the level.
When to use hybrid workflows in high-volume litigation
Hybrid workflows are often the best choice for mass litigation because volume and risk move in opposite directions. A department may receive thousands of similar employment or consumer case updates, yet a small percentage can contain an unusual order, a settlement opportunity, a new procedural phase, or a deadline that does not follow the normal pattern.
Consider a mass employment portfolio. Routine hearing notices and standardized procedural movements can be classified and routed automatically. A publication involving a new expert examination, a material calculation, a collective claim, or a court order with unusual wording should move to assisted review or mandatory validation.
Tax enforcement portfolios require a similar split. Repetitive updates can be monitored and organized automatically, but events involving asset restrictions, guarantees, payment negotiations, objections, or changes in liability should receive stronger controls. The legal team should define the trigger terms and event classes in advance, rather than relying on a generic urgency label.
Strategic commercial disputes usually deserve a more conservative model. Even when the number of cases is small, each publication may affect negotiation strategy, evidence, injunctions, contractual exposure, or executive reporting. Automation can gather the case history, summarize the publication, extract candidate actions, and send the item to the responsible lawyer, but final classification should normally remain human-led.
A practical portfolio split for a retail company might place 40% of routine, well-understood events into full automation, 45% into assisted review, and 15% into mandatory human validation. These percentages are starting hypotheses, not universal benchmarks. They must be confirmed through sampling and error measurement.
How to calculate operational risk and AI triage cost
The business case for automation should include both labor savings and control risk. A simple expected-loss model is: expected operational loss equals the probability of an error multiplied by the consequence of that error. Calculate it separately for missed relevant events and unnecessary escalations, because the two risks have different financial and operational effects.
For example, suppose a sub-portfolio receives 2,000 publications per month. Human triage takes six minutes per item, or 200 hours. If assisted automation reduces review time by 70%, the theoretical saving is 140 hours, but the department must subtract validation time, exception handling, implementation effort, and the expected cost of errors.
A useful monthly model is: net value equals hours saved multiplied by the fully loaded hourly cost, minus platform and integration costs, minus quality-control costs, minus expected loss from false negatives and false positives. The hourly cost should include salary, benefits, management time, and the opportunity cost of qualified legal professionals.
Do not estimate AI accuracy using only overall agreement. Track recall for relevant publications, precision for escalated items, deadline extraction accuracy, correct process matching, and the percentage of items requiring rework. A model that is 95% accurate overall may still be unsuitable if the remaining 5% contains most of the critical deadlines.
For AI use, establish confidence thresholds and escalation rules. The NIST AI Risk Management Framework provides a useful risk governance reference, although each legal department must adapt controls to its own procedures, contracts, and professional responsibilities.
Data protection also belongs in the calculation. Review access permissions, retention, processor arrangements, audit logs, and the handling of personal or sensitive data against the requirements of Brazil's General Data Protection Law. A cheaper workflow is not cheaper if it creates an unmanaged privacy or confidentiality exposure.
How to run a pilot with different automation levels
- 1
Select three representative sub-portfolios
Choose one repetitive, high-volume group, one medium-risk group, and one high-consequence group. For example, use routine retail claims, mass employment litigation, and strategic commercial disputes so the pilot tests different risk profiles.
- 2
Build a labeled baseline
Review a fixed sample of recent publications and label the correct event type, urgency, deadline, owner, and required action. Include difficult examples, duplicate publications, incomplete records, and items that should not generate a task.
- 3
Configure the feed by rules and permissions
Set separate routing, tags, alerts, due-date fields, and review requirements for each sub-portfolio. In Pocket Jurídico, the team can use the monitored names and terms, publication context, comments, and process history to organize work without creating parallel spreadsheets.
- 4
Connect AI as an assistant, not an invisible decision-maker
Use connected tools such as ChatGPT, Claude, or Gemini to summarize publications, extract candidate deadlines, and answer questions about the feed through an approved MCP or API setup. Keep the source publication, generated output, reviewer, and final decision together for traceability.
- 5
Prevent duplicate work in the ERP or CRM
Define which system owns each field and action before activating synchronization. For example, the monitoring platform can own publication capture and source context, while the legal management system owns the approved task, deadline, or matter status.
- 6
Compare results against the baseline
Measure time per publication, relevant-event recall, false-positive rate, deadline extraction accuracy, duplicate task rate, review completion time, and user adoption. Run the pilot for enough publication volume to capture exceptions, not only easy routine cases.
- 7
Expand only after a control review
Move a sub-portfolio to a higher automation level only when performance is stable and a fallback process is tested. Keep a sample-based human audit after rollout, with a named owner responsible for reviewing drift and new court patterns.
How to configure automation without losing legal control
The safest configuration separates capture, interpretation, approval, and execution. Capture should identify and validate the process and publication. Interpretation may include a summary or extracted deadline. Approval confirms the legal action. Execution updates the calendar, ERP, CRM, or case management system only after the appropriate approval level.
This separation reduces the chance that an incorrect AI interpretation automatically creates a binding internal task or closes a matter. It also makes troubleshooting easier because the team can see whether the problem occurred at source capture, classification, deadline extraction, human review, or system synchronization.
A useful control is the exception queue. Send uncertain matches, missing documents, conflicting dates, unfamiliar event types, and low-confidence outputs to a queue with a service-level target. The queue should be visible to supervisors and should not depend on an individual lawyer remembering to check another system.
Comments and internal context can also improve accountability. Keeping questions, decisions, and follow-up discussions alongside the publication creates a record of why an item was treated as routine, escalated, or closed. This is more reliable than distributing context through long email threads or separate spreadsheets.
Pocket Jurídico can support this operating model by delivering monitored publications and process context in a centralized feed, connecting with internal systems through APIs, and enabling AI-assisted summaries and deadline extraction. The objective is not to replace the legal team's judgment. It is to make the judgment happen earlier, with better context and less repetitive checking.
Common mistakes when choosing an automation level
- ✓Automating by department instead of by sub-portfolio. Different case types handled by the same team may have completely different deadline and consequence profiles.
- ✓Using an accuracy percentage without separating false negatives from false positives. Missing one critical publication can matter more than processing hundreds of routine items correctly.
- ✓Treating publication delivery as the same thing as legal triage. A platform may capture the source reliably, while interpretation and action still require a documented review step.
- ✓Allowing AI output to create deadlines automatically without validating the source text, procedural context, calendar convention, and responsible owner.
- ✓Measuring only hours saved. Include rework, alert fatigue, missed escalations, duplicate records, user adoption, and auditability in the business case.
- ✓Failing to revisit the model. New courts, new process types, changes in terminology, and shifts in internal staffing can make a previously safe automation rule unsuitable.
- ✓Launching without an operational fallback. Every automated path should have an owner, an exception queue, a review deadline, and a documented way to recover when a source or integration fails.
Frequently Asked Questions
What criteria should determine full automation, assisted automation, or human review?▼
Use the cost of a missed event, the cost of an unnecessary escalation, deadline sensitivity, source and process-match reliability, action complexity, and the need for legal judgment. Full automation is best for predictable, low-consequence events with clear rules. Assisted automation fits items where software can prepare a reliable draft but a professional should confirm the result. Mandatory human review applies when an error could affect a material deadline, financial exposure, legal strategy, or regulatory obligation.
How do I calculate the risk of delegating publication triage to AI?▼
Start by measuring false negatives, false positives, deadline extraction errors, process-matching errors, and rework. For each error category, estimate the probability and multiply it by the operational consequence, then compare the expected loss with labor savings and technology costs. Test the model on a labeled sample that includes unusual and high-risk publications. Overall accuracy alone is not sufficient because a small number of missed critical events may dominate the risk.
Which litigation portfolios require hybrid automation workflows?▼
Mass employment, consumer, tax, and collections portfolios often benefit from hybrid workflows because they combine high volume with meaningful exceptions. Routine events can be automatically captured, classified, and routed, while unusual orders, expert evidence, settlement developments, asset restrictions, and deadline-sensitive events receive human validation. Strategic commercial disputes also commonly require assisted automation because their lower volume does not reduce the consequence of an incorrect interpretation.
How can a legal department pilot different automation levels safely?▼
Select representative low, medium, and high-risk sub-portfolios, then create a labeled baseline of publications and correct actions. Configure separate rules and review requirements, run the workflows in parallel with the existing process for a defined sample, and compare recall, false positives, deadline accuracy, review time, and duplicate work. Expand only after the team has tested exceptions and documented a fallback process.
Can AI summarize court publications without eliminating human review?▼
Yes. Summarization is a strong assisted-automation use case when the original publication remains accessible and the output is clearly identified as machine-generated. The reviewer should confirm the relevant event, deadline, procedural context, and required action before the result becomes an approved task or update. The more consequential the decision, the more important it is to preserve the source, output, reviewer, and final decision together.
How do I avoid duplicating work between a monitoring platform and an ERP or CRM?▼
Define system ownership before connecting the API. The monitoring platform can remain the source for publication capture, process context, and incoming events, while the ERP, CRM, or legal management system owns approved tasks, financial data, or matter status. Use unique identifiers, synchronization rules, and exception handling so one publication does not create multiple tasks or require manual re-entry.
What metrics should be reviewed after implementing litigation automation?▼
Track relevant-event recall, false-positive rate, deadline extraction accuracy, process-match accuracy, time per publication, review completion time, duplicate task rate, exception volume, and user adoption. Also monitor whether the team is checking fewer diaries and courts manually without increasing missed escalations. Review metrics by sub-portfolio, since an acceptable result for routine retail claims may be inadequate for strategic disputes.