Enhancing Justice System Data Through Structured Settlement Reporting
A proposal to support the safe and effective use of AI tools in courts and tribunals.
1. Executive summary
AI is arriving in courts and tribunals much faster than our data infrastructure is maturing to support it. At the same time, access‑to‑justice problems persist: unrepresented and low‑resource parties still struggle to understand what “normal” outcomes look like or how long disputes typically take to resolve.
The single biggest missing dataset in most common‑law civil justice systems is structured information about settlements. In New South Wales (and in most comparable jurisdictions), the overwhelming majority of civil and tribunal matters settle before final hearing. Yet the terms of settlement are neither filed with the court nor captured in any structured, analysable form. Courts therefore see only the small fraction of disputes that proceed to judgment, leaving a large blind spot in understanding:
- real‑world dispute outcomes
- systemic inequities and power imbalances
- repeat‑player behaviours
- cost drivers and access‑to‑justice barriers
- settlement norms and dispute‑resolution pathways
That blind spot does not just undermine the safety and fairness of AI tools; it also deprives future court users (especially self‑represented litigants) of the basic information they need to make informed decisions about whether to file, how to negotiate, and when to settle.
As governments and courts experiment with AI‑enabled systems, this absence becomes risky. Any AI that predicts outcomes, recommends dispute‑resolution pathways, or benchmarks “typical” results will, if trained only on litigated outcomes, misrepresent the justice system, amplify existing biases, and entrench inequity.
This paper outlines a policy option for Structured Settlement Metadata (SSM): a lightweight, confidential, privacy‑safe mechanism for capturing key settlement metadata without disclosing the full terms. The recommendation is not to mandate filing of full settlement deeds into the public record, but to introduce a standardised settlement outcome report, submitted automatically when filed proceedings are resolved by consent or discontinuance.
2. AI‑enabled justice without settlement data is flying blind
Modern justice‑sector AI tools rely heavily on:
training data
structured case metadata
outcome distributions
patterns across large datasets
triage and risk‑flagging models
guided dispute‑resolution pathways
But the data courts actually hold is heavily skewed.
Courts reliably see:
pleadings and interlocutory steps
some procedural decisions
final judgments (for a small minority of cases)
Courts generally do not see:
settlement amounts (even in banded form)
non‑financial remedies and behavioural commitments
concessions, apologies and systemic issue indicators
power‑imbalance markers and representation status at settlement
reasons for early resolution or repeat‑dispute risk factors
user sentiment or perceived fairness of the outcome
The result is obvious. Any AI trained only on judgments will:
over‑represent high‑conflict, high‑value or atypical disputes
under‑represent low‑resource parties, language minorities and self‑represented litigants
distort the statistical baseline from which digital pathways and risk models are designed
risk recommending sub‑optimal or unfair resolutions based on skewed precedent
If we are serious about building outcome‑predictive or pathway‑recommending tools that are accurate and representative, good settlement data is close to a precondition.
3. Why not simply require filing of full settlement terms?
It is tempting to say: “If courts need settlement data, just make parties file the deed.” In some contexts, courts already require scrutiny of settlement terms – for example in class actions, certain employment or consumer cases, and matters involving minors or unrepresented litigants – to ensure fairness before discontinuance.
But extending a blanket requirement to file full settlement deeds across the civil and tribunal landscape would cut across how settlement practice currently works in several important ways.
3.1 Confidentiality and candour
Confidentiality is not a decorative feature of settlement; it is a core part of the bargain. The law of without‑prejudice privilege and mediation confidentiality exists for a reason: to encourage candid negotiations and realistic offers by protecting parties against their own settlement communications being used against them later.
If every settlement deed filed in a court became part of the public record or was widely accessible, parties and advisers would quite rationally become more cautious in what they committed to paper, particularly in reputation‑sensitive disputes. That does not mean “all settlement would stop”, but it is a credible, system‑wide risk that warrants respect, not hand‑waving.repository.law.
3.2 Sensitivity and power
Settlement deeds often contain:
detailed employment and health histories
financial vulnerability information
allegations without findings
commercially sensitive pricing and trade information
In many disputes, the idea that all of this should now sit on a court file – and potentially be subject to access requests – will be unacceptable to at least one side. Sophisticated repeat players will adapt. Vulnerable parties and their advisers may simply decide they are better off never commencing formal proceedings at all.
3.3 Scope, burden and necessity
There is also a basic practicality problem. Filing millions of settlement agreements per year, across every list, is administratively unrealistic for courts and tribunals already under pressure.
And it is not necessary. We can get most of the value we need for AI safety, system design and policy‑making from anonymised, banded metadata about settlements, without dragging the full deed into the court file.
4. A better option: Structured Settlement Metadata (SSM)
The alternative is a lightweight, mandatory settlement outcome form lodged whenever filed proceedings are resolved by consent or discontinuance.
4.1 Key principles
SSM is designed to be:
non‑intrusive: no disclosure of full terms
respectful of confidentiality: no privileged content, no negotiation history
fast and simple: a short form integrated into existing digital filing portals
standardised: consistent fields across courts and tribunals
privacy‑protecting: anonymised and non‑identifying at source
interoperable: suitable for linkage with national justice and access‑to‑justice datasets
Crucially, SSM is statistical reporting, not evidence. It records high‑level outcome descriptors after resolution, not the content of negotiations.
4.2 What the form captures
For each filed dispute, the SSM form would capture specific non‑identifying fields such as:
nature of dispute (pre‑defined categories aligned with court taxonomies)
issues in dispute
representation status of each party at the point of settlement
settlement modality (mediation, conciliation, direct negotiation, online process)
stage of settlement (pre‑filing, post‑filing pre‑listing, post‑listing pre‑hearing, mid‑hearing)
remedy type (financial; non‑financial; behavioural commitments)
settlement sum band (for example, 0–$5k, $5k–$20k, $20k–$100k, >$100k; calibrated per list)
outcome indicator (resolved fully, resolved in part, withdrawn, referred elsewhere)
repeat‑player involvement (for example, insurer, landlord agency, employer group)
optional fairness/satisfaction indicator provided by parties
optional systemic‑issue flag (for clusters of similar complaints)
The form does not capture:
admissions of liability
personal histories or detailed narratives
commercial confidentiality or trade secrets
legal argument or negotiation content
privileged material
4.3 Relationship to existing doctrine
Because SSM does not intrude into negotiation communications, admissions or detailed terms, it can and should be placed on a clear statutory footing as a protected statistical reporting obligation.
It should be explicit that:
SSM data is not admissible as evidence in the underlying proceedings
SSM cannot be used to attack or infer the content of negotiations
SSM is collected solely for system‑improvement, evaluation and AI‑safety purposes, not for enforcement or commercial exploitation
This approach respects the underlying rationale of settlement‑related privileges, while still allowing courts and governments to understand – in aggregate – how disputes are actually resolved.aboutrsi+1
5. Behavioural effects: when the requirement applies
An important design choice is scope. The SSM requirement would apply only to disputes that have been formally filed in a court or tribunal.
If parties are genuinely uncomfortable with any form of settlement reporting, there is an obvious workaround: settle earlier, before issuing proceedings. In practice, the introduction of SSM is likely to:
move the “high point” of settlement propensity earlier in the life of a dispute, and
make the decision to file more meaningful, because it carries with it a responsibility to contribute to the shared knowledge base about how similar disputes resolve
If the worst thing that can be said about SSM is that it nudges parties towards resolving disputes before they draw on public resources, that is a trade‑off most justice systems should be prepared to entertain
6. Where to start: small‑claims consumer and debt matters
Not every category of litigation is equal when it comes to controversy around settlement reporting. A sensible place to pilot SSM, and to test its impact on behaviour and costs, is high‑volume, low‑complexity small‑claims consumer and debt lists.
These matters share some features:
Standardised, repetitive issues (small credit debts, personal loans, simple consumer contracts).
A high proportion of late settlements, often on or near the hearing date.
Repeat‑player creditors on one side and low‑resource individuals on the other.
Lower privacy sensitivity (in banded, anonymised form) than, for example, discrimination, defamation, family or personal injury cases.
In that setting, SSM enables a simple but powerful set of tools:
early‑stage outcome banding (“most similar matters settle in these bands over these timeframes”)
guided negotiation pathways that encourage resolution before listing
registry and judicial dashboards highlighting matters that historically tend to settle late, for targeted early case management
If those tools shift even a modest share of “day‑of‑hearing” settlements into the pre‑listing or early post‑filing window, the financial and operational case almost makes itself:
fewer vacated hearing days
lower sunk preparation and appearance costs for parties
freed‑up judicial time for the genuinely contested and complex disputes that require it
The same logic can then be adapted, cautiously, to other lists where settlement is common and metadata is less sensitive.
7. Access‑to‑justice and unrepresented parties
Structured settlement metadata is not just an AI‑safety or court‑efficiency play. It also creates concrete access‑to‑justice benefits, particularly for people who arrive at court without representation.
Right now, unrepresented litigants and first‑time users have almost no reliable way to understand what “normal” looks like for disputes like theirs. They see a few published judgments, hear war stories from lawyers (if they ever speak to one), and are otherwise flying blind. SSM changes that. Once settlement patterns are captured in standardised, anonymised form, courts, tribunals and trusted intermediaries can surface simple, legible signals: the kinds of outcomes people usually achieve, the stages at which similar matters typically resolve, and the settlement sum bands that are realistically in play.
That information can be fed back to users in accessible ways – plain‑language guides, online calculators, guided pathways, or even simple “what usually happens in cases like yours” dashboards at the point of filing. The aim is not to dictate outcomes, but to reset expectations away from fantasy wins and towards realistic, evidence‑informed resolutions. For many disputes, that will translate directly into earlier agreements, fewer unnecessary hearings, and less time, money and emotional energy spent on avoidable litigation.
There is a fairness dimension as well. Repeat players (insurers, large creditors, institutional landlords) already have internal data on their own settlement patterns. SSM levels the informational playing field by allowing courts and public institutions to generate aggregate insights that are equally available to everyone, not just the best‑resourced actors. In that sense, a settlement metadata standard is an access‑to‑justice reform as much as it is a data‑governance reform.
8. Governance, privacy and linkage
None of this works without a robust governance framework. SSM should sit within a Justice Data Standard with features akin to those used in health or child‑protection datasets.
Key elements include:
anonymisation at source, with no direct identifiers in the SSM record
clear, legislated limits on use: system‑improvement, evaluation, AI safety
aggregation and suppression rules to manage small‑cell re‑identification risk, particularly in small jurisdictions or niche case types
controlled access for research and AI‑development partners under strict conditions, not free public browsing
Linkage to other justice and social datasets is where much of the value lies – but it is also where the re‑identification risk increases. That risk should be confronted honestly, and addressed through governance, rather than used as a reason to remain in the dark.
9. Implementation: from pilot to infrastructure
A realistic implementation pathway might have three stages.
Stage 1 – Design (around 6 months)
Develop a common taxonomy and the SSM data standard.
Draft the settlement outcome form and embedding rules in court/tribunal procedure.
Establish a justice data governance board (judiciary, government, independent experts).
Map settlement pathways across key courts and tribunals.
Plan integration with digital filing and case‑management systems.
Stage 2 – Pilot (around 12 months)
Begin with one or two low‑controversy, high‑volume jurisdictions (for example, small‑claims consumer/debt matters in a tribunal or local court list).
Collect SSM, deploy basic predictive/pathway tools, and measure:
the proportion of matters settling pre‑listing vs post‑listing vs day‑of‑hearing
hearing days saved and party costs avoided
user experience, perceived fairness and administrative burden
Stage 3 – Scale (2–3 years, contingent on readiness and resourcing)
Expand SSM across civil courts and tribunals as digital infrastructure allows.
Embed SSM as a standard component of e‑filing and online court pathways.
Link SSM with national access‑to‑justice and civil justice datasets.
Use SSM‑driven analytics to train and audit AI tools used in the justice sector.
Timeframes will ultimately depend on digital readiness, legislative support, funding and judicial leadership. But the direction of travel is clear: AI is coming, and it is better to shape its data foundations now than to retrofit safeguards after deployment.
10. Note on sources and influences
This proposal builds on long‑running work on civil justice administrative data and empirical research on settlements. In the United States, for example, judicial settlement databases have collected confidential summary data from settlement conferences to help judges understand settlement ranges and patterns in their own courts.
Empirical work on settlement rates and “party resolution” in federal district courts, and on specific domains such as employment discrimination, highlights both the centrality of settlement and the difficulty of capturing its outcomes in a structured way. On the private side, legal analytics and AI providers are already trying to incorporate “historical settlements” into their models, typically from fragmented and proprietary data sources.
This paper takes those strands and pushes them one step further: proposing a justice‑system‑level settlement metadata standard, tied explicitly to AI safety, fairness and system design.