As a law firm, we have seen first-hand the rise in clients using AI to better understand the legal challenges they face. That has not come as a surprise. Clients have always carried out their own research before seeking legal advice. Before AI, they used Google; and before Google, they turned to family and friends. Legal processes can be daunting, so it is natural to want a clearer sense of the position you are in before speaking to a solicitor.
However, AI is different because some people are now using it not only to understand a legal issue, but to decide what they should do, draft legal documents and formulate legal arguments. That is where the risk increases, especially when even qualified legal professionals have been caught out by false information generated by AI.
AI can help you understand legal information, but it should not be treated as legal advice. It can explain terminology, summarise processes and help you prepare questions, but it cannot reliably assess your individual circumstances, identify every relevant issue or apply professional judgement to the risks involved.
In this article, we explore how AI may be able to support you and where to draw the line.
Initial research
Clients have always researched the legal issues they face before speaking to a solicitor. Legal matters can be daunting and complex, and doing some initial research can help you understand the position you are in, your options and the type of support you may need.
AI can be a powerful tool for this. It can help explain unfamiliar legal terminology, summarise complex information and give you a starting point for understanding a legal process or issue. Used in this way, it can make legal information more accessible and help you arrive at an initial consultation better informed.
However, the risk of relying on information provided by AI was clearly demonstrated in two cases, Ayinde v London Borough of Haringey and Al-Haroun v Qatar National Bank, which were referred to the High Court after false information was put before the court following the use, or suspected use, of AI.
In the approved judgment, Dame Victoria Sharp, President of the King’s Bench Division, gave a clear warning:
“Freely available generative artificial intelligence tools, trained on a large language model such as ChatGPT are not capable of conducting reliable legal research.”
AI hallucinations
AI hallucinations are one of the most significant risks of using AI for legal advice. Google defines hallucinations as “incorrect or misleading results that AI models generate.” In a legal context, that can mean a tool producing inaccurate summaries, misstating the law or referring to cases and sources that do not exist.
From ChatGPT and Claude to Copilot and Gemini, you only have to look at the warnings displayed by the tools themselves to see why you should not rely on AI alone for legal advice:
- Copilot: “AI-generated content may be incorrect.”
- Claude: “Claude is AI and can make mistakes. Please double-check responses.”
- ChatGPT: “ChatGPT can make mistakes. Check important info.”
- Gemini: “Gemini is AI and can make mistakes, including about people.”
Those warnings matter because AI-generated mistakes can have real consequences in legal work. The Solicitors Regulation Authority (SRA) received 42 reports relating to the potential misuse of AI between July 2025 and July 2026 and has identified “AI-generated inaccuracies, or ‘hallucinations’ in legal research, advice, analysis and submissions to the court” as one of its key concerns.
Does this mean we should stop using AI altogether? No. It means we should recognise its limitations and not rely on it as a substitute for qualified legal advice.
In its Misuse of AI warning notice, the SRA explains the balance between opportunity and risk:
“AI can be a valuable tool and, when used appropriately, can support the delivery of legal services. However, it also presents risks that must be understood and managed. […] Appropriate human oversight, informed professional judgement and a proportionate, risk-based approach are essential”
AI sycophancy
Another risk is that AI may tell users what they want to hear. Current research suggests that some AI tools can be overly agreeable, even where a more careful or balanced response is needed.
A recent study by Stanford computer scientists found that AI “chatbots are overly agreeable when giving interpersonal advice, affirming users’ behavior even when harmful or illegal”. Whilst this study looked specifically at interpersonal advice, it raises important questions about the reliability of AI legal advice.
When you seek expert legal advice from a qualified professional, that advice should be based on your individual circumstances, professional judgement and duties owed to you, even if it is not what you want to hear at that moment. AI’s overly agreeable, or sycophantic, responses can be shaped by the information you provide and your own personal bias. This risks overlooking important counterarguments or reinforcing views that are incorrect.
In an article exploring AI sycophancy and its risks in the legal sector, the Oklahoma Bar Association stated:
“AI tools may omit key counterpoints unless specifically prompted, reinforcing biases and creating an echo chamber in which inaccurate assumptions are amplified.”
Expert oversight
These examples and warnings from leading experts and organisations within the legal sector point to one simple conclusion: there is no substitute for expert oversight from a qualified legal professional.
Whilst AI may radically change the way people research legal matters, and the way law firms and the wider legal sector operate, its output will still depend on the judgement of a qualified professional to determine whether it is accurate, relevant and appropriate to the circumstances.
There are parallels with free online will-writing services, which have been widely used since before AI emerged. These services can produce legal documents that may appear adequate on the surface and, in straightforward circumstances, may provide a workable solution. However, these simplistic online processes leave limited opportunity to explore individual circumstances, identify wider legal issues or ask the questions needed to ensure the document reflects what the person actually wants to achieve.
AI is much the same. It can provide information, suggest legal arguments or draft documents that look convincing to the person using them. However, it may not identify what information is missing, which facts are important or whether different arguments or provisions conflict with one another. Without expert oversight, those problems can easily be missed.
The dos and don’ts of AI legal advice
Dos
- Do use AI for initial research to help you understand an unfamiliar legal issue or process.
- Do use it to explain legal terminology or summarise complex information in plain English.
- Do use AI to help you understand legal documents or processes, particularly if it helps you feel more informed and confident. Your solicitor or legal professional will always be able to answer any questions you have, but AI can be useful for quick answers.
- Do use AI to help prepare for a solicitor meeting, for example by organising your thoughts or identifying questions to ask.
- Do seek qualified legal advice before making important decisions, advancing legal arguments or relying on AI-generated documents.
Don’ts
- Don’t rely on AI alone to determine your legal position or tell you what course of action to take.
- Don’t assume an answer is correct because it sounds convincing. AI can fabricate facts, cases, legislation and sources.
- Don’t rely on AI-generated legal documents or arguments without expert review, particularly where the consequences could be significant.
- Don’t overlook your own influence on the answer. The information and assumptions you provide can shape the response and reinforce your existing view.
- Don’t act independently once you have instructed a legal professional. Doing so could undermine the strategy they have developed or negatively affect your position.
- Don’t rely solely on AI to draft communications to your legal adviser. Whilst AI can be useful for organising your thoughts, sending AI-generated responses without checking them can introduce irrelevant questions or points, take discussions off track and potentially increase the time and legal costs involved.
AI can be a useful tool for understanding legal issues, but it should support, rather than replace, advice from a qualified legal professional. If you need advice you can trust, our team of experienced professionals will be happy to help.
