Navigating AI Citations, Canadian Court Sanctions, and Building a Bulletproof Workflow with Tom Macintosh Zheng

Welcome to studying Law Around the World. I'm Claudio Claus. In each episode, I talk with lawyers, law students, and professors from different parts of the world to talk about legal education, careers, and what the profession looks like in real life. We talk about the hard parts, the surprises. And the decisions that. Shaped their paths. Whether you're planning to study abroad, thinking about working in another legal system, or just curious about how law works around the world, this podcast is for you. Today I'm interviewing on Macintosh Zeng. He is returning to the podcast for some new exciting updates. We first talked about court ready, I believe last year and now we have some very, very interesting topics to to be discussing in this episode. Thank you so much for making the time to be here, Don. Well, thank you very much for having me, as always, glad to be back. Amazing. And she she started podcast. I always appreciate if you can just introduce yourself briefly for maybe listeners who are listening to these episode before our previous ones or who are kind of first coming to the podcast. Yeah, absolutely. So my name is Tom McIntosh Zane. I'm the co-founder of Court ready.ca, where we built practical tools to help Canadians navigate the legal system and from time to time we conduct studies into access to justice issues in Canada. I used to be a commercial barrister in practicing in Toronto and I'm also qualified as a solicitor in English, England and Wales. Amazing, fantastic. And you kind of get right on the topic of our conversation today, talking a little bit about court ready and a little bit about, you know, the whole world of technology, AI and so forth. It was interesting that in our first conversation we talked about how there were so many different tools that could be used in legal practice that actually didn't even need AI. They were simpler tools, but they just could really complement the practice. Ever since then, lots, lots has have have happened and everyone you know from junior lawyers, self represented litigants and, and even members of the judiciary are learning about generative AI and maybe using it to some extent to save time, to save fees and, and to get an understanding of different things. But there are some problems with that, right? So we did learn that there is made-up citations, some kinds of hallucinations. And specifically the research you conducted found a great deal, I think over 200 made-up citations that affected over 100 cases or 100 decisions. So I, I wanted to hear a little bit about that about, you know, your, your research how that came about and maybe even are those the result of AI that was used in the past and now AI is better and, you know, doesn't hallucinate as much anymore. What's your view on all of that? Right, So to start with, many of us know that when we use the publicly available AI tools, sometimes the tool has a tendency to hallucinate. So what I mean by that is you ask it a question and it would provide the answer and the answer sometimes is incorrect. That happens also in the field of law, right? So as a as an experiment myself, sometimes I would ask it to give me the answer to a legal question. Now for simple ones, it has the tendency to get the answer correct. But say if you asked any of these tools that do not have access to a law database, what are the top five cases in Canada on a particular topic when you ask that question and you get the answer, I've seen that it might give me cases that don't exist. It might give me cases that has a real name, but the substance of the case is completely different. Or it would give me quotes from from a case, but then those quotes actually do not exist in that case itself. So then I asked myself the question is, has this particular issue of AI hallucination impacting our courts and tribunals because we know that a lot of folks who may not have the means to afford a lawyer or who may not be comfortable writing in legal English and French turn to these AI tools to help them write and help them research us. Other than say a small disclaimer at the bottom of every tool say you know, blah blah blah can make mistakes. The tools don't really prompt you that pay. Wait a second, These cases may not be real. So for someone without a legal background, when they looked at, when they see a confident case citation from one of these AI tools, they think that it is real. So as the context for why we did this research. And what was very surprising to me is that since 2024, there have been 36 court cases on tribunals where the court or the tribunal flagged that a party submissions contain fictitious citations, 136 decisions. So that's where the court caught people submitting cases that don't exist. There are separately 18 other cases where the court warned a party about AI misuse. But looking at the 136, they are all over the country. They're in 45 courts and tribunals. And in total, there are at least 247 fictitious, fictitious citations that were submitted across the 136 decisions. And the trend is accelerating. We had about 7 or 8 instances IN20002480 cases in 2025. And in the first quarter of 2026 alone, there were 50 cases. So naturally the question is these were the instances that were caught by the courts and tribunal. But that means statistically, and again, I am speculating here a little bit, there's there ought to be instances where these cases made into an actual decision unnoticed by everyone along the way. Now, we actually have seen the first instance of this being reported where a decision contained fictitious citations is a case out in Quebec. Obviously we don't know who was ultimately responsible, but that was the first at least reported case of a decision containing AI hallucinated citations that no one was able to stop before it was released. Well, thank you so much for for explaining and and going over that. I find it interesting that that you see this jump and and I'm always wondering different, different things. So I wonder if actually when the citation and and the decision maybe came about, probably those were written quite a quite a while before that happened, right? So I mean, imagining the faction was written probably even a year before actually a decision came about. I'm curious to understand as well as as as far as you know, what the tools goes. And I really appreciate you mentioning this is specifically that, you know, the the database mentions, you know, the the court citations where judges actually noticed and flagged in, but then there could be things slipping under the radar, the radar. So what's really happening behind the scenes with all of these fake cases? So you're right in that there is a delay factor, right? Because sometimes you would write a factum or any submissions and that's gets submitted and of course decision isn't released a while later. So even though these decisions are coming on 2025 and 2026, we are probably looking at the submissions that were happening six months to a year ago. And so then the next question is, is it going to get worse or is it going to get better? And, and that's what we're trying to figure out whether or not because the tools themselves are getting better that we work. We, we hope to see that this trend decreasing, although from what we've observed, there's just more and more decisions coming out. And also because of the way that we're conducting our research, which is primarily relies on reported decision. As a lawyer myself, I know often the decisions are not reported or published or made their way to county. So we might just be looking at the tip of the iceberg. Here, very, very interesting. Well, thank you for for sharing those details. So specifically speaking on legal citations, and I think this might be important even for people who are outside of Canada. I understand that our Canadian legal citations, well, maybe in the US to some extent, but they follow very strict predictable format. So you've shown that that that makes sense and makes it easy for AI to kind of mimic that structure, right. So why does that structure sometimes will trick people or catch, you know, lawyers off guard? Yeah. So that's a great question. I think for the folks who may not be as familiar with the Canadian system as you are. We obviously like the other common law jurisdictions, rely on cases as president. So if a judge has decided a case in a certain way and you have a similar case coming up next, you would want to cite the results or the law as it was interpreted in the prior case to support your argument. Now, Canadian case citations all tend to follow what's called a neutral citation pattern, which is party V versus party B, comma, the year, the court abbreviation and then a naturally increasing integer. So a case could be you know from V Jerry 2021 Glen SC1234. So you know just by that it is a two parties Tom and Jerry and the case and whatever the year and OSC stands for Ontario Superior Court. This system makes it easy for an AI to mimic. And So what I mean by that is we, we tend to use sort of moral terms to describe, you know, lying or making things up. AI, as many of us know it predicts what's the most likely answer based on the prompts that you give. So let's use an example. If you ask an AI what is the, the, the most important case in Ontario that talks about spousal support in the context of family law, you ask that question, if the AI does not have access to a database, it's going to try to figure out what is the most likely answer to that based on patterns. So it's a family case, right? And so in Canada, the most common last name is Smith. So Smith V Smith is the most likely answer for an Ontario case. The chances are it's going to be out of the Superior Court. So OSC is the the, the court abbreviation. So Smith V Smith 2017 OSC 135. Then that's how the AI would produce a case that really looks real but isn't because it is predicting the most likely outcome based on the prompts that you give. And you know, the second part of your question is why does this structure trick people and why does it cache so many junior lawyers? Off guard is when I used to practice, we were always, it was important to cite as many cases as possible to support your argument, have a legal position. And then in your footnote, you want, you might have 5 or 6 different cases. So, you know, the Ontario courts have always ruled in this way. See comma, for example, case 123-4567. There was a time where you could trust that those cases are real. Now they may not actually stand for the arguments the other side claims that if they stand for, but you can trust that these cases actually exist. That's no longer the case anymore because even though it will look real and it can even trick lawyers because of the fact that it follows A neutral citation pattern, they could be actual cases. And one way I can demonstrate it is that I'm going to, you know, show my screen just for a little bit, but to see what some of these fictitious citations look like. So you see here, I'm going to zoom in just a little bit. This is on court ready.ca where you can see this fictitious citations in Canadian courts. So what we have on the left hand side, actually let's do this quiz. Which one of these is a fake citation? Which one, Claudio, you pick? Well, I would say the one that repeats the same last name twice just just for just for that case. But maybe the the lack of the period after V makes me doubt my on. So let's let's go with Viper. Viper. OK, Piper V Piper was a real case and Brown V Swanson is a fake case, right? And again, to me, like if I didn't know British Columbia, BC S is the Supreme Court of British Columbia out in the western part of Canada and let unless I check, I would have not known that one of them is fake. So let's do one more round. Which one of these is a fake, fake citation? Claudio, your your turn. Let's go with Wallace V Crates Marine Sales. Already they're both fake, right? So this again, like they both look very real and I'm pretty sure there's like a similar case that's that's about Valgardson. Alright, so the final round, let's see you pick. Yeah, I mean, it's just a fun guessing game here, but let's go with Alessandra V Briggs. OK, so they're both real, right? And those were actually two of my cases that I did when I used to practice. So this goes to show that you can't really tell if if lawyers can't tell, then self represented litigants who may not even know about how cases work or how case citations working in Canada, they want to be able to tell judges and clerks again, they're so busy already and they rely on lawyers or self reps providing cases that are real. And we almost take it for granted, like, frankly speaking, you know, some of this research, this wasn't an issue before 2020. Four, no, very interesting. I, I also see and I think all of the legal community to some extent have been following the difference, you know, decisions about that. A lot of them, you know, end up making it to the to the headlines. It seems like in the beginning and in in a few different jurisdictions, parts were kind of giving a warning at 1st and then now we have documented consequences or that, you know, costs, whether that be the case, gets referred to the regulator. So for lawyers watching this unfold, what are some of these maybe let's call it an unwritten rules since I guess we haven't codified anything like that, that judges are following when they see a I misuse. I think there's a there's a possibility of using some kind of precedent saying that you're maybe misguiding the courts or, or your other person. But I'm curious, you know, how the this risk landscape changes in the profession when we see these this new phenomena happening? Well, the risk landscape changes dramatically because the courts and tribunals are getting pretty fed up. And you are very much accurate in that in the beginning stages, the courts were almost, well, they're not almost they are giving benefit of the doubt that those who are providing these citations do so in good faith and they're not misleading the court. And secondly, what I think was happening at the time is that many people didn't even know that these are AI generated citations. So I'm just kind of positive for one second to, to, to discuss this is that in our research, we found cases which we didn't put in the database that in my own experience, I could tell that the cases were either hallucinated by AI or had AI's hand in, in drafting. But those while those citations were picked up by the judge or the tribunal member, the judge and tribunal didn't really go the full way by saying you use AI or, or got into the weeds on how and why these cases came into your submissions to begin with. So we are still seeing some instances where the judge doesn't really know or didn't really identify this being an AI related issue. But to your point about sanctions, well we have separately published a database of sanctions and and this is the show how the courts have reacted to this this issue. And so I'm going to share my screen again. This is also part of our court ready.ca database is on our homepage is like how Canadian courts and tribunals are responding to AI hallucinated case law. I miss you. So I'll give you some of the top line data, right. So in in this case we have about 42 + 170 a 147 total decisions that includes some of the AI misuse from our separate database. And sometimes you know the different parties appear multiple times, we group them into one just to not to inflate the numbers artificially. So the 147 decisions where the court had an opportunity to sanction 31%, no record of sanctions record just said, OK, I recognize that you've had fictitious citations in your submissions. I'm not going to consider them, right? 49% warnings. So the warnings will look like, hey, look, you're doing this, you have to look at our practice direction and you cannot do it again kind of thing or threatening of further sanctions of the issue repeats itself. 20%. So 30 cases, serious sanctions, when I say serious, they include imposing costs, regulatory referrals, contempt of court or this is not an interesting one where if you're the successful party, you are normally entitled to cost. So other side paying your legal fees, But the court said no because you used AI and you mislead the court in some way. So we see that as well. And when you look at trends, right? So this is the second graph. OK. So so you look at first of all an increase in the number of decisions dealing with this issue, right. So we had you know from one in the first quarter of 2024 and to let's see this is yeah, almost 50 in the first quarter of 2026. And you see how you're seeing more serious sanctions and you're seeing more warnings. And was also interesting about the breakdown is that as a as expected, right, As a proportion of the total number of sanctions, those with represented parties, legals, lawyers and paralegals tend to have a higher percentage of serious sanctions. 46% of represented parties have serious sanctions where that proportion is only 15%. And that makes sense. Self represented individuals, they're not represented course tend to expect more from represented parties. So if you look at what kinds of decision, right. So here we have a, a tabulation of costs. So thus far the courts have ordered at least $30,000 Canadian of sanctions for improper AI use. And yeah, so if you look at the serious sanctions going down here, but the most significant example came out of Alberta, $17,550. And then next in Q5000, right? So we have a, you can have a, a breakdown of the cost here as well. So all this is to say. Is that forts are more willing to impose serious sanctions in this issue and of course judges have discretion in this regard. So we haven't seen sort of a nationwide standard in, in the penalties that courts tend to give. But the, the, the ultimate message with, to lawyers and self represented folks is that even by looking at the judicial language from 2024 to now, you can tell that the courts are in general annoyed at and, and exhausted by dealing with this issue because they're also have to spent time and, and, and, and their own resources trying to find cases that they can find anywhere. And we see this happening, some of this decisions where the judge said, you know, we're not supposed to be doing this, going to the judicial library to look for cases that frankly don't exist. Right, right, right. No amazing. Thank you for sharing that. And and I understand that somewhat recently there was a US decision regarding privilege and generative AI, especially when it comes to having information in, in one of these, as you said, like publicly available, like free tools, right. So I, as far as I know, there's not such a, an official recognition in Canada about privilege and AI or like the kind of tool you're using. But I'm curious, you know, overall in this, in this situation, I think one of the, the things you showed is that over 80% of the affected decisions do involve self represented parties. And they're turning to AI because it's often, you know, affordable. We, we talked about, you know, the language and then just how popular all of the AI tools have become. Maybe what, what's the system getting wrong about this gap? Are there mistakes in your personal opinion? You know, what are some of the mistakes maybe that courts are making by simply because the technology hasn't been explained properly? Right. And, and the explanation part requires those in courts and tribunals to provide this judicial education to self represented litigants. Because this part is important. Is not that self represented folks are more reckless, is not that they're more willing to, to lie or miss around the law is because in most cases they don't have the means to afford a lawyer or it doesn't make sense to it. Because you have a lot of these tribunals where you're the issue in dispute is, you know, $4000. So they, they can only run high. Well, not can only, but they, they, they, they, they use a tool that is free to help them. And the tool sounds confident. It would be helpful if the course and tribunals would do more to educate the folks who are coming and say look, if you rely on AI to draft and research, this is the consequence and this is the issue because no one wants to lose in a court battle. So if an AI is giving you law that doesn't exist, the chances are you will have less of an opportunity to win the argument. And this is where the see the gap is first is on judicial education for the parties. And second, there ought to be a very quick and easy way to verify if the cases you cite are real. And that's what caused us to build case check, which is a semi automated citation checking tool where you you, you run your argument, you run your submissions through the tool ones. So at least you know that once it makes it to the courts, your cases are authentic. And more importantly, we don't want fake cases to become real. Long Canada. Amazing. No, thank you for sharing that. I'm curious here, you know, if through the, through this entire conversation, you, you mentioned in the beginning right away that there was this, this situation that really wasn't a problem that you could rely on somebody else's memo and all of that. So if we were to hypothesize here, if a junior associate receives a memo full of AI generated content, they have this risk of fabricated case law, right? Or even if it's, you know, in a precedent that didn't get caught in time. And what exact question should they ask should be protecting themselves And, and what are some of the verification steps that could be helping to make sure that this never happens again? I think that most of us will probably always take a look at can leave these cases have actually exist. But I think there is something that you mentioned that is interesting that it, it's not only about the citation itself, but also the context, right? Because sometimes it will be quoting a case where where that is not the case at all. So, so can you tell us a little more about that? Yeah, that's right. So what we're also not now seeing is that is not just the fact that AI tools are hallucinated cases that don't exist. They're making up quotes from cases that that that do exist, as that's one example. So when a junior associate receives any sort of legal position or legal submissions from from anyone, I think the responsibility is on them to check and check thoroughly because there is no way to tell just by looking at it that this is real or not. And that's the scary part. You know, when, when, when the first iteration of these tools came up, it was kind of easy for someone with a legal education to tell. So for example, I know that the Ontario Court, Superior Court doesn't handle, you know, 100,000 decisions a year. So if a decision has the neutral citation number of 2021 OSC 123456. And obviously I know that that cannot be real, but that's no longer the case. These tools are getting better at what I call mimicking reality. So the only solution for the lawyers is to check. And again, there's also this regulatory concern, right? So what we see is that courts and tribunals are asking people to verify or certify that every authority is real. So, so right now, as we see in the system, it relies on the person submitting the legal submissions that the cases you submit are real and they stand for what you claim they stand for. And that responsibility is mostly going to fall on the junior associates, the clerks and the assistants is is about who's the last person verifying everything. And if you're the last person doing it is on you to get it. Right. Yeah. Thank you for bringing that up. I think you should go to the end of our conversation today. I wanted to ask you specifically about the tool that you've built that that you know, have this idea of helping catch this hallucinated citations before they reached the courtroom. Looking at everything you found, what is some maybe the single most important part of our relationship with legal tech that you want people to see? That folks were using any kind of tools have to be a bit more skeptical with the information it is producing, right? And especially when it comes to legal research. Now there are some wonderful tools out there that do have access to a database of law. Those ones, because they're connected with the database, they can actually go and fetch the information and give it back to you. But to date, the, the publicly available tools that people use that I see appear in courts and they sometimes they mention the name of specific tool they don't have access to counting. So how are they getting the information? And, and I don't think a lot of folks, you know, self represented litigants know that they have to be more skeptical because ultimately they're going to be penalized by the courts, not the tools that they use. Now, of course, we can have a whole separate conversation on the extent to which these tools ought to be held responsible for confidently producing things that are wrong. But for now, it is the person using it who bears the ultimate responsibility and burden when it comes to using it for court slash legal purposes. Fantastic. Well, thank you. Thanks so much for for coming on the podcast and and sharing this this this details As for, you know, people who are interested maybe being in finding the two and reading more about the research. Can you just share with us so we can be, you know, easily found in the video. I'll make sure it's also in our show. Notes, that's right. So yeah, thank you for that. So all of the research that there and the tools are on our website, courtready.ca and is right on our home page. So if you go here now, you can see the research for both the sanctions and the fictitious citations and case check as well. And of course, we have our other tools that helps folks navigate the legal system. Fantastic. Thank you once again, and thank you all for tuning into today's episode. Thank you and have a great day ahead.