A new study, conducted by legal AI platform LawGeex in consultation with law professors from Stanford University, Duke University School of Law, and the University of Southern California, pitted twenty experienced lawyers against an AI trained to evaluate legal contracts.
The human lawyers achieved, on average, an 85 percent accuracy rate, while the AI achieved 95 percent accuracy. The AI also completed the task in 26 seconds, while the human lawyers took 92 minutes on average. The AI also achieved 100 percent accuracy in one contract, on which the highest-scoring human lawyer scored only 97 percent. In short, the human lawyers were trounced.
PeiQian (Vivian): What do you think the future challenge for lawyer’s workload? Will all the secretaries and entry-level attorneys disappear? How to keep competitive in the legal market? What’re your thoughts?
I wonder how lawyer professional conduct gets judged when it’s a robot doing the job. Can robots still be negligent? Does the liability shift onto the AI software company? this reminds me of Janson v Legal Zoom, where the Court held that Legal Zoom programmers were still interacting with clients, just behind a screen. So I wonder if AI negligence or even malpractice will become a new legal issue
Ya this is pretty wild. AI is taking over nearly every industry right now. It’ll be pretty cool when they have AI that can mine cryptocurrency. Ray Kurzweil thinks we’ll reverse engineer the human brain and have artificial super intelligence by 2029. Martine Rhotblatt thinks our generation will have cyber clones that allow us to achieve digital immortality. Elon Musk and Nick Bostrom think life as we know it is all an AI simulation. Max Tegmark and Larry Page seem to be relatively conservative and optimistic in their descriptions of humans and AI coexisting for generations to come. Ultimately, I think this is an issue that’s far beyond the control of legal or even governmental institutions because AI is moving far too fast to regulate.
Great post, Vivian. I do believe that we are still far from the day that AI will be able to replace human work as a whole. However, we are already getting in the age of needing to understand how those technologies work and how they can improve the quality of our work. Nathan’s perspective on the negligence is quite interesting and also raises the question whether AI will reach the point to be detached from any human responsibility.
Thanks for posting this Vivian. The numbers that you highlighted are striking but I agree with Marcelo that we are far from having AI replace humans as a whole. Furthermore, I think it’s hard to have AI replace human attorneys since it might implicate some issues such as malpractice and possibly violations of the Rules of Professional Responsibility. Since it all cases are unique and so fact specific, I also think it’ll be difficult to create an AI that will account for all situations that human attorneys might be able to account for.
I like how the article stresses that AI technology will most likely help lawyers by speeding up simple yet time consuming tasks, instead of replacing them completely. While these things may be able to evaluate routine contracts, lawyers are still needed for more specialized tasks. There’s a difference between reviewing boilerplate contract clauses and evaluating the merits of a unique lawsuit.