Five years from now, computing science professor Randy Goebel hopes to turn the Artificial Intelligence program his team created into a legal tool for lawyers.
Goebel worked with a team of researchers and PhD students on an artificial intelligence program that participated in the Competition on Legal Information Extraction/Entailment (COLIEE) in Japan for the last five years.
At COLIEE, participants develop programs that attempt to answer questions based off of the Japanese bar exam. These questions give a scenario to the program that end with a yes or no question and the program must first retrieve the laws that apply to the case, then apply those selected laws to the question.
When the program first started, it could only achieve about 52 per cent in the competition, but this year, Goebel’s program scored 72 per cent, winning them first place.
“When it started out, some were actually doing worse than random,” he said.
Passing the bar
When Goebel appeared on CBC’s Radio Active to discuss his work, he said his AI program could pass a portion of the Japanese bar exam. But when CBC ran a follow up article, “Robot judges? Edmonton research crafting artificial intelligence for courts“, the story implied the program Goebel developed was capable of passing the entire exam, which isn’t entirely true.
Goebel explained there are two parts to the Japanese bar exam: a yes and no component, and another one which requires a written legal decision based off a sequence of events. The program developed by Goebel and his team could only do the yes and no part. So while the program could compete with people on that part of the bar exam, not being able to do the other parts means the program wouldn’t really be able to pass it.
“So they were being a little liberal. But in reality the program does better than most of the students, at least on that part,” Goebel said.
After the article was published, Goebel said he got a call from CTV asking if he could show one of his “robot judges” on TV. Goebel said they changed their minds when he told them it’s only software.
Practical applications of AI in law
Only now is Goebel working on a follow-up project that would put the program’s application to practical use. For Goebel, the challenge was developing a way to convert the information in the law into a language the program can understand so it could draw connections between a case and the law. Potentially this could result in a program in the next five years that can check for mistakes in a legal decision, Goebel said.
According to Dean of Law Paul Paton, the application of AI to law has been considered since the 1970s, but the possibilities have only been explored within the last few years. One example Paton mentioned is ROSS, an artificially intelligent legal researcher developed by a group of University of Toronto students who had access to Watson, the IBM-developed AI that’s known for its appearance in Jeopardy in 2011.
“Essentially what they’re doing is trying to come up with algorithms that can do in many respects what lawyers have been paid for years to do,” Paton said. “Which is to go through reams and reams of cases, do analysis, and try and provide clients with some sort of prediction,”
While such a project has potential, Goebel said he understands how the idea of computers replacing human judges would be concerning. But he says people should look at it as a tool, not a threat.
“If I was a judge, I would want such a system for me to use,” Goebel said. “Let the system help me ensure I’m not making some fundamental errors.”
While Paton said the potential for AI is “incredibly exciting,” he doesn’t believe it could apply the ethical considerations that human lawyers do in their daily practice.
“There is still a place for lawyers, but it’s a question of how these tools can be harnessed to help assist lawyers and their clients, but not replace them completely,” Paton said.
