For the First Time, AI Beats Humans at Reading Comprehension
Marie Donlon | January 17, 2018
For decades, researchers have dreamed of training computer systems in language comprehension. Now, that dream could be one step closer to reality thanks to the efforts of Chinese tech company Alibaba and Microsoft.
Using Stanford University’s Question Answering Dataset, which includes 100,000 questions concerning 500 Wikipedia articles, the two systems were tested. The systems answered questions based on information gleaned from the paragraphs they were fed in the hope of being the first one to beat standard human performance.
Beating the current human score of 82.3, the Alibaba system earned 82.44 points and the Microsoft system earned 82.65 points.
“It is our great honor to witness the milestone where machines surpass humans in reading comprehension. That means objective questions such as ‘what causes rain’ can now be answered with high accuracy by machines,” said Luo Si, chief scientist of natural language processing at Alibaba.
However, experts caution that this achievement doesn’t mean that artificial intelligence can now read better than humans, thereby decreasing the need for human input.
Despite the scores, experts believe that the systems are still far from real comprehension because they were only fed information that was cleanly formatted and that contained actual answers and not text polluted with gibberish, forcing the systems to infer meaning.
Better bone up on my Jibberish....