Canadian A.I. Company Achieves Highest Score to Date, Bringing Near-Human Level Comprehension of Natural Language to Our Machines

/EINPresswire.com/ -- MONTREAL, QC -- (Marketwired) -- 06/08/16 -- Maluuba, a Canadian deep-learning company helping machines think, reason and communicate with human-like intelligence, today announced it has scored the highest results to date on the premier machine comprehension benchmarks: Facebook's CBT (Children's Book Test) dataset and Deepmind's CNN dataset. Maluuba's model achieved leading scores of 67.4 percent and 74.0 percent accuracy on the respective datasets, surpassing top results from leaders in the academic Natural Language Processing (NLP) community including Facebook, Google and IBM Watson.

Machine comprehension is evaluated by posing a set of questions based on a text passage, then scoring a system's answers for accuracy. The CBT and CNN datasets are the new benchmarks for testing the role of memory and context in language processing and understanding. They consist of a set of texts and related questions drawn from those texts, and the correct answers.

The CBT dataset uses text from 20-sentence excerpts from children's books available through Project Gutenberg. Questions are formed by taking the 21st sentence of an excerpt and deleting one of its words, such that the sentence becomes a fill-in-the-blank-style question. Maluuba's model, called the EpiReader (based on the Philosopher Epicurus' principle of Multiple Explanations), achieved a leading score of 67.4 percent on the CBT dataset. The model answers a question by testing multiple hypotheses and selecting the most accurate answer based on context, mimicking how humans make context-based decisions every day. To put this in perspective, Facebook's CBT accuracy currently stands at 63 percent with IBM Watson at 63.4 percent.

The CNN dataset is built using articles from the CNN website. The articles themselves form the text, and questions are generated from short summary statements that accompany each article. In this test, Maluuba's EpiReader achieved a leading score of 74.0 percent. DeepMind's CNN accuracy to date is 63.8 percent with Facebook topping off at 66.8 percent and IBM Watson at 69.5 percent.

"These results mean that our technology can comprehend what is happening in text and start to form an understanding based on what was read," said Sam Pasupalak, cofounder and CEO of Maluuba. "Advancement on these datasets means that we are on track to helping machines understand and make sense of unstructured data, which opens up new frontiers for AI. This has always been our goal at Maluuba and our hope is that our results continue to spark progress in the industry."

Maluuba is working to solve fundamental problems in language understanding, with the aim of bringing us closer to machine literacy. This is a critical step in achieving true artificial intelligence. The company's research results on the CBT and CNN datasets demonstrate significant progress toward machine comprehension using a two-step, neural net-based model.

"Humans make predictions and sensible choices everyday based on context, but teaching machines to do this hasn't been accomplished quite yet. There are still hurdles for us to overcome in order for our machines to truly understand and interact with us as opposed to getting there by process of elimination," said Dr. Yoshua Bengio, professor at the University of Montreal and Maluuba advisor. "Maluuba's advancements in this space and their vision to teach machines true literacy is promising for the industry and consumers alike."

Read more about the EpiReader and its results here; and watch the EpiReader in action here. For more information on Maluuba, visit: www.maluuba.com.

About Maluuba
Founded in 2011, Maluuba is a Canadian A.I. company that has received over $9M in funding to develop the world's most advanced deep learning based language understanding platform in order to train machines to model decision-making capabilities of the human brain. Maluuba's machine literacy platform, utilizing breakthrough deep learning techniques in Question Answering and Conversational User Interfaces, is set to disrupt several billion dollar industries including Enterprise, Customer Service, E-Commerce and many more. Maluuba's engineering and customer operations are located in Waterloo, Ontario, with a research office in Montreal dedicated to solving fundamental problems in language understanding for innovative products that will further advance AI systems. For more information, visit: www.maluuba.com.

Contact Info:
Melissa Roxas
Inner Circle Labs for Maluuba
Email Contact
415-684-9401