What’s all the Fuss about the Challenges of Deep Learning?

The hype around Deep Learning is at an all-time high, but is it justified? While it excels at tasks such as image recognition and language translation, Deep Learning is not without its limitations. Earlier this year, Gary Marcus (http://garymarcus.com), scientist, author and CEO/Founder of Geometric Intelligence (recently acquired by Uber), discussed the 10 challenges for Deep Learning in this paper: https://arxiv.org/pdf/1801.00631.pdf. He then captured the responses and included his rebuttals in this article: https://medium.com/@GaryMarcus/in-defense-of-skepticism-about-deep-learning-6e8bfd5ae0f1. The conversations have been fascinating and are a strong demonstration of sectarianism of the Artificial Intelligence community.

At Kyndi™, we agree strongly with Gary that there is no one way to approach the challenges of Artificial Intelligence. Instead, there is huge value in fusing different Artificial Intelligence techniques to approach these challenges.

Below are different ways to approach some of the challenges outlined in Gary’s paper.

Challenge: Data hungriness.
Deep Learning often needs an impractically large amount of training data, often labeled data, to reach competence on a domain task.  Alternatively, by extracting structure from the data using knowledge representation and machine learning against that information, a system can learn from “small data” instead of “big data.”

Challenge: No natural way to deal with hierarchical structure.
By contrast, a fused approach represents hierarchical structure via logical structure and nesting within graphs.

Challenge: Struggles with open-ended inference, and not well integrated with prior knowledge.
Using Knowledge Representation and Reasoning and machine learning incorporates logical deductive reasoning and import of symbolic knowledge such as ontologies and thesauri.

Challenge: Not sufficiently transparent.
The “black box” problem is a hot topic in the AI world, with more thought being given towards making sure AI algorithms are explainable and unbiased. A fused approach enables explainability via incorporating symbolic methods for deduction, logical semantics of NLP, and provenance.

At Kyndi, we look forward to continuing the conversation. To quote Marvin Minsky, “You don’t understand anything until you learn it more than one way.” Let’s consider different and combined approaches in order to progress.

Subscribe to Kyndi's Blog