What is Natural Language Understanding?

Natural Language Processing (NLP) is a key aspect of artificial intelligence (AI). Most commonly, NLP enables systems to understand natural language in documents (e.g. knowledge articles, tweets, emails) and thus bridges the gap between how people speak and what computers understand. To achieve this, NLP takes advantage of many cutting-edge computational technologies, which include machine learning (or statistically derived language models), speech recognition, natural language understanding, and natural language generation, to name just a few.

While all of these different areas of NLP research contribute to modern AI, with the rise of chatbots and conversational interfaces, Natural Language Understanding (NLU) has been receiving more attention in research and enterprise. This subfield of NLP focuses on comprehension of natural language. So what is NLU really? Among the different approaches to NLU, the most popular one currently relies on classification algorithms to classify inputs.

How Does Natural Language Understanding Work?

While there are a few different approaches to NLU, they share common components. As a subfield of NLP (read our earlier post, "What is natural language processing?"), NLU also relies on lexical and grammar rules to parse natural language. The parser, along with semantic theory of comprehension, guides the understanding of natural language. Once the initial language model is built, it needs to be adapted to actually understand the context. For example, a phrase such as “short sale” can have a very specific meaning in finance while “short sale” when referencing a process or a cycle, has a much less nefarious meaning. NLU models need finessing to be able to distinguish between two such utterances.

As chatbots and conversational interfaces are event more prevalent, it is important to mention that in chatbot speak, NLU is the engine that extracts the intent and the entity from a user’s utterance. Common NLU deployments essentially use machine-learning driven classifiers to quickly label new user utterances as a certain type of intent. While this is certainly useful, many chatbots fail in delivering the answers that match these intents and very often, conversational trees become incredibly complicated as a result.

Business Value of Natural Language Understanding

NLU can bring a lot of business value. Here's how it can impact four different vertices:

1. Machine Translation

Modern translation relies on more than just translating vocabulary directly. Commonplace slang and idioms make translation a complex problem, where understanding the context becomes in key in effective communication. Natural Language Processing can use neural machine translation to retain the meaning across languages.

2. Voice-First Technologies

With the advent of voice assistants such as Alexa and Siri, natural language understanding plays an essential role in taking action when a certain intent is recognized. For example, Alexa’s multitude of skills is only possible because of the advanced voice-to-text processing that enable Alexa to understand the voice input as text. Although it may be attractive to think about voice-first tech in the context of virtual assistants, voice-first technologies are much more pervasive than that. Voice-first tech can be used in call centers to detect fraudulent callers, improve customer service and even drive new sales opportunities.

3. Chatbots

Chatbots are now taking the internet by the storm and even though creating a powerful chatbot experience can be difficult, there are some clear winners in the industry that heavily utilizes natural language processing. The Facebook Messenger bot along with the Wit.AI acquisition are emerging as the leaders in the industry in engaging the B2C market, especially since the FB messenger interface is everywhere.

4. Conversational Search

Last, but certainly not least, is conversational search which often provides the backbone to the other three use cases. Conversational search allows end users to ask questions in natural language and then provide the most relevant result (public or non-public). Conversational search can be executed through a chatbot or voice-driven search, either as the main functionality or as a complementary/fallback service within the solution. To any business, enabling the end users to find the answers to their questions simply and quickly can translate to greater customer satisfaction and improve deflection.

In short, NLU brings a lot of varied business value; however, it is important to remember that NLU is only a subset of NLP capabilities, which are required to provide “smart” answers to “smart” questions. NLU only tells half of the story, or rather, it only asks the question, a smart search engine delivers the answer.