Do Chatbots Know NLP?

We recently posted a blog titled, What is Natural Language Understanding (NLU) that explained why NLU, a subset of Natural Language Processing (NLP), is getting more attention as businesses look to conversational interfaces (e.g. Alexa) and AI-powered chatbots to handle everything from customer support to sales to filing insurance claims.

As explained in the previous blog post, one of the features of NLU is the engine that extracts the intent and the entity from a user’s utterance. While finding the intent and the entity in the user’s mention is undeniably cool, understanding the user’s intent only gets you (or your chatbot) so far. What users are really looking for when using conversational interfaces are answers, which is where intelligent search engines such as Attivio come in. Unfortunately, most chatbot products focus on the NLU side of things—understanding the user’s intent—and omit the meat of the problem-- providing an actionable answer. 

The downside of using intent-driven chat interfaces is twofold. First, the dialog flows end up long and complicated—for example, just getting the opening hours of a business can become a multistep, programmatic process. Now imagine doing that for all the possible conversations that someone might have with your chatbot! Easily, a simple conversation tree becomes a convoluted forest that is hard to create and even more difficult to navigate. That’s a lot of time your marketing and sales team could spend on actually engaging with the clients themselves. 

The second downside is the lack of a proper fallback strategy. There is no chatbot that will be able to handle, let alone provide proper answers to all question—no human can do that either. In lieu of an omniscient AI (which might introduce other, more apocalyptic concerns), your dialogue interface will need to provide the “next best” solution if there is no pre-defined answer already. Today, most chatbots default to a simple “I don’t know that yet!” or continue to ask, “Can I help you with anything else?” Unfortunately, this type of interaction frustrates many users who would prefer a less “tailored” result instead. 

The problems described above are not necessarily issues with dialogue interfaces themselves; rather, these are data and search problems. A chatbot dialogue flow does not necessarily need to become complex if you have FAQs and a smart search engine that can use NLP to find the correct answer. The flow can become as simple as a one or two steps where the chatbot simply finds the right answer in the FAQ. Similarly, a proper fallback can be a search result from your website or other data, where the user can click on the relevant search result and find the answer on the most relevant page available. This is how a chatbot integration with Attivio addresses these issues directly:

1. FAQ Search – Using Attivio’s NLP-powered search engine, you can find the correct answers to your customers’ frequently asked questions. NLP is key here—being able to match up the right answer with the question, regardless of how it is asked, makes Attivio’s search engine the perfect fit for this type of integration.
2. Fallback Answer – Attivio’s index unifies data from many sources, so you can easily search your knowledge base articles, closed customer support request tickets, Github documentation, release notes and any other types of documentation in one place. Attivio’s search engine can then search across all this data and provide the most relevant document in milliseconds, so that your customer can address the issue as quickly as possible.

In short, NLU is a great tool for conversational interfaces; however, it is important to remember that NLU is only a subset of NLP capabilities, which are required to provide “smart” answers to “smart” questions. NLU only tells half of the story, or rather, it only asks the question, a smart search engine delivers the answer.