Natural Language Processing

Remember the frustrations of key word search? If you need a refresher, just check out If Google Was a Guy. It’s funny—in a way that makes you kind of squirm in your seat. Of course, in the early days of search engines, power searchers used Boolean query operators. We have George Boole, 18^th-century English mathematician, to thank for that.

In fact, there are still Boolean purists out there who take Google, Bing and other modern search engines to task for no longer fully supporting Boolean queries. Nevertheless, natural language processing (NLP) evolved pretty much to relieve search users from the burden of having to memorize Boolean query operators and how to use them.

NLP and Text Analytics

One of the questions we often get from our customers is what’s the difference between natural language processing and text analytics? As we explain in a recent 5-minute Guide, there are a lot of similarities. The differences mainly lie in the problem that each tries to solve.

In the search world, NLP analyzes user inputs (queries) to understand their intent. It allows a user to communicate with a machine in a way that is natural for the user. To accomplish this, NLP operates on data so that a computer can understand a document—and the relationships it may infer—in the same way a user understands it. This is where NLP and text analytics use many of the same methods.

Text analytics uses a variety of techniques to extract information from text data and then classify, group, cluster, and mine that information for concepts and patterns as they’re expressed via key words and phrases.

Pattern recognition is where NLP and text analytics overlap quite a bit. NLP uses pattern recognition to parse a query. Text analytics uses pattern recognition to find answers to a query.

A straightforward way to think about NLP and text analytics is that text analytics happens as information is indexed, tagged, and entities extracted and annotated. NLP is an interaction that occurs when analyzing a user query and trying to match that to the work done by the text analytics engine. NLP is the human face of text analytics.

See NLP in Action

If you’d like to learn more about NLP and text analytics and how they operate in cognitive search applications, we’ve got a webinar coming up on Wednesday, April 26, at 2 p.m. EDT.

Brian Flynn and I will dig into the nuts and bolts of Attivio’s Cognitive Search Platform with a live demo. You can register here.