Brian Babineau, September 2009
Over much of this decade, many organizations turned to open source solutions when developing an application requiring search or when adding search capabilities to an existing information repository. The rationale in using open source was straightforward: it was free and it provided "good enough" functionality, namely the ability to find a specific document through a web-like user experience. Entering a keyword and getting a list of results seemed like a reasonable way to address search requirements. However, ESG reports that developers soon realized that most open source search solutions, including Lucene and Lucene-based offerings, required some level of customization—an expense that was not considered when selecting this technology in the first place.
The limitations of traditional enterprise search—the need for users to have some idea of what they are looking for and where to look and the inability to execute queries across multiple sources-is driving the need for more advanced solutions. To streamline business processes and ensure all relevant information is made available for decision-making and service delivery, information must be automatically detected and delivered to not only knowledge workers, but to business processes as well.
It's been 18 months since the initial launch of Attivio, the enterprise search startup founded by former FAST Search & Transfer executives. Over the last year the company has been busy with proofs of concept and implementations, developing new features for customers that have driven enhancements in its new 1.5 release of the Active Intelligence Engine (AIE). Besides new modules for text analysis and optical character recognition (OCR), this version has more security features and querying options for structured data in order to make good on the company's promise of enabling customers to replace relational databases with its search engine.
"Since 2005, IDC surveys have shown the CIOs and CTOs want a single information access application that is capable of handling both data and content from a single, easy-to-use interface. Although a multitude of products all professing to solve this problem have recently appeared, creating such an architecture has proved to be a real challenge. Until recently, these products were better suited to handle either transactional data or textual content - not both. Traditional search engines are optimized to handle unstructured information; they lose the context that a database provides when they add raw data to a search index. Conversely, business intelligence (BI) systems have their own limitations because, although some offer limited text analytics, these tools are geared toward analyzing only transactional data and lose the context that the text provides - the reasons behind the data points.