What AIE and unified information access mean for developers

There has been a lot of press recently on unified information access and how it enables business users and IT staff to reduce the time it takes to provide information to make better business decisions. The Active Intelligence Engine not only provides value to business users, there are also a number of advantages for developers that make life much easier:

Index now, think later

One of the greatest advantages of AIE is the ability to facet and join on fields without having to do a lot of preprocessing or, more importantly, design work. For example, you might not know ahead of time that your customer database records are linkable to customer comments on your website, but you can easily find out with a single query after both information sources are indexed. A number of POCs and development spikes we have conducted have followed the pattern of indexing everything possible and then trying to infer relationships using queries. Unlike a database where primary and foreign keys must be setup ahead of time, the index does not require this sort of predefined and rigid schema definition. Also, many of these features are so cheap to leave on, that tuning isn't always necessary.

That's not to say that there aren't advantages to doing some tuning of the schema or the ingestion workflows, but it isn't required in order to start seeing the power and ROI of using our system.

Develop locally, deploy globally

Developing for a single node, single JVM system is a straightforward process for most any platform. Some platforms also make it fairly easy to write business logic for a large distributed system. The key advantage we've found is being able to develop and test in a small localized environment, but then deploy to a large distributed system and not be surprised by system behaviors. In addition, it's important to be able to use a standard debugger when developing locally in a fully functional system, but then also to be able to use the same debugger in a distributed environment. AIE topology files provide an abstraction that separates the system functionality from the system deployment. This allows operations teams the ability to scale the system for QA, staging and production environments without having to worry about functional issues with the configuration.

Some other systems like Hadoop force users into somewhat complex development models. AIE's development models strive to support the "I want to do X to Y" in the simplest possible manner. We ship a sample transformer that implements some simple business logic and more importantly, we ship a unit test for the transformer.

Learn one API, let us handle the details

One of the hardest parts of building enterprise wide applications is the need to work with multiple different APIs. In addition, each system has its own idea of what a user is, what it means to have permissions to read a document and more importantly, what a document is to begin with. If you can't define and normalize all of these concepts it's impossible to join, group, categorize and make decisions based on the data. AIE not only provides connectors to these back-end systems, we also handle normalizing each system's data to a standard format that is accessible via our API. A user in Active Directory can have permissions to a document in Documentum and a document in SharePoint. More importantly, the permissions are applied transparently at search time so that developers don't have to worry about doing any sort of post filtering of results.

Attivio's development environment is meant to hide all of the enterprise ugliness from developers and present a single user, group, document, acl, and query concept. If userX can see records in 10 repositories, we handle the details. If you want to join data from your internal SharePoint server to your CRM system based on a support person's contact information, we can do that for you as well.

Author Bio

Since graduating from MIT with a degree in Computer Science, Will Johnson has worked for Altavista and FAST for over 7 years. At Altavista Will developed AV's real time indexing solution used by news aggregators who demanded instantaneous access to news as it arrived. In addition he was one of two engineers responsible for developing the Altavista QIndexer product that was used by the large majority of AV's customers. At FAST, Will developed high speed database connectors as well as developing search UI's and tool sets used across the organization. In addition Will also worked on many of the largest and most complex sales engagements and deployments for customers around the world, specializing in distributed systems for many of the largest internet publishers, directories as well as internal knowledge management systems. Will is a founder, one of the Chief Architects at Attivio and a really nice guy.

TrackBack URI for this entry

Comments (0)