Finding Dots vs. Connecting Dots

In a recent blog entry posted at IDG Connect, I wrote about "connecting the dots", in this case recasting the dots as being the silos in which enterprise data remains heavily trapped - incompatible and difficult and costly to combine. Anyone who regurlarly reads this blog understands that I firmly believe that unified information access (UIA) offers real solutions to this challenge.

However, there is a lot more to say about connecting the dots. Some organizations view the problem differently; they can, for some job functions, teams or even departments, bring together an awful lot of information in one place, especially if it is all of the same or relatively similar type. Analysts who depend on published news, for example, probably worry less about the silo problem, because these silos are getting easier and easier to unify. A feed aggregator, more often than not, can provide a lot of value in this regard.

The challenge, in this case, then becomes one of finding the dots of interest in an ocean of dots.

Databases have long had an approach to this...write a better query! Many BI applications ask users to reduce the universe of possible answers very early on by focusing on a date range, specific source, or other "dimensions". The goal is to reduce the scope of the query so the user can more easily comb through the results. The problem of course, is that a reduction that's driven by dates may eliminate the interesting "dots", or, more problematically, that there simply are not enough dimensions defined.

In the UIA world, techniques like faceted browsing can be used to disambiguate any query using any structured data. For example if you search for "bond" it may offer you a choice of more specific noun-phrases such as "James Bond", "Bond Trading", "Chemical Bond" or even higher level categories like "Movies", "Finance" and "Chemistry". The advantage of faceted browsing is that it is dynamic — based on the data available — and interactive — as you click on a facet you reduce the result set and are presented with new relevant facets. Attivio's Active Intelligence Engine (AIE) actually recommends the best facets for each query, and can be used on a per-source basis to show facets that cut across silos, and those that drill into them.

Exposing useful information to allow the user to disambiguate a somewhat broad query is just the first step, however. It's also important to create additional metadata that can aid in this process. Identifying and fielding entities (such as people, locations or organizations) is an example of this. Analyzing the sentiment of documents, or entities within documents, is another. Being able to classify content to a particular set of subjects is another. All of these things create more facets that aid the user and help filter down a huge result set to a more modest and hopefully manageable one.

Another important step to help users identify dots of interest is to retain and use relationships between them. In a previous post on this topic I explained the difficulties of handling highly structured data in a search engine. Flattening information loses relationships. In the context of identifying interesting dots, however, relationships are extremely important. A "dot" may be interesting because they have performed certain transactions, are a member of particular group, or are related to another dot.

Using a UIA approach of combining relationships and content enrichment approaches, along with traditional dimensional analysis, brings a whole new set of possibilities to those seeking to "connect the dots".

TrackBack URI for this entry

Comments (0)