Last year my colleague Jonathan Young wrote a blog post Untangling the Semantic Web: Finding Threads of Gold, in which he noted that "Although the semantic web sounds like a panacea in theory, it does not have a great track record in practice."

At least one reader took this to mean we don't find the core problem interesting. Nothing could be further from the truth! A plurality of our customers and partners place great value on understanding and discovering entities and the relationships between them. Jonathan goes on to explain this later in his post - "...in designing [AIE] we picked a few of the golden threads from the semantic web and combined them with a number of techniques which have been shown to improve the search experience". Some of the relevant capabilities include:

• Dictionary and statistical named entity extraction in multiple languages

• Regular expression extraction of any pattern, including common natural language templates like "Sid Probstein is the CTO of Attivio"; you can combine named entity discovery with other search, e.g., find the term "Attivio" near a person entity, etc...

• Relationship modeling, e.g., friend-of-a-friend, using our query-side JOIN() operator.

We can also easily deal with many of the semantic web standards that are XML based - we speak XML "out of the box". For example it is easy to ingest content in RDF (Resource Description Framework), and emit search results (as well as extracted entities and relationships) as RDF triples.

OWL Web Ontology Framework is also an XML standard that we can make use of. Our automatic classifier can be trained to tag documents organized in most any structure - ontology, taxonomy or controlled vocabulary.

In the near future we are looking at implementing automated extraction of relationships. Initially this will be done by extracting and analyzing more generic patterns (e.g., Noun Phrase - Verb - Noun Phrase) as well as fuzzier approaches to templates (as noted above) and free text.

Fundamentally, our view is that we have an extremely elegant solution to the underlying problem of discovering, modeling and using entities and relationships. Triples are 'hot' because they are simple and easy for people (and legacy tools) to work with. AIE can handle much richer data structures, and by combining those with proximity and relational JOINs you can achieve truly remarkable insight into entities and how they relate to the rest of the universe.

TrackBack URI for this entry

Comments (2)