Content, or soft information, has always been of interest to the business in a wide range of processes, from marketing to executive decision-making. The explosion in volume and variety of soft information driven, in particular, by the Internet has sharpened that interest. However, with years of experience in business intelligence and data warehousing behind them, many users are clear that what they really need is an integrated view of soft information with the harder data already available in the warehouse. While soft information on its own does have value, the real business advantage will come from exploring the entire set of hard and soft information free from the limitations of the pervasive, predefined data structures of hard information.

Content and data are closely related. Data is what IT has made of content in order to control and process it in the structured world of computers. Content as simple as “I’ll buy that red car” is transformed into a purchase transaction, with defined fields, allowed value ranges and keys normalized in a database. The use of two distinct words, “data” and “content” is unfortunate, since both are the same concept — information. Content is softer information, while data is harder; two terms at opposite ends of a continuum. At the softer end, information exists as commonly used and interpreted by humans — documents, images, etc. Hard information is the structured records and fields suitable for logical and numerical computer processing, for example, in operational systems.

Conceptually, soft information is the original source of all hard information. In designing operational databases, hard information is defined up-front through a person modeling soft information for computer use. Simply put, modeling separates the meaning (what is an order) and relation-ships of terms (price and quantity as part of an order) from the values (ten items at $100 each) they may take in a particular instance. Meaning and relationships can also be distilled from soft information on the fly — during ingestion into a content store or even during use of the content — using text mining and analytic tooling that essentially automates the same modeling process.

Perhaps because of its more formal structure, data is often assumed to be more accurate and reliable than content. The aim of a “single version of the truth” is widespread in data warehousing. In reality, both assumptions are misleading. Reliability and accuracy of information depend solely on its source, and the format doesn’t affect the quality of the information. Some sources are simply more or less dependable than others. Like accuracy and reliability, truth is also a relative term, as any reading of eye-witness reports can confirm. Resetting these erroneous beliefs is vital, especially for data warehouse experts, as we bring data and content together.

Harder information exists today in the regimented databases of operational systems, data ware-houses, and so on. Softer information is found in a wide variety of content stores from the “world wild west” of the Web and social media to well-managed stores of e-mails, documents, call center logs, etc. in enterprises. To meet the demand to provide access to all relevant information — regardless of its source or form — technology leaders need to look for methods that unite information without losing either the relationships so valued in the database realm or the context and nuances so important in content.