Big Data Wisdom Courtesy of Monty Python

One of our favorite parts of the hilarious 1975 King Arthur parody, Monty Python and the Holy Grail, is the “Bridge of Death” scene: If a knight answered the bridge keeper’s three questions, he could safely cross the bridge; if not, he would be catapulted into… the Gorge of Eternal Peril!

Unfortunately, that’s exactly what happened to most of King Arthur’s knights, who were either stumped by a surprise trivia question like, “What is the capital of Assyria?” – or responded too indecisively when asked, “What is your favorite color?"

Fortunately when King Arthur was asked, “What is the airspeed velocity of an unladen swallow?” he wisely sought further details: “What do you mean – an African or European swallow?” The stunned bridge keeper said, “I don’t know… AAAGH!” Breaking his own rule, the bridge keeper was thrown over the edge, freeing King Arthur to continue his quest for the Holy Grail.

Many organizations are on “Big Data Holy Grail” quests of their own, looking to deliver game-changing business analytics, only to find themselves in a “boil-the-ocean” Big Data project that “after 24 months of building… has no real value.” Unfortunately, many CIOs and BI Directors have rushed into hasty Hadoop implementations, fueled by a need to ‘respond’ to Big Data and ‘not fall behind.’

That’s just one of the troublesome findings from a recent InformationWeek article by Doug Henschen, Vague Goals Seed Big Data Failures. Henschen’s article cited a recent Infochimps Big Data survey that revealed 55% of big data projects don't get completed and that many others fall short of their objectives. The top reason for failed Big Data projects was “inaccurate scope”:

Lots of companies are blindly building out Hadoop clusters and collecting new data based on only a vague plan to open up that data store to multiple lines of business in 12 to 24 months…

Dreamers who haven't thought through their business priorities might think that Hadoop alone will be enough to deliver big data insight, but Infochimp's study suggests that that's not the case.

You might say that such CIOs and BI Directors on such an impulsive search for the “Big Data Holy Grail” will inevitably reach the “Bridge of Big Data Reality” (so to speak). And the bridge keeper will then ask, “What business problem are you trying to solve with the help of Big Data?”

Too many IT leaders will have to truthfully answer, “I don’t know… AAAGH!” – and regretfully find themselves in… the dreaded Gartner Big Data Trough of Disillusionment!

The correct response, of course, is to first understand essential details behind the question as King Arthur did. Infochimps CEO Jim Kaskade suggested to InformationWeek a very simple yet “practical and refreshing” question to ask:

Whether it's churn, anti-money-laundering, risk analysis, lead-generation, marketing spend optimization, cross-sell, up-sell, or supply chain analysis, ask yourself, ‘how many more data elements can you add with big data that can make your analysis more statistically accurate?’ (emphasis added)

The answer to this prudent question will lead to more smart questions:

• “What data sources, what types of data, are needed to fulfill my business case – structured data, unstructured data and/or unstructured content?”
• “How do I correlate structured and unstructured information together?”
• “How do I bring combined data and content into my existing BI infrastructure so users can analyze it using our data visualization tools?”

Big Data success simply cannot be realized without investing the time and effort to deeply understand your business; identify the particular business problem you are trying to solve, and understand the nature of all information your users will need. Doing so is critical to selecting the right technology for your Big Data infrastructure. Now you’re on the track to tackling the Big Data implementation infrastructure conundrum of wisely evaluating countless vendor products and making the right technology investments.

At Attivio, we help our customers cross that “Bridge of Big Data Reality” with ease. Attivio provides a highly agile, rapid time-to-value platform that offers a single point of access to discover and analyze relevant Big Data – including Hadoop – while integrating and correlating it with all other related, relevant data sources: structured data (databases), unstructured data (system logs, web click logs, sensor data, XML) and/or unstructured content (email, documents, social media, survey comments, CRM case notes) – all with no advance data modeling required.

Attivio provides this essential unified information access in conjunction with data acquisition, analysis and visualization tools such as Cloudera, Informatica, TIBCO Spotfire, Tableau and QlikView. We fully leverage these and other best-of-breed tools to get business problem-solving insight to your users in a cost-effective and timely way unlocking the massive value locked away in the variety and complexity of Big Data.

The final joke of Monty Python and the Holy Grail was that it was all a fool’s errand. Similarly, there is no one-size-fits-all Big Data “Holy Grail” technology out there… period. In reality, a successful Big Data architecture consists of multiple components to address the unique aspects of the data. Keep that in mind and show the wisdom of a king by taking pause and asking a few basic business questions to stay on the right path to business-building Big Data success.

Big Data & New Business Value Creation

This paper will explore this vital Big Data shift towards business value creation, and the new technology essential to realize that new value. You will discover why database and information management systems are not enough, and explore unified information access: a highly flexible, agile “Analyze Everything” platform well-suited to deliver new Big Data value – without ripping and replacing existing information infrastructure.

TrackBack URI for this entry

Comments (1)