If your organization is going to win on analytics, it needs to view all of its information as a strategic enterprise asset. This includes not just the 10% you know about, but the 90% of dark data that hides in information silos. There are big challenges on the path to surfacing all of your enterprise information for business intelligence. The biggest challenge is not in storing data, or in analyzing it, but actually finding the right data. But why is it so hard? Here are the top three reasons:
Forrester just released its latest Wave report. Unlike many on more mature technologies, this report on native Hadoop BI platforms included only six vendors, of which Attivio was one. In Forrester's estimation, there are no leaders in the market, just contenders and strong performers.
We talk a lot of about the amount of data organizations capture and how that data comes in many sizes and formats, from structured to semi-structured and even unstructured. We also know that much of that data isn’t used for decision-making, hidden in silos across the organization. All of this makes it difficult to build a unified view of your data.
But the challenge with building a unified view is only partly due to data silos.
As Dan Woods points out in a recent article for Forbes, technology marketplaces cycle through predictable stages as they mature. He applies this insight to the component versus platform decision that organizations face when adopting new technologies.
Data democratization is about giving a larger group of people in the company access to self-service tools to find and work with the data they need for analysis. This self-service capability needs to happen to enable data-driven organization. But there is a fine line between enabling self-service and ensuring data is accessed by the right people and used in the right situations.
As any developer knows, perfect software doesn’t just happen it, pardon the pun, “develops” over time. Developers engage in a seemingly everlasting iterative process involving bug fixes and changes that can last for the lifetime of an application. But writing the software is only half the battle; it must then be deployed.
For big data companies like ours that run software across distributed networks, this is no small task. In particular, a developer makes changes, runs tests, identifies errors or processing improvements to address, and then makes more changes.
I recently attended Hadoop Summit 2016 where not surprisingly there was a lot of conversation about topics other than Hadoop. For example, the importance of ecosystem partners to any Big Data solution.
When it comes to gathering the right data and finding the relationships that make that data more meaningful, there’s one role that knows how to do it best - the data steward. That’s why they are often referred to as data detectives.