Towards Data Democratization: Thoughts on the Gartner BI & Analytics Summit

Some Attivio folks flew to the annual Gartner BI event last week to take the pulse of Business Intelligence, data discovery, and data democratization. We wanted to hear the latest from Gartner thought leaders and the several thousand data practitioners. In the opening keynote, there were 5 key takeaways. I’d like to zero in on numbers 2 and 3.

Number 2: Accept that the world will get more distributed. No surprise here. The shift began when new ways of consuming data emerged and spread like wildfire.

Number 3: Shift your thinking from control to empowerment. Centralized command-and-control has proven to be balky and often ineffective in the new world order. Organizations are looking towards the alternative, data democratization, as a means to get more value from their data.

Life in the new, distributed world

In the past, the BI architecture was built around the notion of a single source of truth. The EDW housed the data and MDM served as a master index. When the business needed data, they made a request to IT.

That paradigm has shifted and expanded. The EDW still has an important place in the BI architecture, but a different reality has set in. Business analysts download data visualization tools, like Tableau, and powerful new ways of consuming data have emerged. Along with that, the growing volumes of data and new analytics use cases – specific to a line of business, or to an industry – have led to growing legions of analytics and BI users. The new citizen data scientist and citizen analyst are joining the ranks.

These analysts struggle to get the data they need. The bottleneck resides in the slow, manual processes of finding and unifying data sources. So analysts build their own spreadmarts of data, in a consumerization of IT movement. IT is concerned that the data is ungoverned, inconsistent from one user to the next, and uncontrolled.

Empowerment works better than control

The greater demands for data are outpacing IT’s ability to deliver. The business makes a request, and IT uses mostly manual, trial-and-error based processes to fulfill the request. It takes time. And the business doesn’t have IT’s context: they may or may not have a clear idea of the data they need. This creates a costly bottleneck.

Ever larger volumes and varieties of data add to the complexity. This is also what’s driving adoption of Hadoop technologies and the growth of data lakes. This cheaper, more flexible storage now complements the EDW. The new BI architectures are outgrowing the single source of truth paradigm from days past.

The new, distributed world needs a new path to data access. These analysts could move mountains if they had self-service data discovery.

Answer to the control-vs-empowerment dilemma: best of both

More and more, organizations find that the sweet-spot is neither chaos nor control. It’s data democratization: empowering the business user to find and understand data on their own.

 In the next gen BI architecture, a semantic metadata catalog, one that spans every silo, the EDW, even the data lake, provides data access at the speed of business. This actually gives IT more governance, while it gives business users quick, transparent data discovery — as if picking the data off an Amazon shelf.

We’re back from Gartner more determined than ever to provide data discovery that provides the best of all worlds — governance and self-service data discovery, together for empowerment. In a true data democracy, data infuses decision making, and the whole organization runs smarter.