CDOs Clear Path to Big Data Competency

As the Big Data and analytics parade marches on, I often find that the people we're talking to in large enterprises carry the title chief data officer or CDO. Industry analysts back this up. A 2015 report by PwC found there were 100 CDOs in large enterprises in 2013, more than double the number in 2012. Gartner's most recent tally pegs the number at 950. And it predicts that by 2017, 50 percent of all companies in regulated industries will have a CDO. The CDO role is still evolving, but Debra Logan of Gartner describes the CDO as the "glue between data strategy and metrics."

From an efficiency and value-driven analytics perspective, many CDOs inherit an immature and fragmented data ecosystem. Frequently, this situation is compounded by the proliferation of analytics tools across an organization and various shadow IT efforts that involve business units creating their own databases, with no governance or oversight. Newly minted CDOs may also need to close a Big Data skills gap, re-architect various processes that cut across business and functional units, and foster a data-driven culture. In other words, there's plenty of potential for spectacular success—and conspicuous failure.

It's fair to say that today's CDOs are first generation. To some extent—for better or worse—they're making it up as they go along. And as Gartner's first CDO survey points out, "There is a serious lack of consistent, meaningful metrics to measure the effectiveness of the CDO office, especially in relation to business metrics."

Coping with a Complex Data Environment

In most organizations, structured data is still the norm. They have policies to govern it, tools to analyze it, and people with the skills to use those tools. Of course, with the adoption of NoSQL databases and myriad SQL variants even structured data has become more complex. Many CDOs tell us they have a very distributed data warehouse environment that makes it difficult to provide access, determine the quality of data being sourced, and understand what level of detail is needed. Is the use case in depth analytics or simply a report? Then, incorporating unstructured data into the mix demands entirely different processes, skill sets, tools, and data processing frameworks. Moreover, many systems create potentially usable data but no standards exist for extracting data across disparate systems.

Beyond Data Silos

The terms "data silos" don't really do justice to the fragmentation CDOs must deal with to make their data architectures support the analytic needs of the organization. Of course there are data silos—in the traditional sense of application silos. But then we have multiple sources and data types within and outside the organization. It makes something seemingly as simple as identifying customers across an entire company much more difficult than it should be.

Often enterprises rely on third-parties to provide data for specific purposes. The data is never brought in house. It's used operationally, but the databases are not designed for broader analytics. This kind of fragmentation is systemic. It applies not just to the data but to the way systems use data—and the way organizations perceive the value of data.

CDOs Stand at the Cross Roads 

While undoubtedly in the organizational hot seat, today's CDO has the very rare opportunity to transform an organization. Much lip service is paid to competing on analytics, but the CDO can actually make it happen. The challenges are many, not the least of which is a bimodal architecture that allows robust information governance and streamlined, enterprise-wide information access to productively co-exist.

The successful CDO may be a charismatic synthesis of visionary evangelist with nuts and bolts data technologist. Someone who can make the top, middle, and bottom of the data and analytics stack work together to support an organization that really is data driven.