Business analysts and line-of-business (LOB) data users have plenty of robust, self-service BI tools at their disposal. What they often lack is a way to get all the most relevant data into those tools. In a TDWI Checklist Report, Dave Stodder, Director of TDWI Research for Business Intelligence, lists seven best practices for executing a successful data science strategy. Number five: Give Data Science Teams Access to All the Data.
Two or three years ago, when Big Data had started to gain serious traction in large enterprises, there was a rush to hire data scientists. Of course, disagreement reigned about what credentials made a true data scientist. Wonky math geeks were a good place to start. The rush to hire data scientists echoed the trend some decades earlier when investment firms hired quants right out of college and put them in the basement to create trading algorithms.
One thing was clear though. Data scientists were scarce and expensive.
Despite Gartner’s observation that 41% of organizations are unsure if their Big Data ROI will be positive or negative, they remain keenly interested in investing in Big Data technology so as to take advantage of data-driven use cases in an effort to improve predictions and forecasts, exploit IoT opportunities, identify new products and services, and improve operational efficiency. Other tactical business drivers include real-time decisions and insights. It’s tempting for organizations to climb on the Big Data bandwagon while overlooking the unique set of corresponding challenges attributable to the:
Enterprise search is back in the news—with a twist. Companies that really want to accelerate their results with BI and Big Data are looking to enterprise search as a way to help business analysts quickly find the data they need. Note that I said “data,” not information. Enterprise search has always been thought of as a way to find unstructured content in file shares like SharePoint. But now, it’s being applied to strucutured data as well. And if a search solution can combine data with unstructured content so much the better.
The challenges you face as a risk and compliance professional reflect forces beyond your control. You’re between an accelerating expansion in technology – which offers new avenues for misbehavior - and a rising tide of regulatory expectations. It’s a classic problem of supply and demand.
Start with the supply-side. Combine the human resources, known information resources, and the technology solutions you deploy for risk and compliance (R&C), and you essentially determine your supply of time to monitor, investigate, and react. Finding qualified R&C professionals has never been more difficult. Information has never grown faster. And solutions have never been more difficult to replace.
What to Get for A Person Who Calculates Everything
Talking with colleagues and friends in business lately, I’ve taken a new tack. After all, it’s the season that celebrates gifts - large and small. After greetings, I’m asking, “What’s on your list this year?” I get all sorts of answers - from the sublime (typically, peace) to the ridiculous (mostly, political) – but it’s the answers of a friend – call him Jerry for convenience, that I thought I’d share today.
The short answer: “To get them out of their hair.” But seriously…
At Attivio, we work with decision-makers at various companies who own and administer the BI infrastructure. We call them “BI tech owners.” A BI tech owner’s team governs data and delivers it to business users for analysis.
BI tech owners are not in an enviable position at the moment. The proliferation of self-service analytical tools for Big Data and BI have generated orders of magnitude increases in requests for data. And those requests all come with an ASAP attached.
Before joining Attivio, I worked for several years at Tibco Spotfire. It was a great experience. I was on the front lines as the worlds of Big Data and Business Intelligence (BI) collided.
Traditionally, companies relied on canned BI reports to help them understand historical data. Such reporting solutions have been around for decades. It was, therefore, very exciting to see that massive market disrupted by data discovery and analytic solutions such as Tableau, QlikTech, and Tibco Spotfire. These new, easy-to-use, data visualization tools helped analysts, researchers, and data scientists quickly self-serve insights from massive data volumes.
Instead of relying on a combination of static reports and massive spreadsheets to manually comb through huge data sets—an unreliable process that could take months—the new data discovery and analytic solutions instantly revealed trends, patterns, and outliers in mere moments, with just a few clicks.
There was a lot of talk at the Tableau Conference last month about the challenge of getting the right data into Tableau. There’s a process bottleneck between IT and the business that prevents the easy flow of data in the organization. Data democratization is a powerful concept that ultimately enables companies to compete on analytics.
From EDW to Big Data
It’s hardly a new problem. The business intelligence industry has dreamed of the enterprise data warehouse, and now it dreams of big data and the gold it may become. Ultimately, it’s all to pursue the same goal: to use data as the strategic asset it should be. But something has always come in the way.
A central challenge in transforming a ponderous Big Data headache into an agile Big Analysis win is finding a way to scale (often spelled S-U-P-P-O-R-T) Tableau users with the data they seek to visualize. Like most coins, there are two images we can observe – one from the perspective of the data analyst visualizing in Tableau and one from the data manager who structures and provisions data tables worth visualizing. My last post took the former perspective – today, I’ll look at the latter.
Data security, governance, and management are essential elements – non-negotiable elements – in traditional data warehousing or contemporary Big Data initiatives. And, as table stakes, they can effectively limit the degree of freedom for deploying new solutions or tools.