One of the biggest changes in the upcoming Sherlock release of the Attivio Platform is moving to a Hadoop-based architecture for big data applications. Hadoop provides us with a number of low level capabilities that we no longer have to manage at this scale, such as resource management (YARN), coordination (Zookeeper), storage (HDFS) and bookkeeping (HBase). One of the side benefits of YARN based resource allocation is the ability to scale up or down your system’s footprint with a few simple YARN commands.

Good question. What is the point? The point is to create measurable business value from enterprise data. Of course, before measurable business value comes insight. The Modern Data Architecture (MDA) recognizes that insight can lie hidden in data of all types—structured or unstructured, messy or modeled, historical or realtime.

As I’ve mentioned in prior blogs, the biggest use cases we see in Hadoop these days come from the risk and compliance functions of large banks. Initially, many banks and other financial services institutions (FSIs) adopted Hadoop out of sheer necessity despite its early immaturity on the governance front.

Writing on the O’Reilly.com site back in August, CEO Jessie Anderson of Smoking Hand, a training company for Big Data technologies, commented on the overall complexity of Big Data, NoSQL technologies, and the distributed systems that deploy them.

An entire ecosystem of tools and data processing frameworks have grown up around Hadoop. Almost as soon as someone identifies a weakness or limitation—and there have been more than a few—someone else creates a fix. That's one of the reasons the Big Data ecosystem is so complex. And why many large companies hung back before jumping in. They wanted to see if any leaders would emerge from the chaos.

If you missed the Strata+Hadoop Conference in New York City last week, here’s a quick recap.

As many of our customers move their entire compute environment—including analytics and storage—into the Hortonworks distribution of Hadoop, Attivio has focused on gaining greater integration with Hortonworks. Having one platform repository in which to execute makes everyone's life simpler.

Forrester just released its latest Wave report. Unlike many on more mature technologies, this report on native Hadoop BI platforms included only six vendors, of which Attivio was one. In Forrester's estimation, there are no leaders in the market, just contenders and strong performers.

Hadoop is fast becoming the center piece of a modern data architecture. And Cloudera Enterprise provides the centralized management and robust support that you need to effectively operate Hadoop.

The modern data architecture stores data as is; it doesn't require pre-modeling. It needs to accommodate volume, velocity, and variety including structured and unstructured information. Hadoop does this very well.

I recently attended Hadoop Summit 2016 where not surprisingly there was a lot of conversation about topics other than Hadoop. For example, the importance of ecosystem partners to any Big Data solution.

It was certainly on my mind as we had just published a post about Attivio’s role in the EMC Big Data solution. At the Summit, I sat down with Carey James from EMC and Tony Fisher from Zaloni—another partner in the EMC solution—for an interview with John Furrier of The Cube.