Big Data and Analytics, The Analytics Value Chain – Part 3

Part 1 disambiguated the terms Big Data and Analytics. Part 2 reviewed Evolution of Big Data, its complexity and new value. In part 3, we explore the Analytics Value Chain.

[Part 1] [Part 2] [Part 3] [Part 4]

Someone once said “the best predictor of future performance is past performance.” This has been the elusive commercial objective for Big Data and Analytics. Companies invest millions on CRM and ERP systems to collect historical business data. Social media has opened the window of understanding of a different kind of current real-time data requiring us to rethink this objective and put stock in an Analytics Value Chain.

Big Data and Analytics – The Value Chain

Analytics initiatives have typically focused on performance reports with charts, tables and graphs of historical data. Software provider’s sales pitches touting predictive analytics have yet to deliver more value than cost in easy-to-deploy-and-use platforms. Companies, governments and institutions struggle to find analytics tools that will help drive markets, instead of react to them (market driving vs. market driven).

Today, experts are challenged to develop digital tools to automate analysis of vast data created now in real time. The collective objective of IT professionals and computer scientists is to make observations, analyze information, and make predictions. Recent advances in Big Data and Analytics now begin explore how rule-based systems can interpret information, manipulate knowledge and apply artificial intelligence to prescribe specific action. The combination of these advances creates the opportunity to move beyond observing the past and beyond predicting future scenarios. It presents the opportunity to prescribe action, statistically correct and accurate action.

A Value Chain Emerges

After decades of perfecting data collection, structures, storage, access and retrieval, a value chain has emerged. The value chain connects Big Data and Analytics through the convergence of complexity and diversity. Complex content beyond text and numeric data now includes image, audio and video data. Diverse data sources include data warehouses storing silos of historic enterprise data on one end of the spectrum. On the other end of the spectrum is un-stored streaming data from sensors and capturing devices delivered in real-time feeds.

There are five basic links in the Analytics Value Chain, each with a unique objective/outcome. Following Michael Porter’s definition of a value chain, each link (step) in the chain adds value to its output.

The anchor link in the chain is Big Data. At great cost, it is captured, managed, hoarded and often considered a valuable asset. However, it has the least value in the chain, is static, and initiates the other links.

The second link analytically preparesBig Data through unique methods based on its type (text, numbers, images, audio or video). This involves either programmatically or otherwise, increasing the intelligence and value contained in the data. It can be as simple as tagging meta-data, or as complex as face-recognition or movement mapping in image data.

The third link identifies patterns and relationships. Dozens of today’s dashboard reporting solutions consider reporting on what already happened to be an analytics system. Reports and dashboards only present data, the do not interpret, generate insight or approach decision making. People then must glean insight and subjectively make decisions based on the report.

The fourth link analyzes to predict potential future scenarios. This step generates insight from data. Computed insight reduces human variation and risk of error. While many ERP and reporting software vendors sell “predictive analytics,” they often fall short.

The final link automatically prescribes action to take and is the ultimate goal of analytics. Statistics based prescriptive analytics creates great value beyond prediction programmatically identifying “what” to do. Using the power of the cloud to compute statistical probability, distribution, variance, standard deviation, regression p-values and so on, can generate real prescriptive direction out of static and streaming Big Data.

Half-life Of Big Data

Prescriptive analytics shifts our value metric of present-streaming data over past-stored data (see the discussion of “ratio of actively collected vs. passively captured data in Part 2) Understanding of the half-life of data makes this clear. As data ages, it is less valuable. This implies the converse: that younger, or closer to real-time, data has more value. Nucleus Research’s “Guidebook – Measuring The Half Life Of Data” concluded this in terms of tactical, operational and strategic purposes.

Nowhere is fresh data more critical than in healthcare, security and defense industries. Body-monitoring sensors can detect conditions before symptoms manifest. Healthcare providers, the patient and healthcare resources can be alerted simultaneously of an imminent medical event so that prescriptive action can prevent the event entirely. Real-time analytics applied to streaming surveillance, intelligence, and other inputs can shorten emergency response time. Security and defense can take prescriptive action to avert tragedy and saves lives.

Disambiguation Revisited

Gartner Market Research suggested this week in its conference on Big Data that the concept of Big Data also includes the science of Analytics. I do not fully agree. Many new advances are focused solely on IT back-end performance and capacity for data itself. Analytics science for processing and mining value remains fertile ground for nurturing innovation and technological breakthrough. The anticipated advances will deliver disruptively significant value from both predictive and prescriptive analytics. Unlike Gartner, I believe it is a good idea to discern Big Data from Analytics. This clarity ensures analytics budget allocation where the returns are greater than just switching out hard disks for faster units to collect and store more data. Differentiating Analytics as a separate value-creating function will provide returns sooner from investments already made.

Future Challenges

There are not enough people that fully grasp the Analytics Value Chain. Gartner says that by 2015, Big Data and Analytics is “expected to create 4.4 million IT jobs globally.” The report goes on to say there is a shortage of skilled professionals, and that only 1/3 of these positions will be filled. Lack of existing talent and lack of education to produce future talent is a serious issue.

Whenever there is great demand for technical solutions, like in this case, and not enough people to deliver them, it forces innovation of the disruptive kind. Universally applicable platforms technology will emerge. The common functionality for Big Data and Analytics will be repurposed across many purposes, industries, and applications. Those that understand this will be winners.

Image credits: Half-life of data graphic – Nucleus Research.  Analytics Value Chain – self.  Chain – Max Klingensmith via photopin cc.

Addendum – Other Value Chains (section added 30 November 2012)

This series of posts sparked considerable traffic. Clearly this subject is on many a CIO, and other CxO’s minds. A request for more information and sources to research was made. The following figures and links are a few of the many sources out there. Simply search on Google on the keywords “Analytics Value Chain” and dozens of references come up many 5 years and older.  These are the basis for many current interpretations that even show up in last months symposium by Gartner.

The next post is about value chains from Accenture, Gartner and more that I particularl find valuable, as they have depth and provide clear insight over other more brief models. Value chain analysis has been around since Michael Porter at Harvard first described and popularized the approach, and defined that every link in a chain or process, adds value.

The first version of my inverted pyramid figure to describe value chain for analytics was developed about 5 years ago when I built a business intelligence strategy for a position and movement analytics system that connected the PivotLink and Tableau -like dashboards that processed data warehouses (big data) and prepared the data for analysis in dashboard environments. The model in my head has been evolving to this current version as all value chains do. There’s nothing original about it – please copy, use, share and expand on it as you see fit to achieve your own analytics initiatives.  For more research, you may find the following sources valuable. My own perspective is based on combining insight from a number of these other interpretations of an analytics value chain including:

• Cap Gemini Blog post “Analyze This! : Analytics Explained and applied,” on May 27, 2011,

• MIT Sloan Managemnet Review’s article, “Big Data, Analytics and the Path From Insights to Value,” on December 21, 2010,

• Delen, Dursun and Demirkan, Haluk, “Decision Support Systems, Data, information and analytics as services,” from Elsevier, published online May 29, 2012, and

• Davenport, Thomas H and Harris, Jeanne G. “Competing on Analytics,” Mar 1, 2007. (SAS has a brief PDF summary from 2008, at the of this book.)

Ayata has also developed an streamlined version of an analytics value chain that connects the first two steps of collecting data and preparing it for analysis with real statistical analytics as described by Cap Gemini (above) and others. This is profound as it takes the rearward-perspective into enterprise data that many basic dashboard systems perform (like those of PivotLink and Tableau) and allows a forward look attached to decision making and action taking. Advanced data preparation as defined in 2007 by Delen and Hemirkan in their work on Decision Support systems, has been around for some time. But the advent of ubiquitous cloud computing power has increased the potential for democratization.  The Ayata value chain while simple, is profound.

We will no doubt hear more about the Ayata analytics platform in the future.  Full disclosure: 8 days after I posted this Blog article, Analytics Magazine published work from Ayata and Fritz Venter in a co-authored article “Images and videos: Really Big Data” with me that shows this simplified value chain and the increasing value at every step.

Analytical Decision Process Spider Diagram – SAS summary of work by Delen and Demirkan noted above.