Navigating the cloud of Big Data hype:
lessons to live by
Navigating the cloud of Big Data hype: lessons to live by
Inform
When you look up “cloud” in a thesaurus, it’s interesting what turns up. “Mist, fog and haze” appear as common nouns. “Confuse, darken and obscure” appear on the verb list. These are apt metaphors for the all too common discussions between business teams and the data analysts and technology teams that support them. The analyst / tech team
-“I think we should put everything on the cloud.”
“I recommend we go down the ‘jamjim’ route.” “The backend is great. Really scalable.” “I’m worried about latency.”
The business
-“Why should everything be on the cloud?” “I just want to know what sales figures are likely to be.”
“We’ve spent $1M but we still don’t have data we can trust.”
“That analysis is clearly wrong.”
“I don’t want to be on the news about some data security issue”
“Why can’t we have something simple like
Exit the data/IT dilemma
Developments in data collection, storage, processing, and platforms continue apace. More solutions on the market make the data management and technology landscape con-fusing and difficult to navigate for both IT departments and the business groups they serve. As the demand for, and availability of, solutions that harness the power of data analytics increase, this tension between business and technology will only continue to heighten. What is needed is to strike a balance between technical capabili-ties and genuine business needs.
It’s well known that ‘hyped technology’ rarely lives up to its lofty expectations, and all too often a particular technology is selected based upon criteria far removed from actual business objec-tives. Business groups need to clearly define and communicate the return they expect while IT must set realistic goals for delivery and function-ality. Hype and a lack of a common language are stumbling blocks that businesses and IT depart-ments face when trying to maximise the utility of data and technology to support business suc-cess. Both groups are keen to achieve the ben-efits of leveraging data and technology, but strug -gle to make it happen.
Organisations producing information on a large scale are often tempted by the promise of new data analysis technologies and their ability to
im-Inform- Exit the data/IT dilemma
Transform - Lessons
Outperform - Fact-based management
INFORMATION DRIVES SOUND ANALYSIS, INSIGHT, AND ACTION.
Transform
Optimity Advisors has identified six core lessons for leaders to take on board in order to help them maintain or build competitive advantage in the market by using data and technology to build customer loyalty, locate areas for improve-ment and make better decisions earlier. These are based on our 25 years of experience work-ing with global public and private organisations to solve complex technology and data problems in a broad range of industries.
Lesson 1: Take control of
informa-tion management “insight engine”
Knowing your sales volume, accounts receiv-able and customer satisfaction are just a few of the basic metrics that are available and used by businesses. Frequently however analytical tasks which are perceived to be more complex, such as assessing the income risk of different market segments, or projecting future demand based on demographics and other factors, are frequently outsourced to specialist providers who provide a “black box” solution – algo-rithms and analysis which are not transparent.Whilst this outsourcing strategy may have some short term value, an organisation which does not own, and have control of the way in which it un-derstands how it assesses risk and allocates time and resources, or controls the pace and nature of improvement, leaves itself vulnerable. This could be because information providers will be sell-ing the same or similar approaches or may even go out of business. There are many examples of companies now in the world of media, retail, and transportation who have seen the opportunity afforded by analyses which can help businesses identify not only how they have performed in the past, or currently, but also how it looks as though they will perform in the future. In the board room this means changing the conversa-tion from one focussed on what was achieved months before and setting up working groups to investigate what should be done, to one focussed on having facts at the fingertips concerning un -der or over performing “lead” indicators and risk areas - more like a weather forecast. This type of analytical approach is frequently called predictive analytics and the insights it creates we have lik-ened to an “engine” in the way that it keeps run-ning and needs to be well maintained. In order to create such an engine business groups and data/ IT teams need to have a shared view of what this insight engine could deliver, have the skills available to deliver these insights, and then cre-ate an agreed route map. There are also cultural challenges to overcome to ensure that the analy-ses and tools created promote the right type of risk based discussion in the business as weather forecasts are not always correct, but continue to improve through greater insights into how weather patterns work.
capabilities appeal to IT, while business teams desire tools that will create long-term value and growth. When different teams are speaking ferent languages and viewing problems from dif-ferent perspectives, where is the middle ground where transformation can take place?
Lesson 2: Work hard to get the
data in working condition
More often than not, initial investigation of data reveals it to be less of a silver bullet than an-ticipated. Data might be delivered too late to be actionable, or key elements may be missing altogether. Data may be disorganised or contain strange values. These challenges are particularly common when surfacing data that has gone un-used in storage for months or years.
We have two options when faced with ‘dirty’ data: to get frustrated or to take steps to resolve the issues. Organisations that reap the benefits of data are the ones that commit to working with facts and establish a governance program with new data-centric roles such as Chief Data Officers and data managers to negotiate across the business: what the data means, how it is de-fined and how it is used. These roles cut across the business, including finance, sales and market -ing, and have the focus to ensure that the single version of the trusted facts to operate the busi-ness are available.
Lesson 3: Make sure analysis and
systems are flexible and
responsive
to changing circumstances
Whilst getting the data in working condition is the foundation to a successful fact based busi-ness, in life and in business the only constant is change. Customer expectations and needs change, new data are collected, businesses merge or are bought, supply chains alter, behavioral
in-analyses and IT in this environment will inevita-bly limit the success of any business. Planning for improvement by investing in data architecture and innovations in analysis allows businesses to stay ahead of the game by experimenting and quickly delivering valuable and surprising in-sights. In terms of IT, regular feedback by end us-ers, contracts focussing on delivering outcomes in staged cycles as opposed to “Big Design Up Front” approaches will also build in the flexibility needed and help ensure that the business is in the lead, rather than falling behind. For business groups and data/IT this frequently means devel-oping a more iterative and flexible approach to working with business transformation and solu-tion providers, focussing on a long term vision and negotiating business benefits incrementally, as opposed necessarily to creating one off big specifications.
Lesson 4: Face the facts about
cus-tomers
For many years, market surveys, satisfaction forms and focus groups were seen as the way to understand customers. The reliance was on self-reported behaviours and immediate reactions as opposed to facts concerning what people ac-tually did. Now, customer insights are captured digitally at a whole new level of volume and granularity. Customer sentiment and behaviour have become the lifeblood of business strategy by linking data at the individual level and incor-porating new “unstructured” data sources from social media to create new insights.
way that concert tickets are sold. It used to be that an orchestra would focus their customer insight on audience demographics and frequency of attendance. Now, companies examine a whole new set of insights - how and when tickets were purchased, where people arrive from, what food they ate before the performance, how they engaged with the performance through media. Having access to this type of information can help inform how and where to maximise cus-tomer loyalty, develop evangelists and gain new customers through targeted “nudges” which may influence the way that people feel about a business, product or service. Getting hold of these facts, and managing them responsibly and consistently is now possible and those business-es that unlock thbusiness-ese insights are seeing rewards in customer loyalty and market share. For busi-ness groups and data/IT this means being more creative about the data sources that could be used/become available, negotiating access, and then working through the data governance and security considerations. This provides a platform to undertake the analysis to understand more about the existing customer base, how custom-er loyalty could be expanded and how poten-tially new customers could be gained.
Lesson 5: Invest in data
architec-ture
Some believe that only analysts should have ac-cess to data because it might be misinterpreted or misused by business groups. This says less about the data and more about communica-tion. Understanding and communicating risk can be challenging, but many organisation resolve the issue by identifying and communicating key
metrics using well-designed visualisations that are easy to interpret. This is an educational and design journey that requires on-going discussion between analysts and business leaders.
Businesses that successfully leverage data, focus on the “data architecture”. These are the poli-cies, rules or standards that govern what data is collected, how it is stored, arranged, integrated, and put to use by stakeholders inside and out-side of the organisation. A key element of a qual-ity data architecture concerns the automation and delivery of key “buckets” of information to help business groups understand performance and goals. Classically these information buckets cover reputation, quality, revenue, cost, profit, efficiency, pipeline and contractual/legal compli -ance. Business rules can be created as part of the architecture, which trigger different warning signals. This approach means that business users can identify areas for extra focus at a glance by incorporating predictive analytics and established business rules into the data architecture program. A further benefit of this automation approach is that valuable data scientist time can be freed up from performing repetitive manual tasks pro-ducing monthly board packs, by automating data loading, cleaning and processing.
For business groups and data/IT, this means committing to developing a data architecture that supports business success and is fit for the future, as well as establishing a route map to de-liver this architecture. This will inevitably mean working through a range of people and process issues, alongside the data and technology chal-lenges.
Lesson 6: Moneyball your business
This story, frequently recounted, focused on the Oakland Athletics baseball team, which had very limited resources compared with their compe-tition in Major League Baseball. Yet, they man-aged to get to the playoffs in both 2002 and 2003 by rejecting perceived wisdom, creating new insights and applying these new data to help inform decisions and achieve their competitive objectives. To do this the Athletics put their trust in creating new hypotheses on what contributed to “success”, undertaking statistical analysis and then using these insights to shape their roster and trading policy. As a result they were able to pick up undervalued (and less expensive) play-ers who went on to perform well. A similar ap-proach has now been taken on by most, if not all, teams.Just imagine if the same approach could be used in other environments such as the most cost ef-fective way to reduce risk for different patient groups, maximise sales, gain new leads or achieve learning outcomes. Some businesses have start-ed working in this way. Some have automatstart-ed standard reporting functions and created new “intelligence skunkworks” - groups of data sci-entists and programmers testing ways in which people react to different interventions, identify-ing new variables of real value and embeddidentify-ing them in the business on a day to day basis. For the business groups and data/IT teams this means setting aside intuition and conventional wisdom and approaching the business with an open mind. Experiments can then be designed and undertaken - testing assumptions and hy-potheses with the data in order to better
under-Outperform
Committing to fact-based management
Many consistently high performing businesses create fact-based management with ‘one version of the truth’ and ‘intelligence skunkworks’ to de-liver new insights. The journey is not easy, and requires a team effort involving different aspects of businesses that have traditionally worked in silos and don’t speak the same language. Find-ing a way to overcome this language barrier and design a good route map incorporating not just data and technology, but also people and process issues will unlock business potential.
activities deliver the greatest return on invest-ment. If these analyses tell a different story than was expected, the insights should be taken on board by the business to influence which clients it targets, what service lines it operates and how it engages the market.
Matrix Knowledge formally joined the global consultancy group Optimity Advisors in September 2014. As its European arm, the newly combined business trades as Optimity Matrix to run the public policy arm of Optimity Advisors’ global operations. For more info go to: www.optimitymatrix.com. Optimity Matrix and Matrix Knowledge are trading names of TMKG Limited (registered in Eng-land and Wales under registration number 07722300) and its subsidiaries: Matrix Decisions Limit-ed (registerLimit-ed in England and Wales under registration number 07610972); Matrix Insight LimitLimit-ed (registered in England and Wales under registration number 06000446); Matrix Evidence Limited (registered in England and Wales under registration number 07538753); Matrix Observations Lim-ited (registered in England and Wales under registration number 05710927); and Matrix Knowledge Group International Inc. (registered in Maryland, USA under registration number D12395794).
Disclaimer: In keeping with our values of integrity and excellence, Optimity Matrix has taken reasonable professional care in the preparation of this document. Although Optimity Matrix has made reasonable efforts, we cannot guarantee absolute accuracy or completeness of information/ data submitted, nor do we accept responsibility for recommendations that may have been omitted due to particular or exceptional conditions and circumstances.
Confidentiality: This document contains information which is proprietary to Optimity Matrix and may not be disclosed to third parties without prior agreement.
Except where permitted under the provisions of confidentiality above, this document may not be reproduced, retained or stored beyond the period of validity, or transmitted in whole, or in part, without Optimity Matrix’s prior, written permission.
