H T T P : / / W W W . A N A L Y T I C S - M A G A Z I N E . O R G
JANUARY/FEBRUARY 2014 DRIVING BETTER BUSINESS DECISIONS
BROUGHT TO YOU BY:
ALSO INSIDE:
ANALYTICS CAREERS
& CONSULTING
Executive Edge Verisk Innovative Analytics President Marty Ellingsworth on the future, big data and bigger analytics• Eighteen things nobody tells you
about solo practice
• Certification: What it means
for employers, practitioners
• Analytics-driven culture:
Why it’s a corporate
no-brainer
•
Predictive analytics in the cloud
• Analytics & health management
•
Dealing with missing values in data
Special Supplement:
CAP Candidate
Big dreams, small data
INSIDE
STORY
Like everyone else involved in the analytics space, we’ve been yapping end-lessly about “big data” in this column. You all know the story – unfathomable amounts of data coming in from multiple sources at incredible speed have analysts everywhere scrambling to make sense of it all. Let’s face it, big data is the elephant in the room in any discussion of analytics, and the elephant is only going to get bigger (think hybrid data, in-cluding video, images, sound, text, etc. from countless sources and sensors).
But wait, there’s more; there’s a “small” angle to the “big data” story. Even in the Big Data Era, many companies do not have the data they need to make data-based deci-sions. A start-up, for example, almost cer-tainly does not have the historic data that an established firm has collected. Even well-established companies probably lack the data they need when considering intro-ducing a new product or service or entering a new market.
With that in mind, Analytics magazine will launch a new column by Brian Lewis in the March/April issue that will address the issue of insufficient data and how to over-come it. The name of the column: “Big Data Dreams, Small Data Reality.” Chew on that concept for a minute.
Lewis, chief data scientist and co-found-er of Fractal Sciences, provides more de-tails in an introductory column in this issue.
Of course, big data remains the big fish in the analytics pond, so we’ll continue to cover it and all of its ramifications. For example, in this issue’s Executive Edge column, Marty Ellingsworth, president of Verisk Innovative Analytics, discusses the “promise of big data and bigger analytics” that “will drive the future” as the corporate world shifts from a company-centric to a customer-centric culture.
Meanwhile, INFORMS, publishers of
Analytics magazine and the world’s
lead-ing organization for high-end analytics, will
present its inaugural INFORMS
Confer-ence on Big Data in San Jose, Calif., June 22-24. The conference will focus on the
business of big data and making the
jour-ney from data-rich to decision-smart. For a preview of the conference, click here.
The issue also includes a couple of “ca-reer-builder” feature articles that should pique the interest of any analytics professional looking to get an edge in a competitive en-vironment. Veteran analyst Doug Samuelson outlines some of the consulting lessons he’s learned the hard way, while Polly Mitchell-Guthrie and Scott Nestler give an update on INFORMS’ Certified Analytics Professional program and how it can help employers and clients of analytics professionals, as well as analytics professionals themselves.
– PETER HORNER, EDITOR
OPTIMIZE
YOUR BUSINESS
WITH UNPRECEDENTED
SPEED
[email protected] | +1 425 458 4024
To learn more about AIMMS Optimization Apps, visit aimms.com. TO YOUR ENTERPRISE OPTIMIZATION APP STORE
PUBLISHED
INSTANTLY
IN A FEW
DAYS
PROOF OF CONCEPTIN A FEW
WEEKS
OPTIMIZATION APPIN A FEW
MONTHS
MISSION CRITICAL ENTERPRISE APPIN A FEW
HOURS
IDEADRIVING BETTER BUSINESS DECISIONS
C O N T E N T S
FEATURES
PREDICTIVE ANALYTICS IN THE CLOUD By James Taylor
Research survey: Ability to deliver ROI solutions more cost-effectively is driving cloud deployment.
ADVENTURES IN ANALYTICS CONSULTING By Doug Samuelson
Eighteen things nobody tells you about solo practice that you need to know before you take the plunge.
CERTIFIED ANALYTICS PROFESSIONAL By Polly Mitchell-Guthrie and Scott Nestler
Usage guide for employers and clients answers key questions about first-of-its kind CAP® program.
ANALYTICS & HEALTHCARE By Rajib Ghosh
Apply predictive analytics to address population health management and medication adherence problems.
MISSING VALUES By Gerhard Svolba
The origin, detection, treatment and consequences of missing values in analytics.
SPECIAL SUPPLEMENT
Certified Analytics Professional candidate handbook30 36 44 52 58 52 44 36 JANUARY/FEBRUARY 2014 Brought to you by
Tel 775 831 0300 • Fax 775 831 0314 • [email protected]
AnAlytic Solver PlAtform
easy to Use Predictive and Prescriptive Analytics in excel
How can you get results quickly for business decisions, without a huge budget for “enterprise analytics” software, and months of learning time? Here’s how:
Analytic Solver Platform does it all in Microsoft Excel,
accessing data from PowerPivot and SQL databases.
Sophisticated Data Mining and Predictive Analytics
Go far beyond other statistics and forecasting add-ins for Excel. Use classical multiple regression, exponential smoothing, and ARIMA models, then go further with regression trees, k-nearest neighbors, and neural networks for prediction, discriminant analysis, logistic regression, k-nearest neighbors, classification trees, naïve Bayes and neural nets for classification, and association rules for affinity (“market basket”) analysis. Use principal components, k-means clustering, and hierarchical clustering to simplify and cluster your data.
Simulation, Optimization and Prescriptive Analytics
Analytic Solver Platform also includes decision trees, Monte Carlo simulation, and powerful conventional and stochastic optimization for prescriptive analytics.
Help and Support to Get You Started
Analytic Solver Platform can help you learn while
getting results in business analytics, with its Guided
Mode and Constraint Wizard for optimization, and Distribution Wizard for simulation. You’ll benefit from User Guides, Help, 30 datasets, 90 sample models, and new textbooks supporting Analytic Solver Platform. Analytic Solver Platform goes further than any other software with Active Support that alerts us when you’re having a problem, and brings live assistance to you right where you need it – inside Microsoft Excel.
Find Out More, Download Your Free Trial Now
Visit www.solver.com to learn more, register and download a free trial – or email or call us today.
New ASP V2 014 with Dim ension al M ode ling
DRIVING BETTER BUSINESS DECISIONS
REGISTER FOR A FREE SUBSCRIPTION: http://analytics.informs.org
INFORMS BOARD OF DIRECTORS
President Stephen M. Robinson, University of Wisconsin-Madison
President-Elect L. Robin Keller, University of California, Irvine
Past President Anne G. Robinson, Verizon Wireless Secretary Brian Denton,
University of Michigan
Treasurer Nicholas G. Hall, Ohio State University Vice President-Meetings William “Bill” Klimack, Chevron Vice President-Publications Eric Johnson, Dartmouth College
Vice
President-Sections and Societies Paul Messinger, CAP, University of Alberta Vice
President-Information Technology Bjarni Kristjansson, Maximal Software Vice President-Practice Activities Jonathan Owen, General Motors Vice President-International Activities Grace Lin, Institute for Information Industry
Vice President-Membership
and Professional Recognition Ozlem Ergun, Georgia Tech Vice President-Education Joel Sokol, Georgia Tech Vice President-Marketing,
Communications and Outreach E. Andrew “Andy” Boyd, University of Houston Vice President-Chapters/Fora David Hunt, Oliver Wyman
INFORMS OFFICES www.informs.org • Tel: 1-800-4INFORMS Executive Director Melissa Moore
Meetings Director Laura Payne Marketing Director Gary Bennett Communications Director Barry List
Headquarters INFORMS (Maryland)
5521 Research Park Drive, Suite 200 Catonsville, MD 21228
Tel.: 443.757.3500 E-mail: [email protected] ANALYTICS EDITORIAL AND ADVERTISING
Lionheart Publishing Inc., 506 Roswell Street, Suite 220, Marietta, GA 30060 USA Tel.: 770.431.0867 • Fax: 770.432.6969
President & Advertising Sales John Llewellyn
Tel.: 770.431.0867, ext. 209 Editor Peter R. Horner
Tel.: 770.587.3172 Art Director Jim McDonald
Tel.: 770.431.0867, ext. 223 Advertising Sales Sharon Baker
Analytics (ISSN 1938-1697) is published six times a year by the
Institute for Operations Research and the Management Sciences (INFORMS), the largest membership society in the word dedicated to the analytics profession. For a free subscription, register at http://analytics.informs.org. Address other correspondence to the editor, Peter Horner, [email protected]. The opinions expressed in Analytics are those of the authors, and do not necessarily reflect the opinions of INFORMS, its officers, Lionheart Publishing Inc. or the editorial staff of Analytics.
Analytics copyright ©2014 by the Institute for Operations
Research and the Management Sciences. All rights reserved.
DEPARTMENTS
Inside StoryExecutive Edge Analyze This! Forum
Big Dreams, Small Data INFORMS Initiatives Conference Previews Five-Minute Analyst Thinking Analytically 2 8 14 18 22 26 66 72 76
66
76
Tel 775 831 0300 • Fax 775 831 0314 • [email protected]
AnAlytic Solver PlAtform
from Solver to full-Power Business Analytics in excel
The Excel Solver’s Big Brother Has Everything You Need for Predictive and Prescriptive Analytics
From the developers of the Excel Solver, Analytic Solver Platform makes the world’s best optimization software accessible in Excel. Solve your existing models faster, scale up to large size, and solve new kinds of problems.
From Linear Programming to Stochastic Optimization
Fast linear, quadratic and mixed-integer programming is just the starting point in Analytic Solver Platform. Conic, nonlinear, non-smooth and global optimization are just the next step. Easily incorporate uncertainty and solve with simulation optimization, stochastic programming, and robust optimization – all at your fingertips.
Ultra-Fast Monte Carlo Simulation and Decision Trees
Analytic Solver Platform is also a full-power tool for Monte Carlo simulation and decision analysis, with a Distribution Wizard, 50 distributions, 30 statistics and risk measures, and a wide array of charts and graphs.
Comprehensive Forecasting and Data Mining
Analytic Solver Platform samples data from Excel, PowerPivot, and SQL databases for forecasting and data mining, from time series methods to classification and regression trees, neural networks and association rules. And you can use visual data exploration, cluster analysis and mining on your Monte Carlo simulation results.
Find Out More, Download Your Free Trial Now
Analytic Solver Platform comes with Wizards, Help, User Guides, 90 examples, and unique Active Support that brings live assistance to you right inside Microsoft Excel. Visit www.solver.com to learn more, register and download a free trial – or email or call us today.
New ASP V2 014 with Dim ension al M odeling
Whether you’re steering an enterprise, champion-ing an analytics program, drivchampion-ing a venture capital-funded data-modeling product or piloting your own consulting practice, there is no way you have not be-come aware of the promise of big data and bigger analytics.
Companies often deal with an opaque market-place where riskiness is rated less accurately in some organizations than others. That said, some businesses, afraid of taking on new risks, stick to their own market niches and avoid new ones – preventing their abilities to be competitive in the larger marketplace. Others are blazing trails us-ing newer technologies, data sources, modelus-ing approaches and electronic connectivity to better manage risks in an increasingly mobile world. The following framework helps identify how companies are structured to compete on analytics.
SPECTRUM OF PREDICTIVE ANALYTICS CAPABILITIES
Now we’re at the beginning of a long rally race of analytic improvements – from newer, better data to smarter, faster algorithms. Enhancements will include broader, more scalable platforms and will access
How analytics will drive
the future
BY MARTY ELLINGSWORTH
Some businesses, afraid
of taking on new risks,
stick to their own market
niches and avoid new
ones – preventing their
abilities to be competitive
in the larger marketplace.
Others are blazing trails
using newer technologies,
data sources, modeling
approaches and electronic
connectivity to better
manage risks in an
increasingly mobile world.
EXECUTIVE
EDGE
With our Solver App for Office 365,
SharePoint 2013, and Excel 2013, you have all the capabilities of the Excel Solver on your desktop, laptop, tablet or phone.
It works in the Excel Web App with all popular Web browsers, and solves your model “in the cloud” on
Windows Azure.
And it’s free and available now! Just visit solver.com/app or the Office Store online, or use Insert Apps for Office in Excel 2013.
New Solver for Office 365 Excel: Free for Your Tablet or Phone
Analytic Solver Platform: Multidimensional Models with PivotTables
Analytic Solver Platform 2014 brings
multi-dimensional optimization
modeling to Excel. Easily create
dimensions or index sets and cubes
of computed values, using regular Excel formulas – extended to operate over multiple dimensions.
Use PivotTables, created in Excel or from databases with PowerPivot, to populate your model with data. Easily create new PivotTables of optimization results.
New
Find Out More, Download Your Free Trial Now.
Visit www.solver.com to learn more, register and download a free trial – or email or call us today.
Frontline Solvers – The Leader in Spreadsheet Analytics Tel 775 831 0300 • Fax 775 831 0314 • [email protected] New
EXECUTIVE
EDGE
unique sensors – spectra, spatial, temporal – and micro/macro levels of structured and unstructured data. All that will help generate insights into individualized, massively personalized and localized information while getting even more power out of grouped predictive parameters.
For example, if you have operated a car or other moving object, you undoubtedly have been assessed for your risk of loss as an operator of that vehicle and, in some manner, likely have been insured. In the past, that insurance-based risk assessment has blend-ed wide bands of information on a few historically available generic characteristics to achieve a general-purpose estimate of prospective loss risk estimates.
In the future, that historical benchmark will be segmented into ever-more granular and accurate assessments. Those will then again be reinvented, recombined and refined to enhance the data-driven process, culminating in an adaptive analytic that ad-justs expectations to the level of risk in each operat-ing scenario encountered or intuited.
In the world of big data and bigger analytics, in-surers will come to view vehicles as instrumented platforms and operators as real-time learners whose risk may change over time. Operators may drive safer and make smarter decisions about moving between locations, or they may permit distractions into their cockpit (such as texting, talking, smoking, eating and so on). How you drive, when you drive, where you drive and how much you drive are all becoming part of the context in your individual risk profile.
Some businesses apply detailed telemetry, rout-ing algorithms, real-time weather and traffic alerts and driver/crew pairing models to manage more
Now we’re at the
beginning of a long
rally race of analytic
improvements – from
newer, better data to
smarter, faster algorithms.
Enhancements will include
broader, more scalable
platforms and will
access unique sensors –
spectra, spatial, temporal
– and micro/macro
levels of structured and
unstructured data.
A N A L Y T I C S J A N U A R Y / F E B R U A R Y 2 014 | 11
effectively the logistics of moving people, packages and pallets. Similarly, indi-vidual consumers make daily choices to move themselves, their passengers and their belongings along the same road-ways and flyroad-ways and use all sorts of new navigation and alerting applications and devices to do so. (I’ve seen a mobile tablet computer go from a plane to a car to a sofa all in the hands of the same in-dividual within one morning.)
Peering into the future, if a submers-ible helicopter car becomes common-place, we’d have a truly three-dimensional
driving experience. And on those jour-neys, we might need to dodge Amazon’s Octocopter self-driving delivery micro bots along the way.
In the ubiquity of an instrumented world, such a trend is unstoppable. Our challenge will be how we will use ana-lytics to interact with decision-making. If consumers continue to make their own decisions, it’s a certainty that marketing analytics, advertising effectiveness and brand campaigning will merge into mo-bile and content messaging more than ever before. The next frontier will involve
Figure 1: Companies vary widely in their abilities to create and use predictive information. There are seven stages of development for predictive analytic capabilities, and each has a level of investment and an expected return. The companies with the most mature capabilities will have invested in all seven stages shown in the illustration and, depending on individual jurisdictional restrictions, will have deployed analytic models to serve their customers and compete for others.
EXECUTIVE
EDGE
the layered sensing of the temporal and spatial con-text surrounding the customer.
Decisions that address emotional desires of cus-tomers resonate in the behavioral economics that un-derpin our financial world. The closer we can come to a customer’s desires, the better – and better still to be able to influence demand by making customers aware of opportunities they did not know exist. That holds true for business-to-business decisions as well.
The paradigm shift transitions from company-centric to customer-company-centric and from “we always have done ‘IT’ this way” to real time. That shift must be the focus of top management, which needs to take the offense and drive resource allocation for innovation and productivity to a customer-focused, real-time strategy. Executives who embrace such a process of optimization that both considers max-imizing enterprise performance while minmax-imizing risks will effectively revitalize every decision op-portunity in marketing, production, distribution, lo-gistics, operations, servicing and sales. And they will find there is no finish line when generating more shareholder value – only a continual cycle of improvement and a corporate culture of data-driven, sustainable excellence.
Marty Ellingsworth is president of Verisk Innovative Analytics, a
division of Verisk Analytics (www.verisk.com). Verisk Innovative Analytics is a member of the INFORMS Roundtable, and the author is a long-time member of INFORMS.
The paradigm shift
transitions from
company-centric to customer-company-centric
and from “we always have
done ‘IT’ this way” to real
time. That shift must be the
focus of top management,
which needs to take the
offense and drive resource
allocation for innovation
and productivity to a
customer-focused,
real-time strategy.
Join the Analytics Section of INFORMS
For more information, visit:
© 2014 Fair Isaac Corporation. All rights reserved.
Explore and solve complex
problems for FREE
FICO® Xpress Optimization Suite… FREE to Academic Institutions and Their Students
Solve enormously complex problems, faster
Xpress can handle complex problems with millions of variables and constraints.
You’ll quickly narrow down choices to the very best (or the N-best) solution when there are innumerable possible solutions that would otherwise be difficult to compare.
Explore “What If” Scenarios
Xpress offers improved solution sensitivity analysis, making it possible to efficiently explore larger quantities of “what if?” scenarios. It finds the strongest levers by adjusting variables and constraints to measure how far they move optimal operating points.
Take advantage of unique features for large scale optimization
Xpress offers true 64-bit support for optimization and modeling, allowing users to solve ultra-large-scale models with more than 2 billion coefficients.
Learn more about the Xpress Academic Partner Program: fico.com/app
A member of our MBA Program Task Force was talk-ing about recent alums who had been successful on the job market, and early in her discussion, she cited the stu-dents “who had, you know, taken Vijay’s classes.” This did seem a little weird – my classes have course num-bers and names, after all – but we were all in the midst of a very busy semester, and so I happily let it go.
A couple of weeks later, when an MBA staff per-son came by my office to propose adding a section of one of my MBA electives, she mentioned the great demand for “classes in my area.” I suggested that we simplify things by just referring to them as analytics courses (while my department’s name has changed almost annually, the word “analytics” has always been part of it). She responded equivocally, and looked ter-ribly uncomfortable doing so.
Then, just before the holidays, I arrived a few minutes late to a meeting of the Graduate Programs Committee (I was giving a final exam that ran slightly over), expecting to present my proposal for a new MBA course in data min-ing. However, as I organized my handouts, a colleague seated nearby informed me with a chuckle that my new “non-analytics” course had already been approved.
I wondered: “Why all this weird verbal tap dancing?” Well, after some digging around, I got an answer, though it was not a very satisfying one. During the last academic year, my school had launched a new Master of Science in Analytics (“MSAN”) program. The adminis-trator who owned the program had apparently sought to differentiate the content of his program by explaining to
Preparations for a career
in analytics should be built
on a three-legged stool of
computing skills, analytic
capabilities and business
effectiveness skills.
BY VIJAY MEHROTRA
What is ‘real’ analytics?
A N A L Y T I C S J A N U A R Y / F E B R U A R Y 2 014 | 15
anyone who would listen that the courses that we teach to MBAs are not “real” analyt-ics courses, since these classes do not re-quire any computer programming (outside of the Excel environment, which is viewed pejoratively by techies) and do not delve deeply into the algorithmic details behind techniques such as optimization, regression or cluster analysis.
This is just ridiculous.
First of all, in this kind of rhetorical re-sponse, one is required by custom to pro-vide a definition, and mine comes from Davenport and Harris’ book, “Competing on Analytics”: Analytics, they state, is “the extensive use of data, statistical and quan-titative analysis, explanatory and predictive models, and fact-based management to drive decisions and actions.” Based on this definition, it is clear that the skills needed for successful analytics professionals are both broad and deep. George Roumeliotis, an analytics leader at Intuit, believes that a good data scientist needs to be a skilled business consultant who also has a broad array of technical skills for data manage-ment, analysis and modeling [1].
What this means is that preparations for a career in analytics should be built on a three-legged stool of computing skills (in-cluding the ability to gather, merge, clean and manage data), analytic capabilities (with a special emphasis on basic probabil-ity and statistics, data mining, dimensionalprobabil-ity
reduction methods and fundamentals of op-timization) and business effectiveness skills (such as leadership, problem framing, team-work, project management, communication skills and negotiation). Any academic pro-gram that purports to be focused on prepar-ing students for a career in analytics must strive to address each of these three com-petencies in some meaningful way, though there are an infinite number of ways to com-bine each of these somewhat orthogonal vectors.
While I was thinking about all this, I came across a blog entry on Forbes.com entitled “Business Analytics Beyond BI: Rise of the MBAs” [2]. The author, John Furrier, is a tech industry veteran and the founder of the website SiliconAngle. com, which pays an awful lot of atten-tion to analytics and Big Data [3]. Though this relatively short article covered a lot of ground, a handful of interconnected “money quotes” caught my eye:
1. “Every department within a company today is itching to apply data-driven systems to their workloads.” What he’s saying here – and what my business school colleagues are slowly starting to understand – is we’re moving toward a time when most professionals will have to be conversant in working with data and interpreting models. We will need to start expecting more of our MBAs in
ANALYZE
THIS!
I do my best to train
them to think critically –
and wherever possible,
to utilize user-friendly
tools – to address a
variety of data- and
model-driven business
problems.
these areas, and to keep innovating to deliver it. 2. “Automation will empower the data scientist to
empower everyone else at the company, and they’ll need the help of software.” Automating the data scientist role has been discussed ad nauseam [4], but Furrier has a slightly different take on it: Automation is essential so that these critical human resources can be better focused on changing managerial behaviors, rather than having so much of their time consumed with managing data.
3. “The role of the data scientist plays an important part in setting the tone for collaboration within an organization, as these multidisciplinary problem solvers will need to communicate clearly with each other, as well as every other department.” That is, if the more technically trained analytics professionals can’t work well with less technically trained professionals, an organization’s analytic capabilities will fall far short of their potential.
Back here at USF, my colleagues in the MSAN program have made the choice to emphasize the computational and statistical aspects of analytics, which as expected has led to incoming students and outgoing graduates who are suited for very technical roles. My MBA students, howev-er, do not have either the programming skills needed to implement algorithms from the ground up or the inclination to acquire them. Instead, their focus is on the business rather than technology. Instead, I do my best to train them to think critically – and wherever possible, to utilize user-friendly tools, which will be flooding the market for years to come – to address a variety of data- and model-driven business problems, while also working through data qual-ity and management issues as needed. Given the well-documented talent shortages, it is not surprising that both
J A N U A R Y / F E B R U A R Y 2 014 | 17 A N A L Y T I C S
NOTES & REFERENCES
1. For more from George’s view on what makes a good data scientist, check out http://online-behavior.com/emetrics/marketing-metrics-intuit
2. www.forbes.com/sites/siliconangle/2013/12/10/ big-data-beyond-bi-rise-of-the-mbas/
3. http://siliconangle.com/?s=big+data
4. For example, see http://www.allanalytics.com/ author.asp?section_id=1408&doc_id=251420, http://www.forbes.com/sites/gilpress/2012/08/31/
the-data-scientist-will-be-replaced-by-tools/, and http://smartdatacollective.com/ radhikaatemcien/111596/data-scientist-scarcity-automation-answer.
groups are finding good (though very differ-ent) opportunities in today’s marketplace.
But let’s be clear: Both of these types of programs (and both of these types of students) are just as worthy of the term “analytics.” And in the future, we can expect that these folks will be working closely together on all sorts of things.
Vijay Mehrotra ([email protected]) is an associate professor in the Department of Analytics and Technology at the University of San Francisco’s School of Management. He is also an experienced analytics consultant and entrepreneur,
Job Seeker Benefits
• Access to high quality, relevant job postings. No more wading through postings that aren’t applicable to your expertise. • Personalized job alerts notify you of relevant job opportunities. • Career management – you have complete control over your passive or active job search. Upload multiple resumes and cover letters, add notes on employers and communicate anonymously with employers. • Anonymous resume bank protects your confidential information. Your resume will be displayed for employers to view EXCEPT your identity and contact information which will remain confidential until you are ready to reveal it. • Value-added benefits of career coaching, resume services, education/training, articles and advice, resume critique, resume writing and career assessment test services. POWERED BYhttp://careercenter.informs.org
CAREER
CENTER
an angel investor in several successful analytics companies and a longtime member of INFORMS.
“In the final analysis, the root cause of Japan’s de-feat, not alone in the Battle of Midway but in the en-tire war, lies deep in the Japanese national character. There is an irrationality and impulsiveness about our people which results in actions that are haphazard and often contradictory.”
– Mitsuo Fuchida and Masatake Okumiya [1] Business analysis dissolves in an IT culture and in other cultures too. Philosophically, business analy-sis is in its Romantic Era – an era in which analyanaly-sis is applied hither and yon in a tactical swashbuckling manner. Corporations aspiring to improve their de-cision-making to become more analytics-based will want to foster a more analytics-driven culture. They should seek a culture that:
1. Rewards analytics-based decision-making as in a meritocracy.
2. Integrates analytics into their strategy.
3. Embraces the pace of dynamic change during this analytics phase of the Information Age.
4. Accepts that understanding data analysis is part of understanding the business.
5. Fosters experimentation and continual learning about the business.
Analytics-driven culture
BY RANDY BARTLETT
Philosophically, business
analysis is in its Romantic
Era – an era in which
analysis is applied hither
and yon in a tactical
swashbuckling manner.
Corporations aspiring to
improve their
decision-making to become more
analytics-based will want
to foster a more
analytics-driven culture.
FORUM
Left-brainers vs. right-brainers: In the Information Age, it is not wise to
run a company with half a brain.
J A N U A R Y / F E B R U A R Y 2 014 | 19 A N A L Y T I C S
Corporate culture can be defined as “how we do business.” An analytics-driven culture necessarily blends analytics and company know-how. We can raise the analytics content of the culture by adjusting the lead-ership, specialization, delegation and incentives [2].
Analytics-driven cultures have built a legacy of seek-ing great financial opportuni-ties based upon the numbers. They have learned to accept or tolerate the scientific meth-od, plan for analytics, and enable analytics to drive de-cision-making. They are more deliberate in collecting appro-priate data for their decisions. Rather than passively
react-ing to the data available, their proactive planning includes thinking ahead to seek new types of data that does not yet exist.
A crude measure of a corpora-tion’s acceptance of analytics is the extent to which analytics profession-als are spread through the company. If a corporation wants to develop a more analytics-driven culture, then it needs to expose people to business analytics and spread analytics pro-fessionals throughout the company – growing the culture by spreading the approach.
LEFT BRAIN–RIGHT BRAIN CULTURAL CLASH
The explosion of information im-plies that we need to apply scientific tools; this is not about pottery or po-etry. Left-brain purveyors of the sci-entific method sound like Mr. Spock or today’s modern icons, Dr.
Saman-tha Carter and Dr. Daniel Jackson of Stargate SG1. Analytics professionals are trained to accept their ignorance, value humility in presenting results and qualify their statements. They are often self-made.
At a large bank, a group of predictive modelers was told never to say, “I don’t know” when answering questions from senior management. Similarly, they were told not to include caveats in their presentations. All these confessions of ignorance and qualified statements appear like “doubt speak” to the right-brainers. Do you have the answer or not? Captain Kirk, or the sensibly upgraded Dr. Elizabeth Weir of Stargate Atlantis, just want the an-swer so that they can “decide already.” Should we put our phasers on stun or close the stargate? Was that so difficult?
We can benefit by thinking through these cultural differences.
What you need to know if you are dating a right-brainer: Left-brainers, you need to go to charm school and forget about impressing others with your level of preparedness, intelligence and impeccable logic. Okay, we get it; analytics is subject to uncertainty. Now start socializing analytics so it is not so threatening.
Making other people feel stupid does not make you appear very smart. Stop qualifying your results. You dwell much too much on the fact that if your analysis is correct, then there is still a chance that the conclusion is wrong – incomplete information being what it is. Finally, if you are in a non-analyt-ics culture, then you need to do more than write a glossary of acronyms and speak the local language. You must walk the walk, too. You need to behave as much like the right-brainers as you can stand – conform a little, sadly. Just deal with it.
What you need to know if you are dating a left-brainer: Right-brainers, you need to appreciate that the left-brainers have the lonely responsibility for getting the facts right in the face of messed-up, incomplete information. Going forward, you need to evolve, to accept more of the communication burden, to think differently. Is this so threatening? You want to embrace or at
least accept uncertainty. When reading analysis, you should interpret signs of intellectual humility as signs of intellec-tual humility and not weakness. If you want the left-brainers to explain things simply, then you can help by remind-ing them that you realize their work is complex. Keep asking them the same question until you get it. Do not give up. However, you cannot expect them to di-vulge their secret techniques.
If the above was not enough for you, left-brainers will want to share all of the bad news they have discovered. Just deal with it.
What we all need to know: In prac-tice, we all have left- and right-brain behaviors, and anyone who thinks that some group of people is homogeneous does not know much about them. Now that we are in the Information Age, it is no longer wise to run a company with half a brain. Today, our medieval corpo-rate cultures from the Dark Ages place too much of the burden on the left-brain-ers to get the numbleft-brain-ers right and explain it so that a right-brainers can under-stand it [3]. This is unreasonable – or at least not optimal. Instead, we should ask for the caveats and accept the “I don’t knows” on our way to cashing in on analytics. In analytics, there can be something suspicious about someone with all of the answers; they are not
FORUM
A N A L Y T I C S J A N U A R Y / F E B R U A R Y 2 014 | 21
left-brain. The cultural change we seek is to be both left- and right-brain.
DENYING THE SERENDIPITY OF STATISTICS
Before purchasing expensive data or executing a sophisticated analy-sis, you should plan how you are go-ing to use this information or how you are going to analyze a business prob-lem. Having a plan makes sense – just not perfect sense. No one sat down and wrote a detailed plan for the dis-covery of penicillin. It was a complete accident. Many great discoveries hap-pen by chance. Holding a data request up to the standards of a mathematical proof is a bit much. This is a chronic breakdown point and the site of many a discombobulation. In an analytics-driven culture, it should be sufficient for a plan to entail what you expect and emphasize the economics of the pos-sible exploratory work. We may need to make numerous attempts on our way to success.
Finally and foremost, we must resist the temptation of allowing people to pres-ent other peoples’ analytics work. This delays acclimation and creates a decep-tive culture. At a number of corporations, this is the standard. No one below a cer-tain rank is given the privilege of present-ing to senior management, and the token
few qualified analytics professionals will always be below that rank – whatever it takes. This senior management intends to stay insulated in the “executive man-agement bubble,” all right-brain.
Randy Bartlett (Randy.Bartlett@
BlueSigmaAnalytics.com), Ph.D., is a business analytics/big data leader with Blue Sigma Analytics. He has more than 20 years of experience,
which includes leading and organizing analytics resources, reviewing advanced analytics results and providing advancements in business analytics. Bartlett delivers presentations and writes about business analytics, including the article “The Business Analytics Revolution,” co-authored with Girish Malik, that appeared in the May/June 2013 issue of Analytics magazine. Bartlett is also the author of a book, “A Practitioner’s Guide to Business Analytics,” from which this article was adapted. Reprinted with permission from McGraw-Hill Professional. Bartlett is a member of INFORMS.
NOTES & REFERENCES
1. It is an amazing feat to write a book about a naval battle and tie the outcome to a cultural characteristic. See “Midway: The Battle That Doomed Japan” by Mitsuo Fuchida and Masatake Okumiya (1955). 2. “Competing on Analytics: The New Science Of Winning,” “Analytics At Work: Smarter Decisions, Better Results,” “Data Driven: Profiting from Your Most Important Business Asset,” and “Super Crunchers: Why Thinking-by-Numbers Is the New Way to Be Smart,” among others, have made it clear that analytics is too understated in the blend. 3. This is the right-brainers saying they cannot be bothered to think in a left-brain manner for a single moment.
Subscribe to Analytics
It’s fast, it’s easy and it’s FREE! Just visit: http://analytics.informs.org/
There is no doubt that we live in an era of dig data. We seem to have mountains of data about everything from business operations to customer behaviors, from personal health to global disease outbreaks. Be-cause of this abundance of data, many “best prac-tices” rely on making data-based decisions. Yet there are still many situations where we unfortunately do not have sufficient data to make such decisions. So what do you do when you have big data dreams but a small data reality? This is the focus of a new
Analytics magazine column, debuting in the March/
April 2014 issue. Through this column we will explore how to deal with situations where we need data, but it’s limited or nonexistent.
AT SCALE: BIG DATA
I’ll give a specific example from my current work as chief data scientist at Fractal Sciences, a marketing automation software company that optimizes digital ad-vertising and engagement (think Facebook and Twitter ads, but a lot more). Without giving too much away and without getting too technical, Fractal’s ad optimization algorithm is based in part on a proprietary feedback loop that uses prior ad campaign data to automatically pre-dict, recommend, create and target new ads in order to maximize a customer’s ROI for their advertising spend. As a result, our customers’ ad campaign results get bet-ter and betbet-ter the more they use our product.
Big data dreams,
small data reality
BY BRIAN LEWIS
Because of this abundance
of data, many “best
practices” rely on making
data-based decisions.
Yet there are still many
situations where we
unfortunately do not have
sufficient data to make
such decisions.
CALL
FOR TOPICS
J A N U A R Y / F E B R U A R Y 2 014 | 23 A N A L Y T I C S
In our internal data science meetings, we love to think about, tinker with and invent our next generation algorithms for when a customer is “at scale.” When at scale, a customer has run enough ad campaigns that have created enough data that we can finally apply some of our cutting-edge predictive analytics, machine learning and optimization algo-rithms. That is, we actually have some big data to work with.
EARLY ON: SMALL DATA
Long before a customer is at scale, they are essentially in a start-up phase. In this phase, terabytes and petabytes are re-placed by mere megabytes. A/B/n testing is replaced by just … A. Predictive analyt-ics is replaced by anecdotal evidence. And sample sizes are so small that the concept of statistical significance is, well, insignifi-cant. From a data science perspective, we refer to this as small data.
But despite the lack of data during this start-up phase, customers still expect our platform to optimize their ad campaigns. So how do we approach this situation? We will address this and other similar situations in the “Big Data Dreams, Small Data Reality” column.
A few other obvious examples in-clude planning for new businesses, new products or services and new business processes.
A start-up almost certainly lacks the historical data that an established company has collected about its opera-tions, finances or sales and marketing strategies. Yet a new business still needs to plan its future: which products or services to launch, which customers to target, how to set pricing policies, how to promote the brand, how to layout the website, how much to staff up, and so on. All of these decisions could be aided by data, if only you had some. In the absence of data, one of the most important parts of planning to make data-driven decisions is how you structure your decision model. Did you include the right objectives, constraints and other assumptions?
Even though you have no data, you still have to populate your model with something, for example industry bench-mark data, data from public company SEC filings, probability distributions (if you want to use something more sophisticated like Monte Carlo simula-tion), and yes, even gut-feel values. As you start gathering data, you can transition from those external data sources to your own internal data. But when do you make this transition? How much data is enough data?
In contrast to new businesses, well-established companies, such as the Fortune 500, have databases upon
databases of data. Yet when they launch new products and services, they also start off with no direct data to work with when making their de-sign, launch and planning decisions. If the new product is similar to existing products, you might use data from those products as proxy data until you have collected sufficient data about your new product. Examples of proxy data include histori-cal sales data for building demand forecasts or customer profiles for deciding to target audiences for advertisements. Once again, you must start with some form of external data and then transi-tion to your real data over time.
Introducing a new business process is no dif-ferent than the two cases described above. You have no data with which to accurately predict results such as operational efficiencies or inef-ficiencies. So you make assumptions, use some sort of data that is external to the new process, and then slowly transition to your direct data as you gather it.
CALL FOR TOPICS FOR FUTURE COLUMNS
The vision for this column is to be a commu-nity discussion about all types of small data situa-tions at all types of companies and organizasitua-tions. If you have a small data situation that you would like discussed in this column or even want to be interviewed for the column about your small data situation, please let me know! Until then, keep big
data dreaming.
Brian Lewis, Ph.D. ([email protected]), is chief data scientist and co-founder of Fractal Sciences, a digital marketing automation and optimization software company.
CALL
FOR TOPICS
You have no data with
which to accurately
predict results such as
operational efficiencies
or inefficiencies. So you
make assumptions, use
some sort of data that
is external to the new
process, and then slowly
transition to your direct
data as you gather it.
“
Each issue of Foresight contains articles that you’ll use in your day-to-day work, whatever types of forecasting you do.
Here’s what our readers say:
The information is relevant to practitioners and is presented in a way that is not overly academic but with significant credibility.”
- Thomas Ross, Financial Analyst, Brooks Sports
Foresight make(s) important research findings available to the practitioner.”
- Anirvan Banerji, Economic Cycle Research Institute
...an important forum for practitioners to share their experiences....” - Dan Kennedy, Senior Economist, Connecticut Department of Labor
I find Foresight very useful! I use it as a teaching resource to bring theoretical forecasting techniques to life for the students.”
-Dr. Ilsé Botha, Senior Lecturer, University of Johannesburg
Put Foresight to work to improve your forecasts and rally support for the people, processes, and tools that accurate forecasting requires.
Subscribe or Renew today!
forecasters.org/foresight/subscribe/
Connect with the earned expertise
of business forecasters and
practical research from top
academics from around the globe.
THE INTERNATIONAL JOURNAL OF APPLIED FORECASTING
Spring 2013 Issue 29
THE INTERNATIONAL JOURNAL OF APPLIED FORECASTING
5 Forecasting revenue
in Professional Service Companies
14 Forecast value added:
A Reality Check on Forecasting Practices
19 s&oP and Financial Planning
26 cPFr: Collaboration Beyond S&OP
39 Progress in Forecasting rare events
50 Review of "global trends 2030:
alternative Worlds"
THE INTERNATIONAL JOURNAL OF APPLIED FORECASTING
Fall 2012 Issue 27
THE INTERNATIONAL JOURNAL OF APPLIED FORECASTING
5 Special Feature: Why Should I Trust Your Forecasts?
23 Tutorial: The Essentials of Exponential Smoothing 29 S&OP: Foundation Principles and Recommendations for Doing It Right
40 New Texts for Forecasting Modelers
THE INTERNATIONAL JOURNAL OF APPLIED FORECASTING
Summer 2012 Issue 26
5 Setting Internal Benchmarks Based on a Product’s FORecaSTaBIlITY DNa 18 Regrouping to Improve Seasonal Product Forecasting
32 Forecasting Software that Works For – Not against – Its Users
38 Book Review Abundance: The Future Is Better Than You Think
41 Reliably Predicting Presidential elections
THE INTERNATIONAL JOURNAL OF APPLIED FORECASTING
“
“
“
“
The Institute for Operations Research and the Management Sciences (INFORMS) named six fi-nalist organizations that will compete for the 2014 INFORMS Franz Edelman Award. The Edelman Award is the world’s most prestigious recognition for excellence in applying advanced analytics to benefit business and humanitarian outcomes.
This year’s finalists include:
• The Energy Authority for “Hydroelectric Generation and Water Routing Optimizer” • Grady Memorial Hospital, with the Georgia Institute of Technology, for “Modeling and Optimizing Emergency Department Workflow” • Kidney Exchange at the Alliance for Paired
Donation, with Stanford and MIT, for “Kidney Exchange”
• NBN Company, with Biari, for “Fiber Optic Network Optimization at NBN Co.”
• Twitter, with Stanford University, for “The ‘Who to Follow’ System at Twitter: Strategy, Algorithms and Revenue Impact”
• The U.S. Centers for Disease Control and
Prevention (CDC), with Kid Risk, Inc., for “Using Integrated Analytical Models to Support Global Health Policies to Manage Vaccine Preventable Diseases: Polio Eradication and Beyond”
World’s best analytics teams
compete for Edelman honors
The Edelman Award is the
world’s most prestigious
recognition for excellence
in applying advanced
analytics to benefit
business and humanitarian
outcomes.
INFORMS
INITIATIVES
J A N U A R Y / F E B R U A R Y 2 014 | 2 7 A N A L Y T I C S
The six finalists will make presen-tations at the INFORMS Conference on Business Analytics and Operations Research in Boston, March 30-April 1. The winner will then be announced at the Edelman Awards Gala held in conjunc-tion with the conference.
Now in its 43rd year, the INFORMS Franz Edelman Prize competition rec-ognizes outstanding examples of ana-lytics and operations research projects that transform companies, entire indus-tries and people’s lives. Using innova-tive advanced analytical methods, the teams were instrumental in helping their respective institutions make better de-cisions, providing a disciplined way by which management can improve orga-nizational performance in a wide variety of situations and across both public and private organizations.
INFORMS Franz Edelman finalist teams have contributed more than $200 billion in benefits to business and the public interest. The 2014 finalists were chosen after a rigorous review by accom-plished verifiers, all of whom have led successful analytics projects. Finalists are chosen on the merits of how analyt-ics methodologies were applied to solve problems, reduce costs or otherwise im-prove results in real-world environments.
Additional information about the INFORMS Franz Edelman Award
competition, including video inter-views with 2013 finalist executives,
can be found online at https://www.
informs.org/Recognize-Excellence/ Franz-Edelman-Award.
NEW DATES FOR CONTINUING EDUCATION FOR ANALYTICS PROFESSIONALS
Following its successful launch in 2013, INFORMS will once again offer its popular continuing education courses “Essential Skills for Analytics Profession-als” and “Data Exploration & Visualiza-tion” in 2014.
Essential Practice Skills for Analytics Professionals
Gain essential tools for integrating your analytical skills into real-world prob-lem solving.
“The course was expertly presented and outlined methods that were immedi-ately applicable to my everyday work. I would recommend this course to anyone looking to improve their problem-solv-ing skills as well as their ability to pres-ent complex projects and problems in a clear, concise and logical way.”
- Richard St-Aubin, P.Eng., IPEX Management Inc.
Upcoming classes:
Feb. 20-21 - Atlanta March 28-29 - Boston
Data Exploration & Visualization
Hands-on training that focuses on the critical steps in the process of analyz-ing data: accessanalyz-ing and extractanalyz-ing data, cleaning and preparing data, exploring and visualizing data.
Upcoming classes:
March 28-29 - Boston
June 25-26 - San Jose, Calif.
For more information on the courses and available discounts, visit www.informs.org/continuinged
2014 INNOVATIVE APPLICATIONS IN ANALYTICS AWARD
The Analytics Section of INFORMS has named three finalists – Ford, IBM and Fiserv – for its 2014 Innovative Ap-plications in Analytics Award that will be
presented at the INFORMS Conference
on Business Analytics and Operations Research in Boston, March 30-April 1. Sponsored by the section, the purpose of the award is to recognize creative and unique developments, applications or combinations of analytical techniques used in practice. The award competition promotes awareness of the value of ana-lytics techniques in unusual applications, or in creative combination to provide unique insights and/or business value.
Following a series of presentations, the three finalists were chosen by a
panel of judges from a competitive set of 31 submissions and nine semifinalists. Taken together, this work provides great examples of innovative applications and integration of a variety of analytical tech-niques that are making a difference in or-ganizations. The three finalists will make another round of presentations in Boston before a winner is named.
A closer look at the finalists and their work:
Ford: “Enabling greater sustainability in vehicle fleet sales through
customer-oriented analytics”
Authors: Daniel Reich, Sandra L. Winkler, Erica Klampfl, Natalie Olson, Ford Motor Company
Presenting author: Daniel Reich
Abstract: Ford’s Fleet Purchase
Planner (patent pending) is the first of its kind in promoting sustainability as a central purchase consideration for organizations with large vehicle fleets. In recent years, several new green vehicle technologies have emerged that present opportunities for increas-ing fuel economy, but this growincreas-ing number of options is also making planning purchases a significantly more complicated endeavor. This de-cision support system is designed to simplify this process by optimizing for individual driving patterns. We are helping our fleet customers find the
A N A L Y T I C S J A N U A R Y / F E B R U A R Y 2 014 | 2 9
most cost-effective opportunities for increasing their sustainability.
IBM: “A SMS text classification system for UNICEF Uganda”
Authors: Prem Melville, Vijil Chen-thamarakshan, Rick Lawrence, Solo-mon Assefa, Machine Learning Group, IBM Research, Yorktown Heights, N.Y.; James Powell, Sharad Sapra, UNICEF Uganda; Rajesh Anandan, US Fund for UNICEF, New York, N.Y.
Presenting author: Rick Lawrence Abstract: U-report is an
open-source SMS platform operated by UNICEF Uganda, designed to give community members a voice on is-sues that impact them. Data received by the system are either SMS respons-es to a poll conducted by UNICEF or unsolicited reports of problems occur-ring anywhere within Uganda. There are currently more than 200,000 U-report participants, and they send up to 10,000 unsolicited text messages a week. The objective of the program in Uganda is to understand the data in real time and have issues addressed by the appropriate department in UNICEF in a timely manner. Given the high volume and velocity of the data streams, manual inspection of all messages is no longer sustainable. This talk describes an automated
message understanding and routing system deployed my IBM at UNICEF. We employ recent advances in data mining to get the most out of labeled training data, while incorporating do-main knowledge from experts. We discuss the tradeoffs, design choices and challenges in applying such tech-niques in a real-world deployment. We conclude with a discussion of the so-cietal impact that U-report is already driving in Uganda and discuss plans for future deployment.
Fiserv: “Price & revenue optimization for one of the largest acquiring banks in South America”
Authors: Suman Kumar Singh, Aditya Khandekar, Tarang Goyal, Fiserv
Abstract: The Merchant Acquirer had
about a million merchants in mass mar-ket. An opportunity existed to selectively optimize price for merchants based on certain attributes. A statistical segmenta-tion was developed following which non-linear differential price elasticity function was built. For each segment, price elas-ticity along with an optimization algorithm with complex constraints was developed to optimize price and achieve optimal revenue.
– Pooja Dewan (committee chair of
the Innovative Applications in Analytics Award)
Ability to deliver ROI solutions more cost-effectively is
driving cloud deployment.
Decision Management Solutions (DMS) recently conducted research into predictive analytics in the cloud. Sponsored by FICO, Lityx and SAP, the research has at its core a sur-vey of more than 350 respondents from a wide range of industries. Following up on a 2011 survey, the 2013 results make it clear that predictive analytics in the cloud is becoming increasingly mainstream, with broader and accelerating adoption.
The most striking result is that the num-ber of companies reporting a positive impact from predictive analytics has risen dramati-cally since 2011. More than two-thirds of this year’s respondents have seen a posi-tive impact from using predicposi-tive analytics in their business. It is also noticeable how much greater the reported impact is in 2013 relative to 2011. In 2013, many more com-panies reported transformative or significant impact than in 2011, while far fewer reported no usage or no plans as shown in Figure 1.
Predictive analytics
in the cloud
BY JAMES TAYLOR
D
Bucking the trend, 10 percent of the respon-dents said they still have no plans to implement an-alytics, and nearly a third have yet to put predictive analytics into production. As one respondent said, “[There is] still much user resistance to using [the] results of analyt-ics. People still believe in the superiority of human judgment.”
Matching this rise in overall impact from pre-dictive analytics is a sim-ilar rise in both current and planned deployment of predictive analytics in the cloud since 2011.
The research divided predictive analytics in the cloud into three use cases:
1. Pre-packaged, cloud-based
decision-making solutions that embed predictive analytics.
2. Cloud-based predictive modeling –
building models in the cloud.
3. Cloud-based deployment of predictive
analytics – scoring in the cloud.
These three scenarios leverage the scalability and pervasiveness of the cloud as well as the growing use of the cloud to deliver data. More than 60 percent of
survey respondents said they had de-ployed at least one of these predictive analytics in the cloud use cases – a sig-nificant increase over 2011. As Figure 2 shows, an astonishing 90 percent said it was likely they would have at least one class of solution widely deployed in the next few years. Predictive analytics in the cloud is going mainstream and may, in fact, already be there.
The primary driver for the use of cloud-based solutions was reduced cost. Advanced analytic applications have
J A N U A R Y / F E B R U A R Y 2 014 | 31 A N A L Y T I C S
Figure 1: Increasing impact from predictive analytics.
historically been both very high ROI and very high cost. There has been constant pressure on the market to deliver ROI solutions more cost-effectively, and this is clearly driving cloud deployments of predictive analytics. The typical obstacles to predictive analytics also came through in the survey: data security and privacy, along with regulatory and compliance concerns, remain the primary obstacles reported. As one respondent said, “Cloud based solutions mean either storing or transmitting our proprietary data to the cloud. Although there are safe ways to do this, our management is not convinced.”
Predictive analytics has a strong history in credit risk and fraud detection. Recently, much of the market’s energy has been directed toward the use of predictive analytics for maximizing the opportunity from customer interactions, often positioned as cross-sell/up-sell. The big focus area for predictive analytics among respondents is in customer interaction; however, the particular focus of respondents was on customer satisfaction, customer retention and customer management rather than on increased sales. Many respondents use predictive analytics in marketing and cross-sell/up-sell, but the number one focus is using predictive analytics to improve customer engagement.
Given the importance of the cloud to big data, with so many new data sources being cloud based, it seemed appropriate to investigate the impact of big data on predictive analytics in the cloud. In par-ticular, the survey explored the degree to which new data types (the variety as-pect of big data) and “recent-cy” (the ve-locity aspect of big data) were impacting respondents.
When asked what data matters most to predictive analytic models, the vast majority of most respondents indicated it was what you might call traditional data types, and structured data from their own internal systems was by far the most important. The survey also revealed a definite sense that unstruc-tured data from internal systems was becoming mainstream, while no other data types were deemed particularly important.
When more experienced analytic teams were separated out, however, and only those with existing deploy-ments or significant impact were con-sidered, the picture was quite different. These more experienced teams show much higher usage of new data types than in 2011. Social media, sensor, weblog, audio and image data types are all rated as much more important in analytic models among those with successful analytic deployments as
J A N U A R Y / F E B R U A R Y 2 014 | 33 A N A L Y T I C S
shown in Figure 3. This probably reflects the use of new data types to improve the predic-tive power of existing models. Teams are still beginning largely with traditional data types, but they see increased value from new data types once they have some success. With
more successful, more established teams using big data more broadly, it seems likely that there will soon be a rapid and significant growth in the use of new data types in building predictive analytics. More traditional structured data will likely remain broadly central to effective predictive analytic mod-els. One survey respondent put it this way: “Big data is a misnomer as data has always been big. The challenge is making use of semi-structured and unstructured data in solutions. This will be the next giant leap forward in using data.”
The velocity of data also matters. Pre-dictive analytics is increasingly focused on near real-time, operational data. This kind of data grew the most in importance be-tween 2011 and 2013. This corresponds to the general shift in predictive analyt-ics from batch scoring where results are
stored back into a database for later use to real-time scoring. This shift is reflected in the increased use of intra-day and real-time data in predictive models. As one sur-vey respondent put it: “Intra-day data will be the most valuable to our company since we are open 24 hours.”
Scoring streaming data is not yet a mainstream use case though it seems likely that the general shift to a more event-centric, real-time world will bring it squarely into the mainstream before too long.
Back in 2011 it was clear that early adopters were going to get an edge. They were more likely to have plans for broader deployment and saw predictive analytics in the cloud as more valuable. This trend strongly repeated in 2013. Once again, early adopters with one or more use cases deployed were signifi-cantly more likely to have plans to ex-pand deployment. Similarly those with
experience were likely to rate the value of each scenario more highly. Exposure to predictive analytics in the cloud breeds enthusiasm – those who buy into the promise of predictive analytics and get started like the results and want to do more. Therefore, organizations that get started quickly with pre-dictive analytics in the cloud have the opportu-nity to create differentia-tion from slower-moving competitors.
One last result
de-serves a special call out. Recent years have seen increased interest in decision management or prescriptive analytics— the embedding of predictive analytics into operational decision-making sys-tems. The importance and value of this trend was shown clearly in the survey results. More than 95 percent of survey respondents who adopted this approach (tightly integrating predictive analytics into operations) reported transformative or significant impact. Putting predictive analytics to work in operations is strong-ly correlated with the most impressive results as shown in Figure 4.
While a similar result was found in 2011, the percentage reporting the use of this approach has risen significantly since 2011 as shown in Figure 5. Decision man-agement (i.e., prescriptive analytics, with its systematic embedding of predictions into automated decision-making systems) is an effective approach to maximize the transformative power of predictive analytics.
James Taylor (james@decisionmanagementsolutions. com) is CEO of Decision Management Solutions (http://www.decisionmanagementsolutions.com/). This article summarizes the key results of the study. For a full report, as well as a recording of a webinar summary, click here. He is a member of INFORMS.
ANALYTICS
IN THE CLOUD
Figure 4: Decision management transforms results.
sas.com/vademo
Experience Visual Analytics fi rsthand.
Explore billions of rows of data in minutes or seconds, visually represented in a way that brings hidden patterns into plain sight. Then easily create and share reports on the Web or mobile devices.
sas.com/vademo
Experience Visual Analytics fi rsthand.
into plain sight. Then easily create and share reports on the Web or mobile devices.
See your data for all it’s worth.
VISUAL
SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. Other brand and product names are trademarks of their respective companies. © 2013 SAS Institute Inc. All rights reserved. S105651US.0213
