Adobe Target|Optimization Maturity Model White Paper
The optimization maturity model
Know where you are so you can move forward
The journey toward becoming a mature digital optimization organization is one that many start but few complete. That’s not necessarily a bad thing; by its very nature, digital optimization is continuous, with no true end point. Optimization is an organization-wide mindset that involves continuous, iterative improvement of digital experiences to achieve business goals. Yet still, many organizations find themselves stuck at various points along the path toward digital optimization excellence—perhaps aware of the tremendous ROI and qualitative benefits that optimization offers, but unable to determine and execute the next steps forward.
This paper introduces a digital optimization maturity model developed by the Adobe Target consulting group.
The model is based on the experience of over 70 consultants who work with many Adobe Target customers of various sizes across a variety of industries. In the paper, we also:
• Describe what digital optimization is and its relationship to the personalization of digital properties
• Explain why and how the maturity model was developed, including the criteria used to delineate the various maturity levels
• Define the five levels of digital optimization maturity, and discuss the actions an organization might take and characteristics they might exhibit at each level
• Discuss the benefits that customers can expect to receive as they progress through the various optimization maturity levels
• Provide examples of how organizations have transitioned from one level to the next
• Offer concrete next steps that you can take to reach the next maturity level
We conclude with a recommended first step that you can take immediately, regardless of your organization’s current state of digital optimization.
Digital optimization
Digital optimization is a concept or strategy that uses data and iterative design practices to create more efficient, better digital experiences that yield increasingly higher conversion rates. An organization with a culture of optimization uses data to identify areas for improvement on their digital properties, tests ways to make those improvements, and makes frequent and iterative changes based on what works and what doesn’t.
By further aligning these efforts with business goals, the organization maximizes conversion lift and generates greater ROI from all its visitors’ digital experiences.
For an organization to have established a true culture of digital optimization, this continuous, iterative work occurs across all its digital properties and is driven by multiple business units. Doing this requires an optimization tool with robust, comprehensive capabilities. Equally essential is a cultural shift in the organization that involves establishing a new mindset, along with new roles and processes.
While many organizations personalize content across their digital touchpoints to specific visitor types, audiences, or segments, few apply the data-driven strategies of optimization to those personalization efforts.
Optimized personalization means going beyond purely marketer-driven, rules-based personalization to engage in testing and validating segments, content, and locations to provide a truly personalized web experience.
Adobe Target, a comprehensive personalization solution that includes A/B testing, multivariate testing (MVT), audience segmentation, rules-based targeting, automated behavioral targeting, and recommendations, Table of contents
1 Digital optimization 2 Optimization maturity
model 2 Five levels of
optimization maturity 5 Benefits of becoming
a more mature optimization organization 5 How to make progress
and move to higher levels
6 Start your progress right now 6 About the authors
Optimization maturity model
For the past seven years, Adobe Target consultants have worked with numerous clients to help them use the solution’s capabilities to optimize their websites, mobile sites, mobile apps, email marketing, site search, display advertising, and other digital touchpoints. Through these engagements, they have observed varying levels of digital optimization sophistication. Some organizations run just a few simple tests each month, with limited rationale behind what gets tested. Others have detailed optimization roadmaps that span business units and align with overarching business goals. Still others create roles and processes that support the implementation of an optimization roadmap. Some have strong executive sponsorship; others do not.
While these clients vary in the sophistication of their digital optimization, almost all have one thing in common—the desire to understand how creating different digital experiences impacts user behavior, and subsequently, the company’s bottom line. In other words, they want to know how to become a more mature digital optimization organization.
To help evaluate where an organization was in its journey toward becoming a mature optimization
organization, we developed the Optimization Maturity Model, which formalizes our collective observations. By extracting an organization’s tactics and characteristics and relating them to a general level of optimization sophistication or maturity helps inform the next steps that the organization could take to progress along the path to optimization .
The model documents and maps specific criteria associated with an organization’s overall level of optimization maturity. We use it to evaluate an organization’s current maturity level to establish a baseline, make specific recommendations for improvement, and periodically reassess for progress towards its digital optimization goals.
Measuring improvement rather than relying on best guesses is critical to making progress. The maturity model provides an important starting point for answering where you are with your digital optimization. It also lets you establish where you want to be in the future and recognize when you have reached those goals.
The Optimization Maturity Model evaluation is based on a set of weighted criteria. The more mature the organization, the greater the value it receives from its digital optimization efforts.
Five levels of optimization maturity
The model describes five levels of optimization maturity that an organization can reach, with level characteristics ranging from sporadic, ad hoc testing, to establishing an organization-wide culture in which digital optimization is core to any initiative.
Each level is based on a weighted set of assessment criteria—actions an organization takes and organizational characteristics—that collectively indicate an organization’s digital optimization sophistication.
The general categories for the criteria include:
• Culture. Is the organization ready to make data-driven decisions, and is it willing to challenge existing assumptions and the status quo, both within the organization and within its industry?
• Strategy. How well do those involved understand how to approach optimization and use the tools available to improve the digital experience?
• Execution. How developed are the internal processes for optimization, and how quickly can a test idea be executed?
• Leadership. How committed is the program’s executive sponsor, and does that individual have the vision and influence to remove hurdles to establishing an optimization culture?
• Organization. Does the organization have the right people with the right skill sets in place to run the optimization program?
• Reach. Does the organization optimize in the high-value locations of its sites and other customer touchpoints, such as mobile?
Because the model delineates the maturity levels on a combined score, two different organizations may have reached the same level in different ways. However, these organizations have an overall similar level of optimization maturity.
The following sections describe some of the optimization-related activities or characteristics you might see at a given level.
Incidental level
At the incidental level, the organization conducts sporadic optimization activities. At best, isolated teams conduct the activities, but often only individuals do. For example, a team could test two variations of content for a home page banner to visitors who are customers and non-customers. The organization might come up with an idea, set up the test, and then run it. When the results come in, it likely moves on to another test idea. At this stage, the organization lacks the ability to iterate on an idea or operationalize any key learnings from its testing.
These organizations optimize as ideas strike on an ad hoc basis and with no larger optimization plan in place.
The test and targeting velocity is slow, and success metrics often do not align with a specific business goal.
Many companies at this stage struggle to tie optimization efforts to core key performance indicators.
Expanded level
When an organization reaches the expanded level, interest and excitement about optimization grows—it has typically seen and understood that optimization increases conversion lift and revenue. Most likely, ad hoc testing still dominates, but the organization has made a formal commitment to developing a digital optimization program. However, an understanding of what a digital optimization program requires and the ability to execute on those requirements may be limited.
Disciplined level
To reach this level, the organization has adopted optimization as one of its many digital marketing strategies, although the people and processes behind the program might not be operating at high levels of efficiency. A broad range of stakeholders now influences the roadmap, and the organization has begun using analytics reports to guide its testing strategy.
At this stage, the organization wants to run more tests, a variety of stakeholders throughout the organization influence the program, and the program has expanded its reach to other digital channels, such as mobile and social. The organization now begins putting structure around testing and targeting efforts, developing a far more formal optimization program, with a program lead and greater interaction with various business units.
This structure results in an increased testing velocity, greater democratization of optimization with more stakeholders involved, the use of data to drive test iterations, optimization in more locations, and greater value from optimization.
Strategic alignment level
When the organization attains this level, it has aligned the strategic goals of the optimization program with its overall digital strategy. It has also prioritized items on the roadmap to maximize business alignment and value.
At this point, the organization makes a noticeable jump forward in the sophistication and reach of its digital optimization program. It has the permission and support needed to expand its reach to optimize more locations, including high-value ones. It’s also using a breadth of digital strategies—from A/B testing and targeting, multivariate testing and targeting, automated behavioral targeting, rules-based targeting, visitor segmentation, and automated product recommendations.
Rather than relying on a general list of testing ideas, a prioritized, business-aligned roadmap drives the program.
In addition, all stakeholders play an important role in ensuring that optimization meets the overall business goals, and optimization has become truly democratized. Executive leaders have a strong commitment to the program, demonstrated by regular, active participation in it. With their support, individual business unit goals and associated incentives help everyone align to meet optimization program goals and associated business goals.
At this stage, numerous groups have and use optimization technology. Optimization has spread throughout the company and the digital channels. Testing and targeting activities have a good velocity, and they’re also done more thoughtfully, using data to determine where and what to optimize to make the biggest impact on the business. Executive sponsorship ensures that the people, resources, processes, and overall structure required to succeed are in place. The organization is now well on its way to becoming an optimization center of excellence.
Embedded level
At this final maturity level, the organization has embedded an optimization approach into everything it does.
Digital optimization is not a separate program but an integral part of all the activities on the digital strategy roadmap. Digital marketing and all the subsequent customer experiences that it delivers across the organization’s digital properties are accomplished through the data-driven, iterative philosophy of
optimization. Ultimately, this approach delivers maximum business value. At this stage, the organization has become an optimization center of excellence.
Only a few companies in the world have reached this stage. Testing and targeting is a core competitive strategy.
Rather than making a wholesale overhaul of a web or mobile site and then flipping a switch to see what happens, these organizations continuously make iterative changes, always ensuring that what is kept or added meets business goals. Every optimization decision is based on data, and as much as possible, leverages automated, data-driven decisioning. These organizations also use a breadth of optimization strategies and capabilities to optimize their highest value locations.
Removing people and process roadblocks to transition to the strategic alignment level
One of our clients was at the disciplined level. Adobe’s main contact was the lead of the organization’s optimization program, who had committed stakeholders and effective communication with the executive sponsor, defined roles, the headcount to fill those roles, and a roadmap for executing optimization activities.
The program seemed poised for success, but would repeatedly run into roadblocks with getting IT and the user experience (UX) teams onboard, even after receiving approval for releasing winning versions. IT would delay releasing the winning version, and the UX team would water down the test ideas or eliminate testing opportunities based on their own long-held beliefs.
The organization had great ideas that showed incredible wins from testing, but these roadblocks impeded its ability to increase its testing velocity and focus on high-value campaigns. The executive sponsor wanted to increase velocity and asked Adobe Target consulting to help increase the number of test campaigns the company could launch.
We solved this people and process challenge by first defining to the entire cross-functional organization what a best-in-class optimization program looked like. We then assessed where they were and the gaps between their current optimization state and the best-in-class state. One recommendation that resulted from the maturity model assessment was to create a steering committee consisting of cross-functional leads to provide a multidimensional perspective to the optimization program and ensure the program’s success.
By taking this and other recommended actions, the optimization program removed the current roadblocks to success and was able to move the organization into the strategic alignment level. It was not only aligned with the roadmap, but could now execute it successfully.
Benefits of becoming a more mature optimization organization
Almost without fail, organizations increase ROI the further along the maturity level they go. But digital optimization maturity offers additional, more qualitative benefits. As you mature and refine your processes, you can increase your testing velocity, and the quality and value of those tests increase. In addition, as testing spreads throughout the organization, these benefits multiply. When business unit goals align with business- aligned optimization goals, success in one business unit means success for other units.
How to make progress and move to higher levels
In general, we note a “long-tail” distribution of organizations along the maturity model, with many front- loading the incidental and expanded levels, but the numbers dropping off quickly as you move through the remaining three levels. Few have reached that final level. Because we help our Adobe Target clients progress toward higher optimization maturity levels, the majority of our customers tend to fall more in the second and third expanded and disciplined levels.
Clearly, there’s a lot of room for growth and success, especially for the general population. But the question remains: How do you evolve your testing program and progress from one maturity level to the next?
Incidental → Expanded
To make this move, you must generate excitement and buy-in about the incredible potential that digital optimization and personalization offers. Translate that excitement into commitment from leadership to create an optimization program with a defined roadmap. Those involved in digital optimization must help the organization as a whole, including its leadership, see how much value can be gained through digital optimization with the right team and structure.
Clients have accomplished this in a variety of ways; however, the most successful approaches have involved generating excitement by sharing what the program is doing and creating a sense of ownership for and commitment to optimization activities. Some organizations have solicited key stakeholders from other groups to propose test ideas. Some have launched contests to see who could pick the test version winner, tracking winners through multiple tests. Others have sent out emails highlighting successes—a great catalyst for getting people onboard. And some have taken another tactic: They showed how testing averted failure by, for example, demonstrating that marketer intuition or the highest paid person’s opinion (HiPPO) on what content would work actually didn’t.
Expanded → Disciplined
The biggest part of making this transition is removing roadblocks to building a program. To do this, you need to get the staff and processes for optimization in place to execute the optimization roadmap and methodology.
Organizations have made this transition by assigning full or part-time resources to optimization, including individuals who can build the creative assets required for tests. They’ve also made optimization an integral part of the marketer’s job description and provided time and incentives to generate ideas, develop tests, and execute them. Some have set goals for their optimization program and held team members accountable for execution and achievements. Many have tied testing results to ROI for the business and distributed those results and their impact upward in the organization.
Disciplined → Strategically Aligned
Making this transition requires the organization to align the incentives and goals of the optimization team with the overall business goals. Executive stakeholders must firmly participate in the process of determining these incentives and goals and show that they are fully onboard with making this a collaborative, cross-business unit effort. They must also give the necessary permission for optimization to reach more locations, including the organization’s highest value ones. Optimization teams must also rely on analytics reports to help determine where to focus testing and optimization efforts.
In organizations that have made this transition, optimization program leadership has expanded the testing culture to other business units by engaging the unit leaders in defining the optimization roadmap. Program leadership has worked closely with and used the influence of their executive sponsor to align incentives and goals of the various teams to avoid competing and potentially conflicting goals. In many organizations, those involved in digital optimization have refined its associated processes. For example, they’ve used analytics reports to identify and prioritize testing opportunities, formalized result measurement, and formed a digital
Strategically Aligned → Embedded
In the final transition, the optimization leadership must spread and guide the adoption of an optimization philosophy and culture, building an organization-wide mindset that optimization is an essential aspect of every initiative within the organization.
Organizations that have accomplished this transition have expanded a culture of testing upward, through the C-suite and across all business units. Everyone in the organization has jumped onboard with moving away from HiPPO or “expert”-driven decisions to data-driven ones that rely on high-velocity testing and failing-fast to inform optimization decisions. They have now democratized their optimization efforts, with an organizational structure in place that involves all stakeholders in these efforts.
Start your progress right now
Most organizations want to gain the value that digital optimization offers. However, because the concept of ongoing digital optimization is relatively new, a knowledge base of guidance and best practices for building a successful program has not yet been populated. Many see digital optimization as the Wild West.
Based on our experiences, we see a clearly defined path to digital optimization maturity. By using a model based on real, observable criteria related to optimization, we can evaluate organizations to determine how mature—or nascent—they are with digital optimization. That informs next steps to take to progress to higher levels.
Taking that first step—evaluating your current maturity level or having experienced consultants help you do that—is the best possible action you can take to begin your journey toward attaining the highest level of digital optimization maturity and capitalizing on the tremendous value that it offers.
About the authors
Jason Haddock is a consultant in the Adobe Target consulting group. He has expertise helping online retailers build and run their testing and optimization programs with a focus on leveraging automated behavioral targeting to deliver the best possible experience to each site visitor.
Daniel Hopkins is a consultant in the Adobe Target consulting group. He has worked in the Internet industry for over 15 years, is experienced in both business strategy and web technologies, and currently focuses on helping a wide range of companies optimize their online efforts using Adobe Target.
Nik Kreinberg is a consultant in the Adobe Target consulting group. He has worked with Adobe’s top clients to deliver ROI by helping build data-driven optimization programs. He has expertise across all verticals, but focuses primarily on the financial industry.
Ramona Meyer-Piagentini is a senior manager in the Adobe Target consulting group. She has worked with companies ranging from startups to Fortune 100 to realign their business strategies, build out their optimization programs, and become more customer-centric.
Doug Mumford is a consultant in the Adobe Target consulting group. He has expertise in B2B marketing optimization and statistics and focuses on driving a data-centric culture within organizations.
Eddie Ramirez is a senior consultant in the Adobe Target consulting group. He manages the relationships of Adobe’s largest and most strategic accounts, whose analytics and optimization organizations are among the most advanced in the world.
Adobe Systems Incorporated 345 Park Avenue
San Jose, CA 95110-2704 USA
Adobe and the Adobe logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States and/or other countries. Java is a trademark or registered trademark of Sun Microsystems, Inc. in the United States and other countries. All other trademarks are the property of their respective owners.
© 2013 Adobe Systems Incorporated. All rights reserved. Printed in the USA.
