Exactly What Are Video Analytics?
If you’re already familiar with video analytics, we recommend you skip ahead to the next section. However, if video analytics is a new concept to you, please read on.
Video analytics (also know as video content analysis or VCA) is a combination of two technologies: machine vision (also known as computer vision) and data analysis engines. Put simply, machine vision is technology that allows a computer to recognize objects or activities captured by a camera.
In a video analytics system, the computer then applies data analysis to process, categorize and analyze the objects and activities captured by the machine vision. The result is visual intelligence not unlike a human’s ability to see objects and activities and then ascribe meaning to them. Video analytics systems have been developed with a number of common capabilities. These capabilities typically form the basis for the types of analyses that these systems can perform. The typical capabilities of a video analytics system are as follows:
Motion Detection: The ability to sense movement. Motion detection can cover the entire view detected by a camera, or it can be
Trip Wires: Trip wires sense when a moving object crosses a boundary set within a camera’s view.
Object Counting: Object counting systems note how many moving objects pass through a defined area within a camera view.
Object Recognition: The most advanced video analytics systems available can recognize objects in a camera view. For instance, facial recognition systems look for a pair of eyes and a nose, and the measurement of the space be-tween the eyes and the nose, to identify a face. The computing power required for the above capabilities varies based on the type of analysis that needs to be done. In some cases, the video analysis can be performed on the camera with a small, embedded computer. This is often referred to as video analytics at the edge. In other cases, the processing required for video analytics systems requires a powerful computer. For these systems, the cameras simply capture and transmit video to the remote computer, where processing is done at the core. In a few cases, a hybrid of both is used for efficiency.
Since their introduction over two decades ago, video analytics have offered the promise of
revealing powerful insights in a number of areas, from security and surveillance to online
video and bricks and mortar retail.
Retailers, in particular, have continually sought out video analytics solutions to help them better understand customer experience and store execution factors that have typically been difficult to measure and analyze. Yet, for the most part, many of the video analytics solutions that have been utilized by retailers to date have either performed poorly, or have not provided usable data for the pioneers who have tried them.
In this white paper, we’ll outline how past and current implementations of video analytics have failed, and how new technologies are now finally allowing retailers to realize the value of video analytics within their stores.
Computerized Video Analytics
Versus Hybrid Analytics
Until recently, nearly all video analytics systems have used a 100% computerized approach to capturing insights from video. The primary advantage of this methodology is the ability to perform extremely large numbers of simple video analysis tasks rapidly. However, these systems suffer from a number of shortcomings that have limited their utility in environments such as retail. The primary limitations are as follows:
Sensitivity To Environment Changes: Computerized video analytics systems often fail in real world applications because of both unforeseen and planned environmental changes. For instance, something as simple as a shadow or a sign moved by a breeze can confuse motion detection and object recognition systems by changing the expected environment. Similarly, planned changes to an environment, such as moving a display in a store, require a video analytics system to be recalibrated to the new environment to work properly.
Precise Placements And Angles: To work properly, fully computerized video analytics systems often require specific placements and angles for cameras to capture video that can be analyzed. This can result in cameras that are obtrusive and inconvenient to the people that visit the environment being analyzed.
Limitations Of Object Recognition: Computerized video analytics systems are often tasked with recognizing objects to a degree that exceeds their capabilities. For instance, past implementations of systems in retail environments have been tasked with tracking shoppers throughout a store. However,
100% computerized systems cannot discern a shopper from an employee, and therefore the data these systems deliver fails to meet the requirements of the retailer.
Severe Processing Requirements: Many of the most useful video analytics applications require significant processing power that can only be provided by costly servers running complex software. In many cases, this can be impractical for large-scale implementation of video analytics systems from both a cost and technology perspective.
As a response to the above limitations, a number of innovative companies are now introducing hybrid systems that augment the capabilities of computerized analytics with highly efficient intervention by human video auditors at key points during processing. In essence, these systems use computers where they work, and humans where they don’t.
For instance, by using a hybrid computerized/human video analytics system, the above limitations can be mitigated:
Environmental Changes: Targeted human intervention can help computerized systems quickly recognize and ignore common visual “noise” that would otherwise create false data. In addition, by adding a human auditing element, planned environmental changes can be accommodated without significant recalibration of on site cameras and processing equipment.
Angles And Placement: Adding a human auditing component can compensate for the error that may occur when a computerized system is required to capture data from an imperfect placement or angle.
Object Recognition Limitations: Where computerized systems can become confused by the difference between a shopper and an employee, for instance, humans can easily sift through large volumes of video to identify employees and correct the errors made by a computerized system.
Processing Requirements: By introducing human auditing to perform tasks that would otherwise require massive amounts of
processing power, a hybrid system can simplify the equipment required for highly specialized video analytics tasks.
Of the hybrid systems available today, the amount of human intervention used can vary from very occasional data verification to extremely extensive involvement where the balance of data that comes from the system is primarily created by the human auditors’ analyses.
There are a number of technology companies using a hybrid video analytics system today to solve the problems of their retail customers. The use cases for this type of hybrid
process include:
Cash Theft: Helping retailers discover and eliminate cash theft at registers
Sweethearting: Helping supermarkets and convenience stores eliminate product theft and sweethearting in checkout lanes Consumer Behavior: Providing advanced analysis of consumer behavior in retail stores
Implementing Video Analytics
In Retail Environments
Until recently, most attempts to implement video analytics in bricks and mortar retail environments have failed either because of technical limitations, or an inability to deliver data that is meaningful to retailers.
In our extensive experience designing, installing and optimizing video analytics systems for retailers, we have found that there are three main criteria that must be met for a video analytics system to make sense for a retailer.
The first criterion is scalability.
Scalability means that the solution can be practically used in every location in a chain on every day of the year. Many systems fail this criterion because they either are too costly to roll out across a chain or across time, or the data they deliver reaches a limit in value once installed in a limited subset of locations for a limited amount of time.
An example of a scalable video analytics solution would be perimeter traffic counting. It is cost effective to install in every location and use every day, and a retailer can derive benefits from having this data in every location and on every day.
An example of a non-scalable video analytics solution would be video analysis of a
merchandise shelf set or a specific display. It is not cost effective to install in every location to monitor such a small store area year round.
The second criterion is repeatability.
Repeatability means that the solution delivers a similar data set from each location, regardless of differences from location to location. The similarity in data makes it easy to compare the performance of different locations against each other, as well as against a common benchmark.
An example of a repeatable video analytics solution would be queue analysis (also known as speed of service or line measurement). Even if two stores have different layouts, they will still have a register with a line that can be measured to determine speed of service. They can then be compared against each other as well as against a chain-wide benchmark.
An example of a non-repeatable video analytics solution would be shopper path tracking. If two stores have different layouts, then customer paths through those stores will be different by nature. They then cannot be compared against each other, because they represent two dissimilar environments.
The third criterion is actionability.
Actionability means that the solution delivers information that is easy to act upon. This can include one or multiple levels, from store manager to regional manager to corporate manager and executive.An example of actionable video analytics is the measurement of customer and salesperson interaction. If, for instance, a correlation is found between the time a salesperson takes to approach a customer and the resulting conversion rate of browsers to buyers, a store manager can instruct salespeople to approach customers sooner. This type of change is easily implemented and optimized.
An example of non-actionable video analytics is measuring the dwell time in front of a fixed display. While the data may eventually lead to the display being moved, the effort involved in changing the location of a fixed display in every store in a chain is extremely large and may be too costly to justify. In a scenario like this, it would only make sense to install a video analytics system in a smaller subset of stores to test the difference in consumer dwell time between two display set ups.
Many companies that currently offer video analytics systems fail to help their customers understand which (if any) of their services pass through all three filters of scalability, repeatability and actionability. Customers that don’t carefully assess these three perspectives may find
themselves investing in a costly rollout of a system that can only provide limited benefits from a very limited number of scenarios.
In particular, many companies that offer video analytics have recently begun to suggest that their systems can deliver the same types of analytics and metrics available to eTailers via online analytics systems. As we’ll show, this comparison can be a dangerous one, and must be carefully assessed.
Limitations In Applying
“Online Analytics” Concepts
In Offline Environments
Over the past couple of years, a number of retail analytics companies have begun to tout systems that promise the delivery of “online analytics for offline stores.” While this premise has proven to be exciting for bricks and mortar retail executives who see the constant growing threat of online retail, there are a number of reasons why the promise of this premise will never come to full fruition.
First and foremost is the reality that an online store represents one single location that is easy to monitor, easy to test and easy to change. Bricks and mortar retail, however, is often comprised of chains with hundreds or even thousands of units spread across large geographic areas.
To monitor the performance of an online store, one need only install a small snippet of code to enable an extensive suite of powerful analytics, many of which are available for free. To monitor the performance of an offline chain of stores requires much more significant time and expense. One or multiple devices must be installed in every location to capture performance data. Each device must be continually monitored to ensure uptime for consistent data. Data from other technologies such as point of sale systems and labor management suites must often be integrated to provide expanded sets of metrics.
To test different strategies in online stores, a free program like Google Web Optimizer allows an eTailer to easily perform a side-by-side comparison of two or more different strategies to gauge the impact on key metrics. To test different strategies in offline stores again requires much more significant time and expense. Test and control stores must be selected, a wide variety of participants from corporate levels down to store employee levels must participate in execution and subsequent changes to be made after each test require an even greater level of effort in successful execution across an entire chain.
This introduces perhaps the greatest way in which online stores differ from offline stores: introducing change. Much like with enabling the monitoring of the store, executing changes in layouts and merchandise for online stores typically requires very easy-to-execute actions that can often be done by a single person through a web based portal. In the offline world, the reality is
much messier. Making changes to layouts or merchandising requires significant coordination and effort from a wide number of participants at every level within the chain.
What does this all mean for retailers considering the implementation of video analytics? It means that one should take the claims of companies that suggest they can enable online analytics in offline environments with a grain of salt. In fact, the filter by which any retail executive should consider the implementation of video analytics has already been described above. For a video analytics solution to make sense in a bricks and mortar retail environment, it must be scalable, repeatable and actionable. Many of the features and capabilities that make analytics so powerful online simply can’t be executed realistically in the offline world. Therefore, it makes sense to rationalize one’s vision for video analytics based on the filters of scalability, repeatability and actionability.
Using Video Analytics Within The
Retail Organization
Now that we have established what is and isn’t realistic in applying video analytics to retail, how can a retailer effectively apply what is possible? There are, in fact, a number of ways that video analytics bring truly powerful insights to different departments and roles within the retail organization.
Marketing: Until recently, marketers’ ability to track the performance of media and promotions in driving in-store traffic, sales and brand recognition was largely limited to analyzing sales and traffic trends. Now, however, new analytics capabilities can increase the granularity by which marketers can attribute performance to different media choices and campaigns.
Demographics: Understanding the
demographic makeup of shoppers at stores across a chain or at particular locations assists in determining the success of campaigns designed to engage target segments. Category-Level Traffic: For campaigns that are tied to a specific product or category, understanding traffic at a category level provides much more granular data to tie traffic and sales to campaigns.
Merchandising: Category managers, merchants and brand managers currently rely on store traffic and sales to understand conversion rates for categories and specific products. Similar to marketers, new analytics capabilities allow for a more granular and powerful understanding of shopper behavior.
Demographics: For merchandisers who deal with gender- and/or age-specific products, understanding the demographic makeup of shoppers at all levels – from a single store to chain-wide – assists in making decisions related to mix, layout and promotion.
Category-Level Traffic: By gaining deeper insight into traffic within specific categories, merchandisers can diagnose and correct issues that impact conversion.
Operations: Operations can gain a number of new insights around store execution and customer experience that were previously difficult to capture or impossible to capture.
Speed of Service: For retailers who rely on rapid service as a key factor in delivering customer satisfaction, understanding queues at key service points such as cash wraps and customer service desks is essential.
Salesperson Performance: Retailers that use in-store associates or dedicated salespeople to assist shoppers often have little understanding of how these employees help – or sometimes hinder – conversion rates.
Category-Level Traffic: Understanding where shoppers spend time in stores can help in determining when and where employees are staffed to ensure the best possible customer experience.
In addition to the different corporate departments that benefit from the application of a well-designed video analytics system, these systems can benefit those in management positions at a regional and store level as well.
Regional Managers: While sales and store traffic data help regional managers understand which locations are drawing and converting customers, there are other factors that they can now access which can reveal the reasons behind over- and under-performing locations. For instance, benchmarking a customer experience factor like speed of service allows an RM to capture best practices from top performing stores and share them with lower performing stores to raise customer satisfaction across a region.
Store Managers: Store managers can gain greater insight into the performance of staff across shifts in providing a better customer experience. Using a metric like the percentage of customers assisted by salespeople can help in identifying shifts that would benefit from additional training.
When looked at holistically, then, there are clearly a number of ways in which video analytics can be useful across a broad number of roles within the retail organization. Coupled with a management dashboard that translates the data captured by the video analytics system into insights, a well designed system becomes a decision support tool that surfaces easily understood action items that assist each department in their day-to-day activities.
New Video Analytics-Based
Services Available From
ShopperTrak
For any retail video analytics system to be successful, a common denominator must be
compared. Store-level traffic counts provided by people counters at every door provide that common denominator. In particular:
Store-level traffic is a pervasive metric used by nearly every forward thinking, data-driven retailer worldwide
Store-level traffic provides a baseline count of all customers within a retailer environment that any other metric can be easily compared against. For instance, when measuring the number of customers intercepted by sales associates, the obvious comparison is against all customers present in the store.
Once a retailer has implemented a scalable and accurate perimeter traffic count system, then implementing video analytics to further understand and optimize their stores can be done.
In developing video analytics solutions for retailers, ShopperTrak has carefully chosen a set of services that adhere to the need for scalability, repeatability and actionability.
Demographics-Augmented
Traffic Counts
Demographics-augmented traffic counts allow retailers to further understand the actual demographics of their customer base at each individual location, including gender, age range and group size.
By capturing this information, retailers can tailor merchandising and promotions to precisely match the makeup of shoppers in every location. They can also analyze the impact of gender-specific out-of-store marketing campaigns in driving traffic from specific customer segments. Finally, stores that operate gender-specific categories (for instance, clothing retailers) can staff departments based on
While video analytics in retail have been around for years, it has only been recently that they have begun to provide tangible value to retailers that choose to implement them. There have been a number of learnings from the attempts of early pioneers with this technology that have directed many (but not all) current analytics providers to offer a more rational and practical set of solutions based on video analytics.
By adhering to the filters of repeatability, scalability and actionability, retailers can choose to implement video analytics systems that can be implemented year-round and in every location to deliver valuable data and benchmarks that ultimately lead to significant enhancements in customer satisfaction and an increased conversion rate.
Queue Analysis
The queue analysis services offered by ShopperTrak provide retailers with detailed information on average line lengths at key service points, and also deliver staffing recommendations based on historic data, traffic counts and proprietary algorithms.
The benefit to retailers include increased customer satisfaction from shorter queues, and optimized labor costs by correctly matching staffing plans to the need for open registers.
Sales Performance Analysis
The third video analytics service to be offered by ShopperTrak is sales performance analysis, which measures the typical time it takes for a sales associate to intercept customers when they enter a defined area, as well as the percentage of customers that are assisted by a sales associate in that defined area.
Retailers that monitor in-store sales performance typically benefit from understanding which staffing levels and sales strategies will maximize impact on customer satisfaction and conversion rates. In addition, all stores can be benchmarked against best practices to ensure that every location maximizes the impact of their sales associates.
Future Offerings
ShopperTrak has an active pipeline of new video analytics services in development. Each service is tested against the three filters of scalability, repeatability and actionability. In addition, each service is developed to enhance the benefit ShopperTrak customers derive from perimeter traffic counts provided by ShopperTrak’s Managed Service people counting solutions.
Adam Rodnitzky is Director of Product Marketing at ShopperTrak. Prior to joining ShopperTrak, he was a Co-Founder and Director of ReTel Technologies, Inc., an advanced video analytics company focused on revealing unique insights from retail and restaurant environments.
