Implementation Details

We tested these algorithms on an IntelHaswell, Corei7-4771 Processor (4 Cores) with an 8MB Cache, 3.90 GHz and IntelHD Graphics 4600. Our work is implemented in C++, and some components use OpenMP and OpenCL toexploit multiple cores. We adopted a agent-level parallelism: individual pedestrian computations are distributed across the CPU cores (except for the motion-model computations, where pedestrian behavior is interlinked and tasks are highly sequential)

3.5 Limitations, Conclusions, and Future Work

We present a novel real-time algorithm for pedestrian path prediction. The main idea is to learn the local and global movement patterns using Bayesian inference. Our approach can handle low as well as high density videos and is useful for short-term and long-term prediction. We have highlighted its performance on many benchmarks and demonstrate the improvements in accuracy over prior real-time algorithms.

Our approach has some limitations. The underlying formulation does not model many other aspects of pedestrian behavior, including physiological and psychological pedestrian traits as well as age, gender or external environmental factors. The estimation techniques relies on Bayesian inferences and that may not work

well in some cases. In terms of future work, we would like to overcome these limitations. Furthermore, we would like to evaluate their performance with robots, e.g. service robots or autonomous vehicles, navigating through pedestrians.

CHAPTER 4

Learning Pedestrian Behaviors

4.1 Introduction

Modeling and classifying the behavior of different pedestrians in a crowd is an important problem in various domains including psychology, robotics, pedestrian dynamics, and behavior learning. Even simple tasks like walking towards a goal position involve several complex decisions such as figuring out the most efficient path or route, and choosing between the various available paths to avoid collisions. According to Convergence Theory(Turner and Killian, 1987), a well-known approach used in sociology and economics, crowd behavior is not a sole product of the crowd itself; rather, it is defined by the individual pedestrians in that crowd. As a result, it is important to accurately predict the behavior of individuals and their interactions with the environment to capture realistic, heterogeneous crowd behaviors.

Recent advances in sensor technologies have made it easier to capture high resolution videos of pedes- trians and crowds. Moreover, surveillance cameras are frequently used in public places and buildings for monitoring human behaviors. In this chapter, we address the problem of classifying the behaviors of different pedestrians in a crowd video based on their movement patterns and use these patterns for crowd behavior prediction. Besides surveillance, these techniques are also useful for architectural design and collision-free navigation of robots or autonomous vehicles in crowded scenarios.

Figure 4.1: Crowd Behavior Learning/Prediction: Our approach can automatically classify the behavior of each pedestrian in a large crowd. We highlight its application for the 2017 Presidential Inauguration crowd video at the National Mall at Washington, DC (courtesy PBS): (1) original aerial video footage of the dense crowd; (2) a synthetic rendering of pedestrians in the red square based on their personality classification: aggressive (orange), shy (black), active (blue), tense (purple), etc; (3) a predicted simulation of 1M pedestrians in the space with a similar distribution of personality traits.

Many factors including biological, developmental, and situational variations, along with individual personalities, govern people’s overall behavior. We mainly focus on capturing the variations in behavior that arise as humans navigate the physical world and avoid collisions. In general, categorizing the variety of personalities that humans exhibit can be hard. Psychologists have proposed different models to represent these variations, but they have some limitations (Harvey et al., 1995). Therefore, we base our classification

onPersonality Trait Theory, which proposes that a wide range of variations in behavior is primarily the result of a small number of underlying traits. It is also important to model many external or environmental factors, including surrounding pedestrians and the crowd’s movement flow for estimation and prediction.

Main Results: We present a novel learning algorithm to classify pedestrian behaviors based on their movement patterns. We extract the trajectory of each pedestrian in a video and use a combination of Bayesian learning and pedestrian dynamics techniques to compute the local and global characteristics at interactive rates. The local characteristics include the time-varying motion model that is used to compute the personality traits. We also present new statistical algorithms to learn high-level characteristics and global movement patterns. We combine these characteristics with Eysenck’s 3-factor PEN model (Eysenck and Eysenck, 1985) and characterize the personality into six weighted behavior classes:aggressive, assertive, shy, active, tense, andimpulsive. We also use the individual personalities to predict the state of the crowd under different environmental scenarios.

To the best of our knowledge, this is the first approach that can automatically identify the behavior of each pedestrian in a crowd. We have evaluated its accuracy with a user study (88.48%) and evaluated its performance on different videos with tens of pedestrians. One example is the large crowd gathered in Washington, DC for the Presidential Inauguration (January 2017) usingPBS HDvideo footage (see Figure 1). We also want to be clear and mention that our definition for behavior is restricted to specific pedestrian level “motion patterns” observed in crowds. Our approach offers many benefits:

1. Robust: Our approach is robust, can account for noise in the pedestrian trajectories, and classifies the behavior using time-varying pedestrian movement dynamics.

2. General:Our approach is applicable to indoor and outdoor crowd videos and makes no assumption about their size or density.

3. Crowd Analysis and Prediction:Our approach can be used to analyze and estimate the future movement or behavior of the crowd. Furthermore, it can be used to predict different scenarios based on the behaviors and global characteristics, e.g., the distribution and density of a large crowd at the National Mall in Figure 4.1.

The rest of the chapter is organized as follows. Section 2 provides an overview of related work in video-based crowd analysis and personality models. We introduce the terminology and present our algorithm for computing the local and global characteristics in Section 3. We highlight the performance on challenging benchmarks and describe results from our user evaluation in Section 4.

Figure 4.2: Our method takes a streaming crowd video as an input. We compute the state of pedestrians in the crowd, as explained in Section 3. Based on the state information, we learn local and global behavior properties, which are combined for behavior classification and prediction.

4.2 Related Work

In this section, we give a brief overview of prior work on video-based crowd analysis, behavior classifi- cation, and personality models.