– Goldhoorn, A., Garrell, A., Sanfeliu, A., and Alquézar, R. (2016c). Un nuevo método cooperativo para encontrar personas en un entorno urbano con robots móviles. In XXXVII Jornadas de Automática, pages 206–213, Madrid, Spain.
– Goldhoorn, A., Garrell, A., Sanfeliu, A., and Alquézar, R. (2017). Searching and Tracking People with Cooperative Mobile Robots. Conditionally accepted in the Autonomous Robots journal.
1.4 Thesis Overview
In this thesis, the search for a search-and-track method for real mobile service robots is discussed, including the encountered problems. This thesis exists out of several chapters; here, an overview of the contents of each chapter is given:
• Chapter 1 gives an introduction of the problem, indicating the motivation, ob-jectives, main contributions, problem constraints and derived publications.
• Chapter 2 reviews the state of the art of search-and-track, search, track, hide-and-seek, and related games and problems.
• Chapter 3details the experimental settings, such as the usedrobots, the environ-ments where the experienviron-ments were done, and the used hardware and algorithms.
• Chapter 4defines our firsthide-and-seekmethods using anMOMDPfor a small discrete map. Also a Hierarchical MOMDPmethod is presented, which reduces the amount of states by projecting a group of states to a top state. The method was tested in simulation and in a small outdoors environment with real people.
• Chapter 5presentssearch-and-trackmethods that are able to work in large en-vironments and in continuous state space. To handle these larger environments, a RL method—that does Monte-Carlosimulations—was used: Partially Observ-able Monte-Carlo Planning (POMCP). We adapted the method for continuous space and we run iton-line, resulting in the Continuous Real-time POMCP (CR-POMCP). When testing the RL method with the real robot, several problems were discovered, which resulted in adaptations of the method where the probabil-ity map (belief) of thepersonlocation was used to find the person. The location with the Highest Beliefwas used asgoalfor therobotto go to. A second method
1.4 Thesis Overview
to maintain a probability map is based on the Particle Filter (PF). All meth-ods were tested extensively in simulations, and the best were tested in real-life experiments in large urban environments.
• Chapter 6 presents a multi-agent method that is able to search and track co-operatively, and at the same time is able to work on an individual level, thereby, making it more robust. The method explores the most probable locations of the person, according to thebelief. Simulations with up to fiveagentswere done, and real-life experiments with twomobile robotsshowed real cooperative search-and-track behaviour.
• Chapter 7 gives the thesis’ conclusions and future work.
Chapter 2
State of the Art
Search-and-track is an importantrobotbehaviour, for example, for Human Robot In-teraction(HRI),search-and-rescue, RoboCup soccer and theoretical games like pursuit-evasion. For example, [Satake et al.,2013] proposed a method for a mobile socialrobot to approach people that were not busy. Kanda et al. [2007] focused on estimating the relationship between children, using the time they spent together, and they tried to generate a long-term interaction with the children by calling them by their names, adapting its interaction to the child, and confiding personal matters. Mitsunaga et al.
[2008] tried to learn correct interaction behaviour per person, such as interaction dis-tance, gaze meeting and motion speed. They made use of Reinforcement Learning (RL) with the human’s unconscious signals as feedback, such as gazing and movement.
Search-and-rescue focuses on tasks to rescue people from disaster areas, as explained in subsubsection 2.2.1.1.
In this thesis, we present methods to do search-and-track with mobile robots in a real-life urban environment. Searchingandtrackinghas been researched for many years, but much less research has been done in the combination,search-and-track. Therefore, we discuss the work done in searching and tracking separately, and thereafter, the combination. First, we comment the different characteristics ofsearch-and-trackrelated tasks inSection 2.1, then, we explain shortly the different search game variants. Finally, singleagentandmulti-agentapproaches tosearchandtrackare discussed in Sections2.3 and 2.4 respectively.
2.1 Taxonomy and Characteristics
Figure 2.1: The different parameters for autonomous search models, as indicated by Chung et al.. [Taken from Chung et al.,2011]
2.1 Taxonomy and Characteristics
There are a large number of tasks and games related to search,track and search-and-track. Many approaches to solve these tasks are commented in the surveys of [Chung et al.,2011] and [Robin and Lacroix,2016]. In the survey of [Robin and Lacroix,2016], they give a taxonomy of the differentsearch-and-tracktasks. Following this taxonomy, the main focus of this thesis is Probabilistic Search for thesearchingpart, and Following for the trackingpart.
In the survey of [Chung et al., 2011], they set out the different parameters of the different autonomous search models, as shown in Figure 2.1. The parameters can be divided in searcher, target and environment. The searcher is defined by the number of agents tosearch, their motion model and their sensor model. The environment model can be discrete or continuous, and there can be one or moretargets. Finally, thetarget motion model is very important for the search; atarget can be a stationary or mobile object or a person. Also the target’s behaviour is important, a targetwhich wants to flee (i.e. adversarial) is more difficult to catch than atargetwhich moves independently from theseeker(non-adversarial). The characteristics for thesearch-and-trackmethods presented in this thesis are listed in Table 2.1.