Environment

The vehicle environment is essential for the testing of autonomous vehicle system in simulations (cf. Fig. 2.8). The environment model defines the virtual world as a representation of the real world and contains all relevant objects O for the simulation [Neu14]. It includes the road topologies, vehicles, and pedestrians as traffic, as well as the automated (ego) vehicle. The automated (ego) vehicle moves through the virtual world under consideration of the road infrastructure and other objects O. Neumann-Cosel defines the environment to include all objects o ∈ O within a defined range rE ∈ R around the automated (ego) vehicle (cf. Definition 2.24). This thesis considers the environment model to contain all (relevant) objects within the maximal range rE ∈ R of the vehicle sensors.

Definition 2.23 (Object). An Object o ∈ O, is an atomic entity of the environment

or a combination of atomic entities. The atomicity of objects is subject to the level of detail in the simulation.[Neu14]

Definition 2.24 (Environment). The (vehicle) environment Env is the set of all

objects o ∈ O, whose distances d(E, o) ∈ R to the automated (ego) vehicle E is smaller

than the defined range rE ∈ R: EnvE = {o ∈ O | d (E, o) ≤ rE} (cf. [Neu14]).

The behavior of the automated (ego) vehicle and other traffic participants result in changes for the environment model at specific points of time over the duration of the simulation T : t0, . . . , ti, . . . , tn. Discrete simulations abstract from continuous changes by evaluating the static representations of the environment at these specific points in time

ti. These static representations are denoted as scenes (cf. Definition 2.25) and represent configurations of the vehicle’s environment as an spatial-temporal arrangement from an observers point of view — including the scenery, dynamic objects, and self representation (cf. [Mau00a]). The scene preserves the state z of each object o ∈ O in the environment for a given point in time. The object state zo encapsulates all relevant information about the object o ∈ O for simulation — including its position and orientation. Ulbrich et al. enrich the scene with information about the actors’ and observers’ self-representations (cf. [Ulb+15]).

Definition 2.25 (Scene). The configuration of the vehicle’s environment as a spatial-

temporal arrangement from an observers point of view — including the scenery, dynamic objects, and self-representation (cf. [Mau00a]).

Definition 2.26 (Object State). The object state zo encapsulates all relevant infor-

mation about an object o ∈ O, e.g., position and orientation.

The environment can be separated into two parts: static and dynamic environment. The

static environment is denoted as scenery and is presented in section Section 2.2.1.1.1.

The dynamic environment is describe in terms of traffic in Section 2.2.1.1.2.

2.2.1.1.1. Scenery

The scenery encompasses all geo-spatially stationary aspects of the scene and, therefore, encapsulates all objects o ∈ O whose states zo do not change over time. This entails information about roads, lanes, lane markings, road surfaces, or the roads’ domain types as well as houses, fences, curbs, trees, traffic lights, or traffic signs (cf. [Ulb+15]). Sceneries are modeled once before corresponding simulations and remain consistent throughout these simulations.

2.2. Simulation-based Testing of Autonomous Vehicle Systems

Definition 2.27 (Scenery). The scenery S encapsulates the set of static objects

o ∈ O in the environment EnvE (cf. [Neu14]).

Definition 2.28 (Static Object). A static object o ∈ O does not change its state zo

throughout the complete simulation (cf. [Neu14]).

In addition to the static content of the environment, traffic has to be considered for simulations as the environment’s dynamic objects.

2.2.1.1.2. Traffic

Autonomous vehicle systems require the simulation of the road traffic as dynamic part of the environment. Dynamic objects, e.g., vehicles and pedestrians, have to perform specific maneuvers in order to stimulate reactions by the SUT. A dynamic object is an object ˆo ∈ O which changes its state zoˆ over the duration T of simulations.

Definition 2.29 (Dynamic Object). A dynamic object is an object ˆo ∈ O which can

change its state zˆo for each time stamp ti ∈ T of simulations (cf. [Neu14])

Dynamic objects are placed in the environment model in relation to objects of the scenery, e.g., roads and their lanes, with an initial state z0. The logic for the behavior of dynamic

objects defines changes of the object state zˆo over the duration T of the simulation for each dynamic object ˆo ∈ O. For example, maneuvers are defined in conjunction with

velocities and accelerations for vehicles.

Traffic can be simulated with different levels of detail; as traffic flows with or without individual vehicles, as individuals objects, as individual objects with individual subcom- ponents. The reader is referred to [Sch17] for a detailed description of the different detail levels for traffic simulations.

2.2.1.2. Vehicle

In addition to the SUT, the remainder of the vehicle which is not part of the SUT has to be modeled. This may include vehicle sensors and vehicle dynamics (cf. Fig. 2.8). Models of vehicle sensors address the perception of the virtual environment and its virtual objects for input to the SUT. Sensor models are described in Section 2.2.1.2.1. The models of the vehicle dynamics define how the output of the SUT changes the state of the vehicle in the virtual world. Models of the vehicle dynamics are described in Section 2.2.1.2.2.

2.2.1.2.1. Vehicle Sensors

Autonomous vehicle systems require input from a perception of the virtual vehicle environment in simulations (cf. Fig. 2.8). Real sensors are substituted by sensor models which transfer the state of the environment at each time point ti ∈ T into valid inputs,

e.g., object lists or points clouds, for the SUT (cf. [Tel12]). The input from sensor models will have to be further processed by additional environment perception algorithms if these algorithms are not part of the SUT (cf. image processing and senor fusion in Fig. 2.8). Sensor models may have varying levels of detail ranging from perfect high-level to realistic (low-level) sensor models. Idealized high-level sensor models provide the SUT with the exact information, e.g., position and orientation, of relevant objects in the virtual environment. Realistic low-level sensor models try to mimic real sensor hardware as closely as possible. This imitation commonly includes the imitation of the physical measurement principle of the real sensors in order to provide the SUT with realistic data about the environment — including data jitter. For example, realistic RADAR models may simulate the propagation of radio waves through the virtual environment. Realistic low-level sensor models help to evaluate and improve the robustness of autonomous vehicle systems to jittered input data. [Bel+12; Ste+15]

2.2.1.2.2. Vehicle Dynamics

While sensor models are concerned with the perception of the virtual environment in simulations, models of the vehicle dynamics address changes of the automated (ego) vehicle in the virtual world in response to the output of the SUT. The models of the vehicle dynamics have to mimic the dynamic movement of the vehicle in the real world depending on the level of realism in simulations. Specified models, e.g., the two-track model [Gie+06], have to sufficiently consider the inherent inertia of the vehicle and variances in forces by vehicle actuators, e.g., engine, brakes, and tires, for the longitudinal and lateral movement of the vehicle.

2.2.1.2.3. Driver

Instead of deterministic logic for the behavior of traffic participants (cf. Section 2.2.1.1.2), specific driver models can be introduced for vehicles in the virtual environment. These driver models commonly represent the behavior of real drivers stochastically where each possible driving maneuver is assigned with a possibility. In simulations, the vehicle maneuvers are nondeterministically chosen under consideration of their possibility. Stochastic driver models have the disadvantage that they impede the validation of simu- lation results. Repetitions of simulation with stochastic driver models commonly result in different results because the behavior of vehicle deviates due to the nondeterministic decisions by these driver models.

Hochstaedter et al. separate the driver model into a decision model and an action model. The decision model calculates a trajectory as action intention under consideration of the environment, while the action model sets the setpoint values for the vehicle actuators, e.g., engines and braking system, in order to follow the trajectory. The action model interacts with the model of the vehicle dynamics for the correct execution of the setpoint values. [HZB00]

In the automotive domain, the individual components are predominantly used for XIL simulations. These tests are described in the following section in more detail.

2.2. Simulation-based Testing of Autonomous Vehicle Systems

to environment sensors to vehicle state sensors

Vehicle System (SUT) Emulated Real Simulated d Vehicle Actuators Emulated Real Simulated d Vehicle Dynamics Emulated Real Simulated d Environment Emulated Real Simulated d Traffic Emulated Real Simulated (Vehicle) Perception Emulated Real Simulated d Driver Emulated Real Simulated

Figure 2.9.: Realization of control loop elements in XIL simulations (cf. [Gie+06]).

2.2.2. X-in-the-Loop Simulations

In the automotive domains, autonomous vehicle systems are predominately verified in x-in-the-loop (XIL) simulations because they allow for an efficient and cost-effective testing process (cf. [BNF16]). Autonomous vehicle systems can be systematically tested without disturbances from other connected but unrelated systems.

XIL simulations incorporate simulation models and real system components in varying degree in order to reproduce the control loop between SUT and their environments. As shown in Fig. 2.9 (cf. [Gie+06]), each element of the control loop, e.g., driver, vehicle system, vehicle dynamics, vehicle sensors, vehicle actuators, environment, and traffic, are either represented by a simulation model, emulated by hardware, or implemented by the real system component. Additional disturbances d can be injected for vehicle and environment elements of the control loop. This controlled injection of disturbances d allows verifying the dependability of the SUT (cf. [Gie+06]). The configuration of XIL simulations depends on the SUT and the target of the tests (cf. [Neu14]).

In the following sections, the different types of XIL simulations are described and classified in Kiviat diagrams (cf. [MR82]) based on the way elements of the control loop are integrated in the simulations. The dimensions of the Kiviat diagram corresponds to previously presented components of simulations (cf. Section 2.2.1): vehicle system, driver,

vehicle, dynamics, perception, traffic participants, and scenery (cf. [Sch17; Str12]). The

scale of all dimensions is three folded: (1) simulated by models, (2) emulated by hardware, and (3) as real system component. Some approaches are able to provide multiple solutions for the consideration of system components (dimensions). This is depict by solid areas in the Kiviat diagrams. In addition to the XIL approaches, real-world testing is identically classified for completeness.

The individual dimensions of the Kiviat diagrams are described in the following (cf. [Sch17]):

Vehicle system: The vehicle system represents the SUT and encompasses function

which is verified in the XIL simulation. In case of autonomous vehicle systems, this can be a function on a real ECU, a function emulated by related hardware or a pure software function without any execution hardware.

Driver: The dimension driver addresses the behavior of the driver and its interaction

with the vehicle via the HMI (cf. [Ina06]). The driver behavior can be subject to a real driver, emulated by a driving robot, or simulated by virtual driver models.

Vehicle: The dimension vehicle describes the remainder of the vehicle which is not part

of the vehicle system. This may include vehicle components, e.g., the vehicle chassis. XIL approaches may incorporate real vehicles for realistic results. Otherwise, the vehicle can be emulated by similar vehicles or identical hardware. In software-in- the-loop (SIL) tests, the interaction of vehicle system and the remainder of the vehicle is commonly modeled by rest-bus simulations.

Dynamics: Dynamics describe the behavior of the vehicle on the road. In case of

XIL approach incorporate a real vehicle, the real vehicle dynamics are present. Otherwise, a vehicle with similar dynamics can be used to emulate the dynamics of the original target vehicle. A simulated vehicle dynamics uses a model to process the motion of the vehicle in the virtual world. Thus, the vehicle behavior is not implemented in the real world.

Perception: The dimension perception addresses the remainder of the vehicle’s perception

system which is not part of vehicle system. A real perception will be present if environment perception is implemented on the later target platform. Otherwise, a similar hardware platform can be used to emulate the vehicle perception. Artificial generated object lists or sensor data as input to the vehicle system represent a simulated perception.

Traffic participants: The dimension traffic participants addresses the realization of

other dynamic objects in the tests, e.g., vehicles, cyclists, and pedestrians. Traffic participants can be real if the tests are performed in the real world on public roads or testing grounds. The usage of crash targets or balloon vehicles represents an emulation of traffic participants. Traffic participants will be present in the simulation if they are modeled and implemented as software.

Scenery: The scenery describes the static environment of the vehicle system. Tests in

the real world commonly incorporate the real scenery. An emulation of the scenery will be present if public roads are rebuilt at testing grounds by, e.g., artificial roads, lanes, curbs, and vegetation. In case, scenery objects are part of the virtual world which is generated by a simulation framework; a simulated scenery is present. The various types of XIL simulations are described each in more detail in the following sections.

2.2. Simulation-based Testing of Autonomous Vehicle Systems

2.2.2.1. Model-in-the-Loop Simulations

Model-in-the-loop (MIL) simulations allow the evaluation of initial system designs early in the development process (cf. [BNF16]). A model of the system is integrated into a control loop with models of vehicle dynamics, sensor, actuators, environment, and traffic [GRS14; Ise06a]. As shown in Fig. 2.10, all dimensions of the simulation are modeled and simulated. The model of the system represents the model M in Definition 2.10.

Vehicle system Driver Vehicle Dynamics Perception Traffic par- ticipants Scenery (1) (2) (3)

Figure 2.10.: Kiviat digram of MIL and SIL simulations [Sch17].

2.2.2.2. Software-in-the-Loop Simulations

Software-in-the-loop (SIL) simulations share extensive similarities to model-in-the-loop (MIL) simulations. The model of the as SUT is substituted by its real software im-

plementation (cf. model M in Definition 2.10) [GRS14; Ise06a; Kap+16]. As for the MIL simulations all other entities of the control loop, e.g., vehicle dynamics, sensor, actuators, environment, and traffic, are modeled and simulated (cf. Fig. 2.10) [Sch17]. Novel software implementation can be evaluated in SIL simulations without any real hardware components.

2.2.2.3. Driver-in-the-loop simulations

Driver-in-the-loop (DIL) simulations focus on the behavior of the driver and his interaction with the vehicle and its systems. Therefore, the model of the driver is substituted by a real driver. Human drivers are seated in artificial vehicles on fixed-base (stationary) or motion-based simulators with projections of the virtual environment and vehicle-like controls (cf. [Sch17; Ste+15]).

Most dimensions of driver-in-the-loop (DIL) simulations are simulated — except for the remainder of the vehicle and the vehicle dynamics (cf. 2.11):

• Vehicle dynamics are simulated in fixed-base simulators while a motion-based simulator emulates the vehicle dynamics by a motion platform. This platform reenacts the forces on the vehicle based on the simulation of the environment.

Vehicle system Driver Vehicle Dynamics Perception Traffic par- ticipants Scenery (1) (2) (3)

Figure 2.11.: Kiviat digram of DIL simulations [Sch17].

• The remainder of the vehicle is either emulated by, e.g., a dashboard with steering wheels or a real vehicle is incorporated in these simulations.

Motion-based simulators exhibit different levels of fidelity (cf. [Slo08]). Immovable, fixed-based simulators are categorized as low-level simulators. Mid-level simulators are simulators which accelerated just in one degree of freedom (DOF), which is often y-sled, x-sled or a yaw-table. Any force feedback is commonly applied to the steering wheel. High-level driving simulators actuate the payload, e.g., a real vehicle, in at least six DOFs. In many high-level simulators, the vehicle is surrounded by a dome for the projection of the virtual vehicle environment.