Richness Analysis of Training Data

To more thoroughly investigate the misbehavior of agent D and the relation to the possible lack of richness in the training data set, agents B and D were presented to different light scenarios and their behavior was recorded. Since the knowledge learned is transparent, the evolved source code was also investigated to better investigate agent D’s misbehavior.

Table 11

Behavior of drivers B and E at the traffic lights changing from green to red Light 2 Light 3 Light 5 Light 6 Light 7 Light 9

Driver B S S S R R S

Driver D S S S R R S

S stands for stop and R for running the light while it’s still yellow

Agent D performs poorly in the validation scenarios when the validation data set is used. To evaluate the performance of agent D further, the performance of agent B is also investigated here. Agent B is of interest since it performs well during validation, and driver B and D had the same qualitative behavior in the training scenarios (i.e. they stopped and ran the same lights) (See Table 11).

To investigate the agents’ performance, they were presented with two different traffic lights: one where they are going straight and one where they are about to make a turn. Furthermore, each light changes its state when the agent is at six different distances to the light: 100, 50, 40, 30, 20 and 10 meters from to the light. At these distances, the light goes from green to yellow; three seconds later, they turn red. Note that there are six runs shown in each diagram in Figure 26 and Figure 27, but many of them are overlaid because the behavior of the agent is the same at certain times, even if the light changes at difference distances.

Agent D - Traffic light driving going straight 0 10 20 30 40 50 60 70 0 10 20 30 40 50 60 70 80 90 100 Distance [m] Sp ee d [ km /h ]

Traffic Light Driving Urban Driving

Agent D - Traffic light driving - turning in an intersection

0 10 20 30 40 50 60 70 0 10 20 30 40 50 60 70 80 90 100 Distance [m] S p ee d [ km /h]

Traffic Light Driving Intersection Driving Urban Driving

Figure 26: Agent D’s behavior at traffic lights

Black dotted lines indicate that Traffic-Light-Driving context is active, red lines that Urban-

Driving is active and yellow that Intersection-Turning is active. The diagrams do not show how the Sub-sub-Context’s Green-Light-Driving and Red-Light-Driving are accessed. When

Traffic-Light-Driving is accessed, it will in some manner access its Sub-sub-Contexts.

Figure 26 shows the performance of agent D. When turning at an intersection, Traffic-Light-

Driving activates and decreases the speed until agent D is closer than 76.5 meters to the light. Then, the Intersection-Turning context takes over and lowers the speed even more, which after a short time of Urban-Driving will revert to Traffic-Light-Driving again. (A Sub-Context needs to release the activation to enable another Sub-Context at the same level to be activated. Hence, the Major Context will temporarily get activated.) Now, the agent will come to a stop at the light, most of the time. When agent D is going straight, the Traffic-Light-Driving will not be activated before 76.5 meters prior to the light. The Traffic-Light-Driving will not lower the speed enough to make a stop in time. In APPENDIX B: SENTINEL RULES OF AGENT B AND AGENT D, the evolved sentinel rules that activate Traffic-Light-Driving for agent B and D are presented. The code there is presented in a structured manner and has been manually cleaned of unnecessary code. The code generated by GenCL is on a single line, but to better understand the code, it was restructured. Branches of code that could never be reached (because of conflicting conditions) were also removed. The code shows that if agent D has spotted a traffic light and plans to make a turn, the Traffic-Light-Driving context immediately activates. If agent D is going straight, nothing is taken under consideration unless agent D is closer than 76.5 meters.

Agent B - Traffic light driving going straight 0 10 20 30 40 50 60 70 0 10 20 30 40 50 60 70 80 90 100 Distance [m] Sp ee d [ km /h ]

Traffic Light Driving Urban Driving

Agent B - Traffic light driving - turning in an intersection

0 10 20 30 40 50 60 0 10 20 30 40 50 60 70 80 90 100 Distance [m] Sp ee d [ km /h ]

Traffic Light Driving Urban Driving Intersection Driving

Figure 27: Agent B’s behavior at traffic lights

Figure 27 shows agent B’s behavior. When agent B is going straight, the Traffic-Light-Driving context activates at 70.6 meters prior to the light, and if the distance is far enough from the light when it turns yellow, agent B will make a stop. When making a turn, the Intersection-Turning

context activates at approximately 45 meters prior to the light and the Traffic-Light-Driving is not activated until agent B is closer than 17.4 meters as shown in the source code in APPENDIX B: SENTINEL RULES OF AGENT B AND AGENT D. This makes room for the Intersection-

Turning context to be activated, which reduces the speed to a level where the stopping rate of the agent significantly increases. Observe that the sentinel rules for Intersection-Turning are not presented in APPENDIX B: SENTINEL RULES OF AGENT B AND AGENT D since the misbehavior is related to traffic light driving.

The interesting result of this analysis is that the poorly behaving agent D doesn’t trigger on the obvious 30 meter mark. Rather, the action triggers on whether agent D is going to make a turn or not. If we compare this to the drivers’ behavior in Table 11, we see that the drivers stopped at all lights turning red except light 6 and 7. Their driving path was so laid out that the drivers were making a turn at all lights except light 6 and 7. Hence, there is a correlation between stopping at the traffic light and whether the driver is making a turn or not. This shows that there is a lack of richness in the training set. If the training set were equipped with a light where the driver stopped but did not make a turn, it would have been more complete.

This analysis actually shows that there was a lack of richness in the training data set, but the lack of richness was not related to the constant distance that triggered the light change as expected. It shows that agent D actually learned that the driver stops at a traffic light if he is going to make a turn at the light. If not, he actually runs all the lights as shown in APPENDIX B: SENTINEL RULES OF AGENT B AND AGENT D. The GP found this relation and learned it. Note that there

are probably other relations in the training sets, such as approaching speed at different distances to the light. These other relations could have been the reason why agent B manages to avoid the relation to turning at the intersection and generalizes better than agent D. The stochastic nature of GP might also influence which correlations in the data set that the learning mechanism will trigger. Also note that the problem addressed here is the lack of richness in training data not how to cope with messy training data.

If the knowledge about the lack of richness in the training data had been present at the initial stage of the training phase, the most reasonable conclusion would be to partition the two data sets and create a new training set partially from the original training set and partially from the original validation set. The remaining two data sub-sets would then be merged to constitute the validation set. This would have overcome this richness problem since there existed occurrences in the validation set where the driver stopped at a light even if he would not turn in the

intersection. Another problem present was the fact that the two data sets were collected four months apart. If both sets had been available at the beginning of the training phase, the chance of discovering this problem might have been greater.