Master’s Programme in Machine Learning and Data Mining
Alejandro L´opez Vidal
Traffic flow simulation and
optimiza-tion using evoluoptimiza-tionary strategies
Master’s Thesis Espoo, June 7, 2011
Supervisors: Doc. Timo Honkela, Ph.D., Chief Research Scientist, Aalto University
Professor Yago S´aez Achaerandio, Universidad Carlos III de Madrid
Master’s Programme in Machine Learning and Data Mining MASTER’S THESIS Author: Alejandro L´opez Vidal
Title:
Traffic flow simulation and optimization using evolutionary strategies
Date: June 7, 2011 Pages: xii + 88
Professorship: Information and Computer Science Code: T-61 Supervisors: Doc. Timo Honkela, Ph.D., Chief Research Scientist
Professor Yago S´aez Achaerandio
Instructor: Doc. Timo Honkela, Ph.D., Chief Research Scientist
Different studies, such as the survey that IBM yearly conducts about commuting to work, verify the importance of a well known problem: traffic congestion in big cities. Solving this problem has concerned professionals from many scientific and technological disciplines, including physics and artificial intelligence (AI). AI researchers have contributed to mitigate the problem by adapting their techniques to control traffic, obtaining remarkable results. An important tool in AI is the use of Artificial Neural Networks (ANN), whose numerous features make them a suitable technique in a wide range of problems. However, we found that the use of ANN to control city lights has never been fully seized by any of the past researches, in our opinion, as a consequence of the learning process adopted in their approaches, that was not adequate due to the nature of the problem. This thesis studies the effect of different neuroevolutionary methods in adapting ANNs to efficiently control traffic semaphores. These methods include biological, cultural and linguistic evolution. Furthermore, the performance of this methods is compared with previous approaches using a microscopic traffic simulator, which was enlarged in order to include different realistic scenarios in a square shaped city. The study has been implemented using a combination of Java language, Netlogo social simulation environment and Matlab®.
The results of this work illustrate the potential of the adaptation of neuroevo-lutionary concepts to control systems, which opens the door to further research in the topic and possible expansions to other research areas that includes control systems, such as decision support systems in air traffic control or harbor control. Keywords: Traffic flow, Traffic light, Social simulation, Evolutionary
al-gorithm, Neuroevolution, Biological Evolution, Cultural Evo-lution, Language emergence, Cellular Automata, Learning Al-gorithms, Artificial Neural Network
Language: English
Writing a thesis is not an easy task, and I would not have been able to do it if it was not for the support of many people, not just during the process itself, but also during my whole education. First, I want to dedicate this work to my family, specially my parents Alejandro and ´Elida Josefa, and my sister Gloria, because without their support and guidance during all these years, I would not be the person I am today. I also want to express my gratitude to all my teachers, specially those who were excel in their work and passed me more than just knowledge but also values that make me who I am. I would like to express a special thanks to my supervisor Dr. Timo Honkela, who directed this work and always believed in me. I certainly do not forget my friends and colleagues, specially Alberto L., Alberto S., Andr´e, Ant´onio, Isabel, Katariina, ´Oscar and Paloma, who not only have supported me and been with me both the good and the bad times, but also contributed to this work with their invaluable comments. As a final note, I would like to apologize to all of those who I inadvertently omitted in this acknowledgment, because I cannot possibly name everyone that contributed significantly to this work.
Espoo, June 7, 2011 Alejandro L´opez Vidal
ANN Artificial Neural Network EA Evolutionary Algorithms
GSM Global System for Mobile Communications MLP Multilayer Perceptron
IL Inductive Loop
ITS Intelligent Transportation Systems SLD Single Inductive Loop Detector DLD Double inductive loop detector
CLO camera Linkerover
UML Unified Modeling Language SSH Secure Shell Network Protocol API Application Programing Interface VNC Virtual Network Computing SFTP SSH File Transfer Protocol
GNU GNU’s Not Unix
GPL GNU General Public License
JVM Java Virtual Machine
ECA Elementary Cellular Automata
αi coefficient of drag of vehiclei, depending on vehicle’s cross-section and
its aero-dynamic shape. ¯
vf f the free-flow speed
β dimensionless slope of the guideway, considering that sin (α)≈tan(α) being α is the angle of the slope
ρi the occupancy time of vehiclei
τi reaction time of the driver from vehicle i
ai the acceleration of vehiclei
f dimensionless coefficient of friction (about 0.3 for cars and buses) Fb braking resistance force
Ff fluid resistance force
Fg guideway resistance force
fn friction coefficient between the wheels and the ground, a suitable
ap-proximation of this coefficient is shown in Eq. 2.5 Fp propulsive force
g acceleration of gravity (about 9.8m/s2) gri the time gap of vehicle i
gsi the space gap of vehicle i
hsi the space headway of vehiclei
hti the time headway of vehiclei
k density of the traffic flow
ki power to weight ratio of the vehicle i
kj the jam density
li length of the vehiclei
mi mass of the vehicle i
N number of vehicles in the measurement region
oti on-time. Time period the ivehicle is above the detector.
P Lenght of the population qcap capacity flow
qout the outflow from a jam to a queue discharge capacity
Rs space region of measurementK during an instant dt
Rt,s region of measurements corresponding with a region of the road K
during a given time Tmp
Rt region of measurements corresponding with a point in the road dx
during a given time Tmp
ti time instant of the passing of vehiclei
Tmp time of measurement
vi velocity of the vehiclei
vr vehicle speed relative to the fluid
w the characteristic/kinematic wave speed xi position of vehiclei
h hour
Km Kilometer
m meter
s second tic ticks
Abbreviations and Acronyms IV
1. Introduction 1
1.1. Definition of the problem . . . 2
1.2. Scope of the thesis . . . 3
1.3. Layout of the thesis . . . 3
2. Traffic flow theory 4 2.1. History . . . 4 2.2. Microscopic . . . 6 2.2.1. Car-following model . . . 9 2.3. Macroscopic . . . 11 2.3.1. Density . . . 13 2.3.2. Flow . . . 14 2.3.3. Mean speed . . . 15 2.3.4. Occupancy . . . 16
2.3.5. Fundamental relation of traffic flow theory . . . 16
2.3.6. Shock waves . . . 17
2.4. Fundamental diagrams and empirical measurements . . . 17
2.4.1. Space-mean speed versus density . . . 19
2.4.2. Flow versus density . . . 19
2.4.3. Space-mean speed versus flow . . . 20
2.5. Traffic flow regimes . . . 20
2.5.1. single-regime models . . . 21
2.5.2. multi-regime models . . . 21
2.5.3. 3 states theory . . . 22
2.6. State of the art . . . 23
2.6.1. Marching . . . 23
2.6.2. Green wave . . . 23
2.6.3. Self organized algorithm (SOLA) . . . 24
2.6.4. Manual tuned system . . . 26
3.1.1. Model and topology . . . 29 3.1.2. Training . . . 31 3.2. Evolutionary Algorithms . . . 33 3.2.1. General process . . . 35 3.2.2. Neuroevolution . . . 36 3.2.3. Cultural Evolution . . . 39 3.2.4. Language Emergence . . . 40 3.3. Simulation . . . 41 3.3.1. Characteristics . . . 42 3.4. Software tools . . . 43 3.4.1. Java5.0 . . . 43
3.4.2. Neural Network Toolbox in Matlab® . . . 44
3.4.3. NetLogo . . . 45
3.4.4. Other software . . . 45
4. Development 47 4.1. Workbench: Overall architecture . . . 47
4.1.1. UML Class Diagram . . . 47
4.1.2. Example of an experiment . . . 49
4.2. Scenarios . . . 53
4.2.1. Simple city . . . 55
4.2.2. City with four directions . . . 59
4.2.3. City with four directions and turns . . . 59
4.2.4. Fitness Function . . . 60 4.2.5. Plan . . . 61 5. Results 63 5.1. Simple city . . . 63 5.1.1. Biological evolution . . . 63 5.1.2. Cultural evolution . . . 65 5.1.3. Evolving communication . . . 67 5.2. Other scenarios . . . 69
5.2.1. City with four directions . . . 69
5.2.2. City with four directions and turns . . . 69
6. Discussion and Conclusions 72 6.1. Discussion of the results . . . 72
6.2. Validation of the NetLogo simulator . . . 72
6.3. Future work . . . 73
6.3.3. Implement the solution . . . 74 6.4. Conclusions . . . 74 A. Possible Combination of Lights 81 B. Structure of a Network File 86
2.1. Mathematical formulation of the car-following model. . . 5
2.2. Greenberg’s relation for the traffic flow. . . 6
2.3. Rules of the SOLA algorithm. . . 25
3.1. Back-propagation algorithm in pseudo-code. . . 32
3.2. General evolutionary algorithm in pseudo-code. . . 36
4.1. Principals measures in the simulator . . . 55
4.2. Rules of the cellular automaton that simulates the simple city 59 5.1. Summary of the experiments realized with the simple city. . . 64
5.2. Summary of the experiments to obtain the best value for the variance in the cultural evolution. . . 67
5.3. Summary of the experiments realized with the city with four directions. . . 69
5.4. Summary of the experiments realized with the city with four directions and turns. . . 70
2.1. Main properties of a car and the interactions with the leading
vehicle. . . 9
2.2. Trajectories of vehiclesi and i+ 1 with constant speed vi. . . 10
2.3. Results of a simulation of 2 cars using a formulation of the car-following model. . . 12
2.4. Principal measures used in macroscopic models. . . 13
2.5. Fundamental diagrams of the Greenshields theory and scatter plots of real data. . . 18
2.6. qe(k) diagrams representing an inverted-lambda shape and a Kerner’s three-phase traffic theory. . . 21
2.7. Principal measurement areas around a traffic light. . . 25
3.1. Multilayer perceptron . . . 30
3.2. Encoding a neural network on a Chromosome . . . 38
3.3. Imitation learning with 2 neural networks . . . 40
4.1. UML Diagram of classes in the main program. . . 50
4.2. UML Diagram of classes in the node . . . 51
4.3. Sequence diagram of an experiment . . . 52
4.4. Sequence diagram of a simulation . . . 54
4.5. Example of the 3 scenarios and cellular automata simulator . . 56
4.6. Rules applied in an intersection regulated by a traffic light. . . 60
4.7. Plan for controlling the density of cars . . . 62
5.1. Results of the evolution in the simple city. . . 65
5.2. Results of the direct imitation . . . 66
5.3. Results of the cultural evolution in the simple city. . . 68
5.4. Results of the evolution in the city with four directions. . . 70
5.5. Results of the evolution in the city with four directions and turns. . . 71
Introduction
“Life is too short for traffic”—Dan Bellack [26]
Traffic congestion is one of the most acute problems in big cities. Capitals and big cities all over the world have rates of over 64% of people commuting to work by car, e.g. Stockholm, Toronto, Johannesburg, Melbourne, New York and Los Angeles. The average time spent on going to work is 32 minutes, covering a distance of about 21 km. In addition, Beijing and Mexico City, despite their high bus rates, are the two cities with the most painful commutes according to the survey carried out by IBM [26]. In general, the participants of the survey felt that the traffic situation had become worse or a lot worse in the past 3 years, with 20% of them saying that it had not improved at all, and a modest 5% stating that the condition had improved substantially.
Although the information provided in the previous claim might be suf-ficient to justify the investigation of new ways to improve traffic flow, and hence the overall efficiency of the commutes, the same survey released addi-tional interesting information. Participants of the study reported that their stress and anger had been incremented by traffic when they were asked to tell how traffic had disturbed their lives. They also recognized that their pro-ductivity in school or at work was affected. Furthermore, 38% of the polled (8,192 people) declared to have canceled a planned trip due to anticipated traffic. [26]
Nevertheless, these subjective opinions are not the only consequences of congested traffic. There are other issues that have to be taken into account, such as the increase in greenhouse gas emissions caused by the extra time that the car engine is working or the increase of accidents on account of bot-tlenecks. Fortunately, current developments in traffic control technology are obtaining promising results, helping people to arrive faster at their
tion1.
There are several approaches on directing the flow of traffic using traffic lights. Some of them are simple, consisting in changing the light color ev-ery pre-established amount of time. Others are more complex and require synchronization among several signals, e.g. the green-wave method [51]. Al-ternative methods are self-organized [3, 10, 18] and require the presence of several sensors in the road. Moreover, Artificial Neural Networks (ANN)2 have been applied to control traffic lights [45, 56].
ANN present several advantages, such as adaptability, flexibility and ex-tensibility. Nevertheless, from our point of view, these advantages were never completely attained by previous approaches. The main problem is the learn-ing process they follow to adjust the ANN. It is based on supervised methods, while we think it is more adequate to use optimization algorithms according to the nature of the task. The optimal coordination of traffic lights is in-tractable [39]. This implies that there is a high complexity in obtaining the data necessary to perform supervised learning.
The purpose of this thesis is to use ANN evolved using Evolutionary Al-gorithms (EA)3 to control traffic lights. The methodology followed during the research consists of two phases. First, the implementation of the dis-cussed method as well as some of the approaches mentioned in section 2.6. And second, computer-based simulation of the different solutions in distinct scenarios to compare the results.
1.1.
Definition of the problem
Consider the following problem: In a given city there are several traffic lights. Each one of these traffic lights has sensors that detect the number of cars before and after it along different distances, i.e. using an inductive loop (IL) or a camera to detect vehicles [22]. Traffic lights have also a way to communicate with each other using radio signals, Wi-Fi, Ethernet, GSM4 or any other communication method. Furthermore, signals can record historical data from their sensors. With all this information, a rational decision should be made to improve the current traffic situation. This decision may consist on maintaining the same state of the lights or changing to a different state. The decision can be made either by a local controller at the signal location
1Some of these developments and their results are showed in Section 2.6 2For a further discussion of ANN see Section 3.1
3EA are explained in more detail in Section 3.2
4For a complete explanation of a real implementation of communication among different
or by a central controller. We decided to implement the latter in this thesis. In this case, all the data from the sensors is collected and sent to the central control unit. When the decision is made, the central controller sends the information back with the new state to the traffic light. Of course, when using this approach the communication among lights has to be simulated.
1.2.
Scope of the thesis
The goal of this thesis is to simulate and evaluate different traffic con-trol approaches. In more detail, we simulated three different scenarios with different levels of realism.
Additionally, we develop our own approach that consists of controlling the traffic lights with an ANN evolved using different EA: biological evolution, cultural evolution and the evolution of communication.
1.3.
Layout of the thesis
This thesis is divided in six chapters: introduction, traffic flow theory, methods and tools, development, results, and conclusions and discussions. In the traffic flow theory chapter, we provide the basic notions in this subject to understand the problem, and, in the state of the art section, an overview of related work, focusing on previous solutions.
In the third chapter, we provide the theoretical framework that supports the development of our solution. We explain ANN and EA as well as Simu-lation from a theoretical point of view. Finally, the last section describes the software tools that support the development of our system.
The development chapter explains the implementation of the necessary software to carry out different experiments along three scenarios.
The results of these experiments are summarized in Chapter 5.
Chapter 6 discusses about these results comparing them with those from other methods. Finally, we explain our final conclusions and consider future developments.
Traffic flow theory
Traffic flow theory1 is fundamental for a correct comprehension of the un-derlying traffic mechanisms. It provides necessary calculations for an efficient transportation plan and also measures of performance of these plans.
For a better understanding of this theory, we firstly present a brief history of it. Secondly, we introduce some important concepts about the principal perspectives of this theory, namely microscopic and macroscopic. Finally, the principal traffic flow regimes are identified and described in the last sub-section.
Additionally, in the State of the art section we present several approaches that deal with the problem presented in Section 1.1.
2.1.
History
“In the beginning there was the Ford” [15]
Traffic flow theory started with the contribution of scientists from different disciplines, such as mathematics and physics. Early attempts adopt either the microscopic or the macroscopic theory [15], depending on the knowledge background of the scientists who study the phenomenon.
Microscopic models examine every vehicle individually considering its characteristics, including length, speed, position and acceleration [33]. One of these original models was developed by Reuschel and Pipes [41]. This model is based on the concept that the speed of the following car is a linear function of the distance between the lead car and the following car. Unfortunately, this concept has never been proven empirically [15]. However, in 1958 Her-man and Montrol [9] created the car-following model, which continues being
1Also knows asintelligent transportation systems (ITS) by the industry
Mnan(t−T) =λv(n−1)/n(t)
where,
Mn: Mass of nth vehicle
an(t−T): Acceleration of nth vehicle after reaction
v(n−1)/n(t): Relative speed of (n−1)th to the nth vehicle in time t
T: Response delay of a driver. T ∼= 1.5s λ: Sensitivity Term. λ/M ∼= 0.37s−1
Table 2.1: Mathematical formulation of the car-following model and its prin-cipal variables.
useful at present and is the basis of many current theories. The car-following model is similar to the Reuschel and Pipes one, but with a slightly different concept; that is, the acceleration of the following car is proportional to the relative speed between the lead and the following car, with a time-lag. The mathematical formulation of this model is presented in Table 2.1
On the other hand, the macroscopic approach describes the traffic flow as a whole. In this case, rather than individual characteristics, it is measured the mean speed, the flow rate, or traffic density among other variables [27]. Lighthill and Whitham [31] derived in 1955 a model using this point of view. It is based upon the fluid mechanics and represents the traffic as a continuum similar to a fluid. This model is useful to describe some intrinsic peculiarities of traffic, such as the propagation of shock waves [43]. However, this model produces errors in a wide range of scenarios and should not be use in practical applications to model the movement of traffic [15].
In 1959, the apparent rivalry situation between these approaches changed when Gazis et al [17] showed that a macroscopic relationships can be obtained from microscopic models. They used the car-following model to derive the main relation of the Greenberg theory [20]. This relation, showed in Ta-ble 2.2, is based in the equation of continuity of a compressiTa-ble fluid.
During the following years, the two-fluid model [24] was the only re-markable contribution of the scientific community to this field. This model presents the traffic in cities as if it was composed of vehicles that are moving and vehicles that are stopped. This helps to describe any city with only two parameters, the percentage of cars that are not running and the average speed of those that are moving. Those parameters can be extracted from a
q=ρu=cρln (ρj/ρ)
where, u: Velocity ρ: Density
c: Optimal speed, depends on road and vehicles characteristics
ρj: Density of traffic on traffic jam, also depends on external conditions
Table 2.2: Greenberg’s relation for the traffic flow and its principal variables.
concrete town and empirical evidences show that the model is very accurate and reliable.
In the past two decades, an increasing number of studies have been pub-lished in this area. Currently, the number of specialties involved in ITS are increasing, involving aspects from sociology, psychology, economy or envi-ronmental science among others. In addition, there are theories that cover almost the entire set of possible traffic situations. In our opinion, the only problem is the lack of an unified theory that embraces all the aspects of the rest of theories. However, solving this problem is out of the scope of this thesis.
2.2.
Microscopic
2When talking about microscopic characteristics, we mean those that are related with individual vehicles and interactions among them as well as with the road and other infrastructure. This characteristics start with the condi-tion of the driver. This condicondi-tion can be separated into several factors such as the age, stress levels, visual perception, fatigue and medical conditions. These factors lead to different variables, among which, the reaction time of the driver to diverse situations τi is the most important one affecting the
outcome. Due to its stochastic nature, the usual implementation of this be-havior is along with a computer simulation model [27]. Although all these factors and variables may be considered to build an accurate model of traffic, it creates a more complex model which in many cases will be discouraged as 2The name of the variables and symbols from this point agrees with the notation
it increments unpredictable situations [32].
More commonly, microscopic models contemplate aspects from physical properties of vehicles and roads. The most common ones are the follow-ing [11]:
Propulsive force: This force applies in the direction of the vehicle. It is the result of the engine power, the coefficient of friction and the gravity force. An approximation of the formula is:
Fp m =ap ≤gmin fn, k v (2.1) This approximation assumes that the power supplied by the engine is constant, which can be assumed for electric motors but is more inaccu-rately for internal combustion engines. However, this formula is useful as it shows that the maximum acceleration is inversely proportional to the speed, which implies the existence of a theoretical maximum speed regardless of the power of the engine.
Fluid resistance: This force represents the importance of the wind and the aero-dynamism of the vehicle. A general purpose formula of this force is:
Ff
m =−αv 2
r (2.2)
From this formula we can extract that the force opposes the motion. Nevertheless, a pair of considerations must be done. First if the wind blows in the direction of travel, the sign of the formula should be re-versed, implying that the force favor the movement; and secondly, the speed of the wind should be considerate 0 in case of cross-winds; that is, the component of the wind in the vehicle’s direction is 0.
Rolling resistance: When the tire rolls on the ground, it generates a force which has a component that always opposes the movement. This force, called rolling resistance, is the main cause of the sound and heat pro-duced by the wheels. It can be approximated by a linear function of speed. Nonetheless, despite its importance at low speed is not very noticeable at high speeds when the fluid resistance is more important due to the square exponentiation of the speed in the formula of the latter.
Braking resistance: This force is proportional to the intensity with which the brake pedal is depressed. Generally, when the propulsive force is greater than zero, this force is equal to zero, and vice versa, due to the
fact that the throttle and the brake pedal are rarely depressed at the same time. The bounds of this force are:
0≥ Fb
m ≥gfn (2.3)
Guideway resistance: The last force to be consider here is the guideway resistance. This force is only present when the trajectory described by the vehicle is not linear and/or not horizontal, i.e., there is a positive or negative slope; and/or there are curves in the road. For convenience, we can divide this force into two components.
When the path is not horizontal exists an acceleration that opposes the movement, in case of a positive slope, and helps the movement if the slope is negative. Road inclination is crucial to understand some cases of congestion near bridges and tunnels [33]. A suitable approximation of the function of this forces is:
Fg
m =−gβ (2.4)
In this equation, we assumed that the slope of the guidewayβ is small enough to consider that the sine and the tangent of the angle are ap-proximately the same.
Finally, the effect of the curves is important as it affects the coefficient of friction used in Eq. 2.1 and Eq. 2.3. The updated version of the friction coefficient is:
fn≈f
1 +v2c/g (2.5)
where c: d2dyx(2x)
Finally, all the microscopic theories share the use of properties from the vehicle and basic interactions between that vehicle and the one ahead. The most important of these properties are presented in Fig. 2.1. In that figure we can see the position xi and length l, of a given car, as well as the space
gap gsi and space headway hsi of that car with respect to the next one. By
general consent, the reference point of the vehicle is its rear bump. There are two perspectives to understand the space headway hsi, it can be seen as the
difference between the vehicles position xi+1−xi and the sum of the length
Figure 2.1: Main properties of a car and the interactions with the leading vehicle.The car i is in position xi and the leader i+ 1 in position xi+1. The position is typically measured from the rear bumper of the vehicle. The car has a lengthl, which added to the space aheadgsigives the space headwayhsi.
The next figure (Fig. 2.2) shows the trajectories of two vehicles in a space-time diagram. Both trajectories correspond to vehicles driving at constant speed vi, which is the tangent of the trajectory. In that figure we can see
time and spatial characteristics. The main time characteristic is the time position ti, which represents the time passed since a initial instant t0. All the other attributes can be derived from this one. It is also of importance the time headway hti which, as well as the space headway, can be seen as
the difference between the vehicles time position when passing a concrete point ti −ti+1 or the sum of the occupancy ρi and the time gap gti. The
spatial characteristics are the same described for the previous figure.
2.2.1.
Car-following model
The car-following model is one of the fundamental microscopic models of the traffic flow. The original formulation of this theory is presented in Table 2.1 for historical reasons. It was first formulated by Chandler et al in 1958 [9], but it is still currently valid with slight modifications. An example of one of the current modification of this theory is:
ai(ti−τi) = Sens
vi+1(t)−vi(t)
(xi+1−xi)2
(2.6)
Sens: Sensitivity of the driver, tipically 5000m2 /s
The main difference between the two approaches is that in the latter case the distance between the vehicles is decisive. In the modern formulation, when the vehicles are separated enough, the influence is almost canceled,
Figure 2.2: Trajectories of vehicles iand i+ 1 with constant speedvi. The x
axle represents the time and the y axle represents the distance. The headway time hti can be seen as difference of times when the cars pass the same
point ti −ti+1. This value can be decomposed between time gap gti and
occupancy ρi. The space measures are the same than in Fig. 2.1. Diagram
while in the original, the influence is constant. In typical circumstances, the former theory behaves the same as the latter with a constant speed of approximately 116 m.
In the next figure (Fig. 2.3) two cars are simulated using Eq. 2.6. The acceleration of the following car is calculated using a reaction time τi of 1 s.
It is clear that, as expected, the speed of the first car is replicated by the second one with a certain delay around 1 s. Another point to notice is that during the stationary time, from t = 20 s to t = 35 s the speed difference is decreasing as well as the acceleration of car 2. However, the prior formula doesn’t behaves as expected when the leader car stops. In that case, the speed of the following car becomes slightly negative even though the cars are separated. Moreover, when the initial distance is smaller (e.g., 50 m), the behavior is more erratic, with minimum accelerations of -1.12×10−10 m/s2 which is physically impossible. Thus, in order to agree with Eq. 2.1 and Eq. 2.3 we can change Eq. 2.6 as follows:
ai(ti−τi) = min max Sensvi+1(t)−vi(t) (xi+1−xi)2 ,-9.8m/s2 ,9.8m/s2 (2.7) In this equation we assume that the friction coefficient is 1. Additionally, we force the speed to remain positive regardless of the acceleration. Although this changes made the simulation operate between acceptable bounds, the acceleration and speed of the second car do not result realistic. Furthermore, a new simulation, with a initial distance of 20 m, shows that the second car overtake the leader. In conclusion, the car-following model of Eq. 2.6 is useful to simulate a free flow traffic in a highway, but is inadequate to describe the traffic in a city with cars stopped at traffic lights and similar situations.
Additionally to this analysis, the fundamental diagrams derived using this model are shown in subsection 2.4.
2.3.
Macroscopic
A different, but related approach to traffic flow theory is the microscopic point of view. From this perspective, traffic is seen as a whole and individual properties are hidden for the benefit of collective measurements. The most important of those measurements are explained in the next subsections. In order to obtain these characteristics, any region shape can be used; neverthe-less, for convenience only rectangular regions are used here. Thus, Fig. 2.4 presents a summary of the most characteristic macroscopic measurement re-gions. In this figure there are three regions well delimited. Rsdenote a spatial
Figure 2.3: Results of a simulation of 2 cars using the formulation of the car-following model described in Eq.2.6. Both cars start in a resting position separated by 100 m. The leading car has a initial acceleration of 1m/s2 during 20 s, stays with the same speed during another 15 s and brakes with a rate of -1m/s2 for 20 s; finishing at rest position for another 5 s. a) Represents the acceleration of both cars; b) the distance from the reference point; c) the speed difference, which is used in the formula to calculate the acceleration of the following car; d) shows the speed of both cars.
Figure 2.4: Principal measures used in macroscopic models. The time-space diagram represents the trajectories of several cars when passing through dif-ferent measurement regions. Rt represents a point in the road for a time of
measure Tmp; Rs is the space region K in a concrete time and Rt,s is the
space region K during the measurement time Tmp.
region in a concrete instant of time which can be obtained, for example, from an aerial photography. The temporal region Rt corresponds with recorded
data of vehicles occupancy in a point of the road obtained, for example, by single inductive loop detector (SLD). Finally, the region Rt,s represents a
general area of measurement with K length duringTmp time whose data can
be obtained for example from a succession of photographs from a aerial video camera.
Note that we not explicitly describe the multi-lane counterparts of the equations presented in the following subsections for space economy and be-cause the derivation is in most cases straightforward. For a more detailed derivation refer to [33].
2.3.1.
Density
Following the notation described in [33], density is represented by the symbolk with vehicles per kilometer as its unit. The most intuitive approach to calculate this variable is to count the number of vehicles in the region Rs
of Fig. 2.4 and divide it by the length of the area K. This process leads to the following equation:
k = N
K (2.8)
Density can be also seen as the total time spent by all the cars in a region divided by the area of the region[27]. This point of view is useful when it is not possible to measure the density in a spatial area. The previous definition leads to the following equation in the temporal region Rt:
k = PN i=1Ti Tmpdx = 1 Tmpdx N X i=1 dx vi = 1 Tmp N X i=1 1 vi (2.9) According to this equation in order to calculate the density in Rt it is
necessary to know the instant speed of the cars passing through the mea-surement point.
One step further is to calculate the density in any measurement interval, for example Rt.s. For this case, we can use Eq.2.9 also, but we need to know
the total travel time, which is not always possible. However, we can calculate the density for any given time using Eq.2.8 and then, add all those partial measures as follows: k= 1 Tmp Tmp X t=1 k(t) (2.10)
This equation represents the discrete version, where, in case of a contin-uous function, the summation can be substituted by an integral.
Finally, it is important to highlight the link between microscopic and macroscopic characteristics. In this case, density is equivalent to the recip-rocal of the average space headway ¯h−1
s as shows the following equation:
k = N K = N PN i=1hsi = 1 1 N PN i=1hsi = ¯1 hs (2.11)
2.3.2.
Flow
Flow is the temporal counterpart of density. It can be described as the number of vehicles passing through a point in the road per time unit. This description leads to this equation:
q= N Tmp
(2.12) This calculations are easily performed in the temporal region Rt but are
more difficult to perform in Rs. Fortunately, flow can also be described as
the total distance traveled by all the vehicles in the measurement region, divided by the area of this region[11]. This formulation is equivalent to the next equation: q= PN i=1Xi Kdt = 1 Kdt N X i=1 vidt= 1 K N X i=1 vi (2.13)
Continuing the analogy with density, we can add consecutive measure-ments of the flow in order to obtain the flow in any arbitrary measurement region Rt,s: q= 1 Tmp Tmp X t=1 q(t) (2.14)
There is also a microscopic equivalent to the flow, the reciprocal of the average time headway h¯−t1. This equivalence can be seen in the following equation: q= N Tmp = N PN i=1hti = 1 1 N PN i=1hti = ¯1 ht (2.15)
2.3.3.
Mean speed
The space-mean speed or simply mean speed corresponds with the total distance covered by the vehicles divided by the measurement time. In Rs
consist in the arithmetic mean of the speed of the vehicles present in that area while in Rt is the harmonic mean:
¯ vs= PN i=1Xi PN i=1Ti = PN i=1vidt N dt = N1 PN i=1vi (regionRs) N dx PN i=1dvix = 1 1 N PN i=1 1 vi (regionRs) (2.16) It is important to note that in the temporal area, the mean speed is calculated using the harmonic mean instead of the arithmetic mean and in the spatial area exactly the opposite. In case of using the other formula, the obtained variable is called time-mean speed ¯vt in contrast to the
space-mean speed ¯vs, the latter being the most used and important. ¯vtis generally
greater than or equal to ¯vs, according to the generalization of the inequality
of arithmetic and geometric means. This last inequality can be understood easily if we consider that, when the time-mean speed is assessed, faster cars are consider over a much longer road section than slower cars [27]. As it is shown by Wardrop [55] ¯vt = ¯vs+σ
2
s
¯
vs, where it is easy to see that if the sample
variance σs2 is close to zero, both, ¯vt and ¯vs are the same.
On the other hand, sometimes it is necessary to calculate ¯vs when having
¯
vt instead. To solve this problem we can use ¯vt≈v¯s+ σ2
t
¯
vs, which leads to the
following approximation derived in [7]:
¯ vs ≈ ¯ vt 2 + r ¯ v2 t 4 −σ 2 t ∀v¯t≥2σ2t (2.17)
2.3.4.
Occupancy
Occupancy is the last macroscopic variable to be explained in this sec-tion. In theoretical analysis, it is not as significant as the other three ones, but it is remarkably more important during empirical measurements. This importance appears when it is not possible to obtain vehicle’s instant speed, for example, when it is not possible to install a DLD (double inductive loop detector) and is only present a SLD. That kind of detectors can only report the occupancy, which corresponds to the fraction of time the measurement location was occupied by a vehicle [33]:
ρ= 1 Tmp N X i=i oti (2.18)
The vehicle on-time corresponds with the effective length of the car seen by the sensor divided by the speed of the vehicle. This effective length corresponds with the length of the car added to the sensor’s own detection zone.oti =
li+Kld
vi .
Finally, the importance of the occupancy appears with the following re-lation that holds for stationary traffic [11]. This rere-lation allows us to obtain the density, when knowing the occupancy measured by the detector and the average length of the vehicles.
ρ= ¯lk=⇒k = ρ¯
l (2.19)
2.3.5.
Fundamental relation of traffic flow theory
Density, flow and time-mean speed are related through the fundamental relation of traffic-flow theory [55], which is very useful when knowing two of the variables, because it allows obtaining the third one.
q =kv¯s (2.20)
This relation can be used wheneverq,k, and ¯vsare continuous variables or
smooth approximation of them, and the traffic consists of substreams com-posed by homogeneous and stationary traffic. Homogeneous traffic stands that every traffic substream has a homogeneous composition, i.e., the same type of vehicles. Finally, stationary traffic holds when all vehicle’s trajecto-ries are parallel and equidistant. This last condition is difficult to appreciate during a small window of time. However, it can be relaxed because it cor-responds to a macroscopic scale. This relaxation allows to determine that there is stationary traffic when the total distance traveled by the vehicles and
the time needed to cover that distance is the similar regardless of the size of that measurement area [11].
2.3.6.
Shock waves
Shock waves [43] represent the apparent movement of disturbances through the traffic stream. A good example of a shock wave occurs when the distance headway is small due to high density (k > kc). In this situation, a car in the
stream can brake for any reason and rapidly accelerate again, to compensate the deceleration and continue with the initial speed. This event generates a chain reaction in the form of a wave, where the following cars mimic the behavior of the leading car.
Shock waves are of two types, upstream and downstream. The exam-ple above illustrates an upstream shock wave, which are the most noticeable ones as they happen continuously during traffic jams. However, despite the apparent predominance of this type of shock waves, downstream ones also occur during the free-flow phase of the traffic. Along that regime, when the density is higher upstream than downstream, can happen that the accelera-tion of the leading vehicle generates a downstream shock wave as is shown in computer simulation as [19].
Shock waves are a decisive factor in the generation of traffic congestion. Some experiments regarding traffic waves suggest that traffic jams can be diminished or even vanished due to a driver special care on avoiding stop and go waves [5].
2.4.
Fundamental diagrams and empirical
mea-surements
The fundamental diagrams represent a visual tool for understanding dif-ferent traffic flow theories and their implications. It is important to mention that the existence of a direct causal relation has never been shown between any of the variables displayed in the following diagrams. Therefore, the fun-damental diagrams represents models that try to fit the experimental data with their curves. As a consequence, each theory draws a different diagram. For example, the car-following model presented in Eq. 2.6, can be derived in the following expression when there is homogeneous and stationary traf-fic [27]:
Figure 2.5: Fundamental diagrams of the Greenshields theory [21] and scatter plots from data collected by video camera CLO3 located at the E17 three-lane motorway near Linkeroever, Belgium [1]. a) Is the space-mean speed versus density or (k,¯vt) diagram, b) is the flow versus density or (k, q)diagram, and
c) space-mean speed versus flow or (q,v¯t)
¯ vs =− ¯ vf f kj k+ ¯vf f (2.21)
and the relation between qand ¯vscan be derived as follows:
¯ vs= ¯vf f 1− k kj ⇔1− v¯s ¯ vf f = k kj ⇔ > q(fund. rel.) ¯ vsk kj = ¯vs− ¯ v2 s ¯ vf f ⇔q =kj ¯ vs− ¯ v2 s ¯ vf f (2.22) This expression, when depicted, represents the fundamental diagram of Greenshields [21] which can be seen in Fig. 2.5.
2.4.1.
Space-mean speed versus density
This first fundamental diagram represents the equilibrium relation be-tween space-mean speed and density ¯vse(k). Fig. 2.5 a) depicts this
funda-mental diagram according to Greenshields theory, as well as a scatter plot of real data. This visual aid helps us to verify the first intuitive feature: as the density increases, the average speed decreases monotonically.
There are a number of other features we can also extract from this dia-gram. First, that the density is always delimited between zero and the the jam densitykj. Second, the mean speed is restricted between zero and a
max-imum, which corresponds with the free-flow speed ¯vf f. Also, there is a small
range of low densities, in which the space-mean speed remains constant cor-responding with the free-flow speed more or less. Although, this third feature is only present in the scatter plot, in the original Greenshields’ theory there is a an artificial flattering of its upper-left part, which is not present in the figure because the function was created according with Eq. 2.21. The final feature is that the flow can be obtained as the area of a rectangle delimited by the origin and a point of the fundamental diagram. This is represented in the picture by the area of the gray rectangle.
2.4.2.
Flow versus density
The most extended fundamental diagram is the one representing the equi-librium relation between flow and density qe(k). In Fig. 2.5 the diagram b)
shows one of this possible relations. Like in the previous case, this diagram is the one corresponding with Greenshields theory. In that sub-figure is also plotted real data which shows the discrepancy between that theory and the reality. It also shows that no simple relation can be made between flow and density because the points are extended along a big cloud. Nevertheless there are several features that can be extracted from this diagram:
From the origin to near the critical density kc, the flow increases in a
linear manner. Usually, this region is called the free-flow branch of the fundamental diagram.
Close to the critical density point, there is a bending of the fundamental diagram. This circumstance is a consequence of the blockade produced by slow cars impeding the fastest.
At the critical density point kc, the flow is maximum. This point is
After the critical density point is reached, the traffic state changes to the congested regime. During this regime, the state of the traffic deteriorates as the density increases. When the jam intensity kj is
reached, the flow becomes zero and the vehicles are stopped.
As in the case of the previous diagram, the other variable that appears in the fundamental relation of the traffic flow theory can be extracted for any point. In this case, the mean-speed can be calculated as the slope of the line through that point and the origin.
Finally, it is important to mention that from this diagram can be extracted the kinematic wave speed w. The value of this variable corresponds with the slope of the tangent in any point of the diagram. According with this description, the shock wave travels downstream during the free-flow regimen, disappears when the critical point kc is reached and travels backwards or
downstream in the congested phase.
2.4.3.
Space-mean speed versus flow
This third type of diagram differs from the previous ones in that it does not represents an injective function. In Fig. 2.5 c) can be seen that, for almost any flow values, there are two values of the mean speed. To be more precise, the inflection point is the only value of the flow to which corresponds an unique mean speed. This inflection point matches with the capacity flowqcap,
which separates the free-flow regime and the congested regime, i.e., upper branch and lower branch respectively.
The mean speed versus flow diagram ¯vse(q) is easily understandable
for some researchers, such as economists, depending on their background. Nonetheless, is not as intuitive as the ¯vse(k) diagram and therefore it is not
used very frequently [33].
2.5.
Traffic flow regimes
There are many different traffic flow theories. One of the first ones is the Greenshields’ theory [21] which was first proposed in 1935 and whose fundamental diagrams are depicted in Fig. 2.5. One important characteristic of this theory is that the diagrams are explained using a single equation. On the other hand, newer models use different equations depending on the regime in which there is the traffic flow. As a consequence, different models can be classified depending on the use of one or multiple equations to obtain its fundamental diagrams.
Figure 2.6: qe(k) diagrams representing an inverted-lambda shape (a) and a
Kerner’s three-phase traffic theory (b).
2.5.1.
single-regime models
Models that use only one equation are usually named single-regime mod-els, for example, Greenberg and Underwood models [20, 53] . The aim of these models is to simplify traffic phenomena at the same time that tries to fit the observed data into its curves.
However, the existence of a single equation to cover the entire range of circumstances, does not mean that it exhibits a homogeneous state, but quite the opposite. As we claim in the previous subsection, the fundamental diagrams of the Greenshields theory depicts two different traffic flow states, namely free-flow traffic and congested traffic. Furthermore, in this theory, those regimens are separated by the global maximum kc of the function in
the qe(k) diagram.
2.5.2.
multi-regime models
Likewise, models that use more than one equation are named multi-regime models. Multi-regime models were first introduce by Edie in 1961 [13] and other examples are the theories form Smulders [47] and Newell [36]. These models evaluate the original conditions of the traffic flow and then determine in which regime the flow is placed and thus applying the correct equation.
A good representation of this type of models is represented in Fig. 2.6 a). This fundamental diagram presents two straight lines overlapping in what is called inverted-lambda shape. During the free-flow period, the flow augments with the density until it reaches kc, which coincides with qcap (1).
At that point the flow drops dramatically as it enters in the congestion regime (2). During that regime the flow decreases with the increase of the density.
However, the same process does not apply when the model is in the congestion regime and the density starts to decrease. In that situation, when the density reacheskcnothing anomalous happens and the flow continues increasing with
the decrease of the density. This condition continues until the two lines intersect at the outflow from a jam qout, point when it shifts back to the
free-flow regime. The hysteresis behavior presented by this model can be observed in real situations as Treiter and Meyers shows in their work [52].
2.5.3.
3 states theory
The previous models assume the existence of an underlying causal rela-tion between two of the variables involved in Eq. 2.20. On the other hand, Kerner is more skeptical and he refuses this idea, which is substituted by his fundamental hypothesis of three-phase traffic flow theory [29].
This theory is based in the assumption that the traffic flow can be sep-arated into three phases, namely free flow (F), synchronized flow (S) and wide-moving jam (J). Theqe(k) diagram corresponding with this theory can
be found in Fig. 2.6(b). Observing that figure is obvious that the F and the J phases corresponds with the concept of equilibrium relation between density and flow, or in other words, a fundamental diagram. Moreover, those phases are similar to the inverted-lambda diagram discussed in the previous point. However, the synchronized flow phase contrasts with this notion as it covers a two-dimensional region. Due to this crucial factor, this theory differs from the other multi-regime models. Kerner uphold that the direct transition from F to J can be possible but not likely to happen. For him, the natural transition is F->S and S->J because it is not necessary a critical perturbation to change the states.
An important issue regarding this model is that it gives no explanation for some transitions and simply describes them. As a consequence, Kerner himself accepts that his theory is merely qualitative. However, several micro-scopic models show an agreement between their results and Kerner’s theory. Some of these models are [30] and [28], using both cellular automaton in their construction. The popularity of this theory is incrementing due to the support of those microscopic models along with the fact that it fits nicely with the observed data. At present, thanks to this popularity, this theory is a trend topic that many researches continue investigating and expanding [33].
2.6.
State of the art
An important issue in controlling the traffic in cities are the intersections. In most cases, when the quantity of the traffic is high, it is necessary to use traffic lights to control the vehicles and the pedestrians equitably and with security. Due to its importance, traffic engineers strive to increment the efficiency of the controlling systems. Their effort includes the development of new technologies to synchronize the lights and the addition of sensors to detect the state of the traffic. Our intention is to provide a method that uses all these information and infrastructure to support the decision process in controlling these signals. In this section, we review some of the most typical techniques to control the lights. Furthermore, we discuss the integration of some of them in our model to validate the obtained results of our solution.
2.6.1.
Marching
This technique consists in a manual or automatic synchronization of the traffic lights, which permits them to change colors at the same time, every fixed period. In Fig. 4.5 the three scenarios adapted for our experiments show a snapshot of the city controlled with this technique. In the first and second scenarios (a and c), all the lights in the north-south direction are green, meanwhile in the west-east direction is the opposite and all the signals are red. When the period is about to finish, all the lights that are green change at the same time to amber. Finally, when the period expires, lights that were red change to green, and those that were amber change to red, thus completing the period. The second phase is carried out in a similar manner, which allows a equitable distribution of the preference in the intersection for the 2 directions. In the third city (d) the process is similar, but the cycle is divided into four periods instead of two. In any of these periods, one of the four directions has the priority to pass and the other three wait in red light.
2.6.2.
Green wave
This technique is very similar to the previous one: it also needs a syn-chronization among the signals, and it lacks of any response to the actual situation due to the absence of sensors to control the lights, meaning that they are passive, or likewise, that their their decision on changing the lights is only based on their internal state.To understand its generalization for sev-eral streets, first we will focus on a single street. Imagine a straight street with only one direction that has a number of intersections directed by traffic lights. At any given time, when a signal changes from red to green, the next
one remains in red for a bit less than the travel time between the lights, and then also, changes to green. If this synchronization continues for all the lights, and the traffic is free-flowing (i.e., there is not a long queue waiting in the semaphore) any car can virtually travel through the entire length of the street stopping zero or one times at most.
In case of the city from Fig. 4.5a, the generalization is straightforward and both directions take advantage of the green wave solution in all their streets [51]. The case is different for the second city; due to geometrical constraints, the green wave can only benefit two directions, while cars in the other directions traverse opposite to the green wave, and hence, they stop in several lights, leading to a worse situation than with the marching technique. Finally, no beneficial implementation of this technique exists in the diagram that represents the third city, as a consequence of the turns and the and its layout.
When the geography of the area permits the use and full benefits of this technique, it represents an affordable solution that increments the perfor-mance of the traffic. Moreover, due to its advantages, it is used in many cities such as San Francisco, Amsterdam, Copenhagen, Dresden (Germany) and Keynsham (UK) [4, 37, 49]. Furthermore, it can be improved by the addition of sensors, as in the GLIDE system [38].
2.6.3.
Self organized algorithm (SOLA)
As stated before, the previously described methods are passive. On the other hand, self organized methods are active, acquiring multiple advantages due to the reaction to the current state. These factors allow them to adapt their behavior automatically to, for example, prevent further problems in a situation of strong traffic or to make changes more dynamically when the traffic is more sparse.
Here we explain briefly the self-organized method based on rules that Gershenson and Rosenblueth [19] presented in their article and that expanded the idea from a series of articles by Ball [3], Gershenson [18] and Cools et al. [10]. We call it SOLA after the name they use to reference it in their implementation. The diagram with the schematic of an intersection showing the principal areas needed for the rules is in Fig. 2.7 .The rules that controls the light are enumerated in Table 2.3.
In a normal traffic situation, rules 1 and 2 tend to create an improved marching like scheme; that is, if the demand is similar in both directions, these rules distribute equally the time the light is green, but if it is not the case, they benefit the direction with more traffic. The 3th rule permits the emerging of a green-wave, as it allows few cars (that should take part
Figure 2.7: Principal measurement areas around a traffic light.
1. On every tick, add to a counter the number of vehicles approaching or waiting at a red light within distance d. When this counter exceeds a threshold n, switch the light. (Whenever the light switches, reset the counter to 0.)
2. Lights must remain green for a minimum time u.
3. If a few vehicles (m or fewer, but more than 0) are left to cross a green light at a short distance r, do not switch the light.
4. If no vehicle is approaching a green light within a distance d, and at least one vehicle is approaching the red light within a distance d, then switch the light.
5. If there is a vehicle stopped on the road a short distance e beyond a green traffic light, then switch the light.
6. If there are vehicles stopped on both directions at a short distance e beyond the intersection, then switch both lights to red. Once one of the directions is free, restore the green light in that direction.
of a green-wave platoon) to pass through the intersection without stopping. Furthermore, rule 4 is useful in a situation of light traffic (e.g., during the night), making the changes very dynamic, which directly increases the per-formance in that situation. Finally, rules 5 and 6 appear when the traffic is very intense, and help avoiding dead-locks (whose presence quickly degrade the traffic performance), thus allowing a quick recovery when the situation is better.
2.6.4.
Manual tuned system
The Minnesota department of transportation maintains an updated man-ual on Traffic Signal Timing and Coordination, whose last revision is from March 2009 [34]. This manual explains the complete process on placing a traffic light, since the emergence of the need, to the posterior plan of revi-sions. The most important points of the planning are the following:
A complete research, with several sources, is carried out to gather the information needed to successfully complete the plan. Data and infor-mation obtained during this phase include: geometric conditions of the intersection, volume of the traffic, travel time and delay data.
Study of the area to:
place the sensors in a convenient position design and install the signal
Creation of the timing control plan, taking care of factors such as pedes-trians, cycle lengths and interval of lights.
Integration of the system in a simulator. Due to the nature of the task, the computer simulation is of a great importance in the process. It allows to check the validity of the solution prior to the real imple-mentation, saving money and improving the performance without any inconvenience for the driver3. In this manual, the suggested simulator is Synchro and SimTraffic.
Use the simulation to fine-tune the model, which may lead to obtain more efficient results.
Implementation of the model in the real situation, with the aid of the results obtained during the previous steps.
3A further explanation of the simulations and their characteristics and advantages can
Test of the implementation. This step is necessary to analyze the con-gruency between the model and the simulation and to check if the re-sults are as expected. In this point, if the expectations are not fulfilled, the solution is reevaluated to find the origin of the problems.
The final point is to create the future plan of revisions. Normally, the correctness of the timing plan is verified at least once a year and a complete analysis should be made every three or five year. Nevertheless, this values can change due to special circumstances; for example, in case of an important intersection or a point where it is expected to have large fluctuations in the future.
Methods and tools
This thesis is grounded in concepts from different disciplines, such as com-puter simulation, artificial intelligence and transportation engineering. These areas are necessary during different phases of the development. Concepts from transportation engineering are necessary to understand the problem and to evaluate the solution. Computer simulation is essential to construct a realistic model of the traffic, which is used both to obtain our solution and to compare different solutions. Finally, artificial intelligence is crucial to construct the adaptable and robust solution that represents the core of this thesis. This chapter presents a brief description of concepts from AI and simulation as well as the software used to implement them1.
3.1.
Artificial Neural Networks: Multilayer
Perceptron
2Artificial Neural Networks (ANN) is an important branch in the field of artificial intelligence. It has become one of the most extended and known tools to solve a wide range of problems, due to its multiple qualities such a great versatility, adaptability, and self-regulation. An ANN is a computa-tional model that mimics a biological neural system. It represents a model that, given an input or stimulus react producing an output. To be more concrete, it is composed by one or more neurons connected through synaptic links with each other. Thus, the output of one neuron can inhibit or excite 1Concepts from transportation engineering are explained in the traffic flow theory
chap-ter.
2This section is based in the Ham’s book [23], for an extended discussion about this
topic we refer the reader to that book.
other neurons, propagating the stimulus into the entire network until the final output.
Some problems that are usually tackled with ANN include supervised, unsupervised and reinforcement learning; regression, classification, pattern recognition and data processing. Additionally, the continued investigation in the field derives into the development of new tools dedicated to each type of problem.
There is a large variety of ANN, whose principal differences are the topol-ogy, the transition function and the learning algorithm. Thus, this section is mostly focused in the multilayer perceptron (MLP), which is the model used in our experiments.
3.1.1.
Model and topology
The MLP presents a feed-forward, multi-layered and fully connected ar-chitecture. Therefore, the MLP layers are ordered and every neuron is con-nected with all the neurons of the next layer, but only with them; that is, no feedback connections are allowed in the model. This structure can be seen in Fig. 3.1 which shows an schematic MLP with a single hidden layer,pinputs, m outputs and Lneurons in the hidden layer. This diagram shows the three different types of layers, whose key features are the following:
Input Layer: Its inputs correspond with the external inputs. Thus, the number of neurons in this layer correspond with the size of the input vector. These neurons do not need to make any transformation to the data and its only function is to connect each variable of the input vec-tor to each neuron in the first hidden layer. Nevertheless, this layer frequently normalizes the input vector, that is, it transforms the origi-nal data in such manner that it can fall into the range between 0 and 1.
Hidden Layer: These layers are responsible for carrying out the computa-tion. Each hidden layer can have any number of neurons and there can be any number of hidden layers, although in the representative figure there is only layer. The number of layers and neurons in each layer is determined by the complexity of the problem, but relatively small networks can perform acceptably well. Each input of each neuron has assigned an importance or weight for that neuron. That weight is rep-resented in Fig. 3.1 as wh
ij. Additionally, all neurons possess a weight
attached to a virtual input that is always 1. This weight is typically called bias and represents the activation threshold of the neuron. Every
Figure 3.1: Multilayer perceptron
weight is multiplied by its corresponding input value and then added together with the bias to obtain the value uj. Finally, a non-linear
function σ is applied to the value uj to obtain the final output hj.
Frequently this function is a variant of the sigmoid to keep the output within a small range.
Output Layer: The final layer is similar to a hidden layer with few differ-ences. First, the number of neurons is not arbitrary and correspond with the length of the output vector. Hence, the output of the neuron corresponds to one variable in the output vector and it is not attached to any other neuron. Finally, the typical function used in this layer is the identity function to avoid restrictions in the output range.
The ANNs, and therefore the MLPs, can be seen from a purely mathe-matical point of view, although its first purpose was to emulate the biological neural networks. From this point of view the weights and biases of the first layer can be arranged in a matrix in the following form:
Wh = wh 11 wh12 · · · w1hi · · · w1hp wh 21 wh22 · · · w2hi · · · w2hp .. . ... . .. ... ... wh j1 wjh2 · · · wjih · · · wjph .. . ... ... . .. ... wh L1 whL2 · · · whLi · · · whLp Bh = bh 1 bh 2 .. . bh j .. . bh L (3.1)
Now, it is straightforward to obtain the vector U, which is simply a matrix multiplication between the weights matrix W and input vector plus
the addition of the biases vector U = Wh ×X + Bh. To continue the calculation of the output it is necessary to apply the functionσ to this vector to obtain the vector H which is the final output of the first hidden layer in this example.
H =σh(U) =σh Wh×X+Bh (3.2) Finally, to obtain the output vector, the vector H is used as the input of the output layer and it is applied the same methodology, thus:
Y =σy(V) = σy(Wy×H+By) = σy Wy×σh Wh×X+Bh+By (3.3) When there are more hidden layers, the same process is repeated for every hidden layer. This mathematical treatment allows us abstract the MLP, thus simplifying it to matrices and matrix operations.
3.1.2.
Training
Training an ANN consists in the adaptation of the network to the current problem. This process may radically differ depending on the type of ANN and the problem. In the MLP’s case, the procedure consists of two steps: select the topology and adjust the weights and biases (or simply weights) of the layers. The topology selection is usually arbitrary, due to the fact that, relatively large changes in the structure, causes a minor impact in the final result. This selection is frequently part of the designer’s choice and it is adjusted using trial and error. Nonetheless, depending on the algorithm, this selection can be automatic or even adapt itself as part of the learning process [48]. On the other hand, the adjustment of the weights is very im-portant. An appropriate adaptation derives in a suitable network, which accurately represents the desired function; while a poor adaptation leads to an undesirable solution.
The training method used in this thesis is crucial to understand its results. Our method is based on genetic algorithms and is discussed in the Section 3.2. Nevertheless, the back-propagation algorithm is briefly explained here for practical reasons as it provides a good example of one of the most known training methods.
Back-propagation
The back-propagation is a supervised learning algorithm that iteratively minimizes the training error. The principal aim of a learning algorithm is
initialize the network do
for each example
calculate the output obtain error
update the weights done
while the stopping criteria is not reached
Table 3.1: Back-propagation algorithm in pseudo-code.
to teach the network how it should behave. In the case of the supervised learning, it is a set of examples which makes it possible. An example (in this domain) is a pair of vectors with a possible input X and its desirable outputD. Theseexamples act as teachers that enhance the understanding of the problem. Hence, as a rule of thumb, the training set should be sufficient large to give a overall view of the problem.
This algorithm consists in three parts: network initialization, learning process and stopping criterion. These parts and their more important sub-steps are presented in Table 3.1.
The initialization consists in the selection of the original weights. This selection typically uses random small numbers, although there are other vari-ants.
The learning process is divided into several sub-steps, the most important of them is to update the weights. This change in the weights is proportional to the mean-square error. Given an example, the error is calculated as the difference between the output that the network generates Y and the desired one D. This error is propagated backwards, that is the reason which gives name to the algorithm. Thus, the output layer is updated in first place using this equation: wyji(k+ 1) =wyji(k) +µδjyhi (3.4) where, δyj =−∂Eq ∂vj = (dj−yj) { 1}{σ0(vj)}=dj−yj (3.5)
The µ > 0 is the learning parameter. This number indicates how much the weights should change in every iteration.
The update formula is slightly more complex in the case of the hidden layers, because it requires the error calculated in the previous layers:
whji(k+ 1) =wjih (k) +µδhjxi (3.6) where, δjh = Lh+1 X k=1 δkh+1wkjh+1 σ0 uhj (3.7)
When h+ 1 exceeds the number of hidden layers, δkh+1 =δyj and whkj+1 = wykj.
This formula is applied to every hidden layer from the last to the first one. When all the weights has been updated, the second example is used to repeat the process. Finally, when all the examples have been used it is called an epoch.
Every time an epoch finish, the stopping criterion is evaluated. If the criterion is not accomplished, the algorithm runs for another epoch. The process is repeated until the criterion is reached, moment when the method stops and thus, finishing the learning process.
Stopping criteria
There is a number of stopping criteria for using in this algorithm. First, the number of epochs can be determined previously, and the process stops when that number is reached. Second, the algorithm stops when it reaches a certain error level. A third criterion is that the system stops when the error is the same or very similar during a certain number of epochs. Finally, any combination of these or other criteria can be used to determine when the algorithm terminates.
3.2.
Evolutionary Algorithms
This thesis addresses an optimization problem, which was already pre-sented in Section 3. The main characteristic of this type of problems is the impracticality to find the best solution, if any. This problems are usually
Related documents