Evaluation Policy

The planner output contains the following information about the plan: Total Time Total time to generate a new plan

Makespan Total plan execution time

Plan Length Total length of the output plan which includes the total number of pro- cesses, actions and some events effects in the plan. This also gives the quality of the output plan, the shorter the length the better the plan.

NodeCount Number of nodes explored in search space

No. of Actions Total number of actions required to achieve the goal condition No. of Processes Total number of processes initialised within the plan

In the field of AI planning, the total time taken to generate a plan is a metric that shows the efficacy and speed of the planner. The total time depends majorly on the planner algorithm. It is also dependent on some other factors such as the language used to implement the planner and the hardware configuration of the system that the planner resides on. The faster it is to achieve the goal condition the lesser the total time to generate a plan and vice versa. The total time taken to generate a plan is an essential criterion for the evaluation of planners in AI planning. A planner is effective in a domain of problem if the total time to generate a plan for problems in that domain remains steady and stable. However, if the total time to generate a solution in a domain of

MAKESPAN OPERATOR STATUS PARAMETER DURATION

0.0 switch_to_green [JunctionA, phase1, nLNorth, nLSouth] 5.0

5.0 switch_to_green [JunctionA, phase2, jWNorth, jWSouth] 5.0

10.0 switch_to_green [JunctionB, phase2, brStr, lDSouth] 5.0

15.0 [process] Jtraffic_flow [JunctionA, phase1, nLNorth, nLSouth] 5.0 20.0 [process] Jtraffic_flow [JunctionA, phase2, jWNorth, jWSouth] 5.0 25.0 [process] Jtraffic_flow [JunctionB, phase2, brStr, lDSouth] 5.0

40.0 switch_to_green [JunctionA, phase1, nLNorth, nLSouth] 5.0

45.0 switch_to_green [JunctionA, phase2, jWNorth, jWSouth] 5.0

50.0 switch_to_green [JunctionB, phase2, brStr, lDSouth] 5.0

55.0 [process] Jtraffic_flow [JunctionA, phase1, nLNorth, nLSouth] 5.0 60.0 [process] Jtraffic_flow [JunctionA, phase2, jWNorth, jWSouth] 5.0 65.0 [process] Jtraffic_flow [JunctionB, phase2, brStr, lDSouth] 5.0

70.0 [process] Rtraffic_flow [nLSouth, wDStr] 5.0

75.0 [process] Rtraffic_flow [lDSouth, pTStr] 5.0

90.0 switch_to_green [JunctionA, phase1, nLNorth, nLSouth] 5.0

95.0 switch_to_green [JunctionA, phase2, jWNorth, jWSouth] 5.0

100.0 switch_to_green [JunctionB, phase1, wDStr, lDSouth] 5.0

105.0 switch_to_green [JunctionB, phase2, brStr, lDSouth] 5.0

110.0 switch_to_green [JunctionC, phase1, pTStr, bmStr] 5.0

115.0 [process] Jtraffic_flow [JunctionA, phase1, nLNorth, nLSouth] 5.0 120.0 [process] Jtraffic_flow [JunctionA, phase2, jWNorth, jWSouth] 5.0 125.0 [process] Jtraffic_flow [JunctionB, phase1, wDStr, lDSouth] 5.0 130.0 [process] Jtraffic_flow [JunctionB, phase2, brStr, lDSouth] 5.0 135.0 [process] Jtraffic_flow [JunctionC, phase1, pTStr, bmStr] 5.0

140.0 [process] Rtraffic_flow [nLSouth, wDStr] 5.0

145.0 [process] Rtraffic_flow [lDSouth, pTStr] 5.0

160.0 switch_to_green [JunctionA, phase1, nLNorth, nLSouth] 5.0

165.0 switch_to_green [JunctionA, phase2, jWNorth, jWSouth] 5.0

170.0 switch_to_green [JunctionB, phase1, wDStr, lDSouth] 5.0

175.0 switch_to_green [JunctionB, phase2, brStr, lDSouth] 5.0

180.0 switch_to_green [JunctionC, phase1, pTStr, bmStr] 5.0

185.0 [process] Jtraffic_flow [JunctionA, phase1, nLNorth, nLSouth] 5.0 190.0 [process] Jtraffic_flow [JunctionA, phase2, jWNorth, jWSouth] 5.0 195.0 [process] Jtraffic_flow [JunctionB, phase1, wDStr, lDSouth] 5.0 200.0 [process] Jtraffic_flow [JunctionB, phase2, brStr, lDSouth] 5.0 205.0 [process] Jtraffic_flow [JunctionC, phase1, pTStr, bmStr] 5.0

210.0 [process] Rtraffic_flow [nLSouth, wDStr] 5.0

215.0 [process] Rtraffic_flow [lDSouth, pTStr] 5.0

230.0 switch_to_green [JunctionA, phase1, nLNorth, nLSouth] 10.0 240.0 switch_to_green [JunctionA, phase1, nLNorth, nLSouth] 10.0 250.0 switch_to_green [JunctionA, phase2, jWNorth, jWSouth] 10.0 260.0 switch_to_green [JunctionA, phase2, jWNorth, jWSouth] 10.0

270.0 switch_to_green [JunctionB, phase1, wDStr, lDSouth] 10.0

280.0 switch_to_green [JunctionB, phase1, wDStr, lDSouth] 10.0

290.0 switch_to_green [JunctionB, phase2, brStr, lDSouth] 10.0

300.0 switch_to_green [JunctionB, phase2, brStr, lDSouth] 10.0

310.0 switch_to_green [JunctionC, phase1, pTStr, bmStr] 10.0

320.0 switch_to_green [JunctionC, phase1, pTStr, bmStr] 10.0

330.0 [process] Jtraffic_flow [JunctionA, phase1, nLNorth, nLSouth] 10.0 340.0 [process] Jtraffic_flow [JunctionA, phase1, nLNorth, nLSouth] 10.0 350.0 [process] Jtraffic_flow [JunctionA, phase2, jWNorth, jWSouth] 10.0 360.0 [process] Jtraffic_flow [JunctionA, phase2, jWNorth, jWSouth] 10.0 370.0 [process] Jtraffic_flow [JunctionB, phase1, wDStr, lDSouth] 10.0 380.0 [process] Jtraffic_flow [JunctionB, phase1, wDStr, lDSouth] 10.0 390.0 [process] Jtraffic_flow [JunctionB, phase2, brStr, lDSouth] 10.0 400.0 [process] Jtraffic_flow [JunctionB, phase2, brStr, lDSouth] 10.0 410.0 [process] Jtraffic_flow [JunctionC, phase1, pTStr, bmStr] 10.0 420.0 [process] Jtraffic_flow [JunctionC, phase1, pTStr, bmStr] 10.0 Total Makespan: 430.0

Number of Actions Operators in PLAN: 26 Number of executed PROCESSES in PLAN: 32 Total Output Steps: 66

Node Count: 10 Timings in milliseconds: Preprocessing Time is 77 Total Searching Time is 1382 Total Planning Time is 1459

Total Planning Time in Seconds: ~ 1seconds

problem is astronomically increasing with an increase in the complexity of the problem, it means the planner might get stuck during certain problem situation in such domain.

The total length of a plan is the number of the steps it will take to get to a goal con- dition from an initial problem situation. The total length of a plan for a given problem varies from planner to planner. The shorter the length of the generated plan, the better the quality of the plan. The number of Actions and processes in the plan, contribute to the total length of the plan. The lesser the number of actions and processes needed to achieve a goal state the better the quality of the plan for such problem domain.

To investigate the applicability and effectiveness of UTCPLAN, we use three eval- uation criteria for comparison: total time taken to generate a plan; the average number of processes used and the average number of actions initiated in the plan. We did not consider the makespan in this criteria because this implementation does not include a scheduler for makespan optimisation in the plan. Thus, using makespan as a major metric would not be suitable as criteria for evaluation of the planner.

We consider the performance of the planner with fixed and controlled signal value. Fixed duration means the duration of the green split is fixed at the initial state of the problem and would be the same throughout the planning time. The planner accepts the initial fixed timing from the problem file and uses this during the entire search space to generate the best combination of steps needed to achieve a goal condition that satisfies the problem situation. Controlled duration means that the duration is fixed at the initial state, but subject to changes whenever the planner anticipates a better optimum green phase than the fixed value during search space. In this case, the planner stores the past trends of numeric changes within the roads as search progress within search space. The past information of individual road input along with some dynamic information is retrieved from the planner. This numeric information is used to generate a dynamic prediction table over a fixed period of time. The generated values are sent to an optimiser to compute the best green split values for the next set of alteration taking into considerations all the constraints in the domain. The duration of green split at a

junction is updated dynamically during search space while the search progresses. This approach helps the planner to track and maintain changes in numeric fluents during search space.

We generated different traffic situation by increasing the percentage of road queues to create heavier traffic flow in the experiment. The simulation results for fixed time duration and controlled strategy are presented in Table 8.1. Given that x2 is the new

average value and x1 is the previous average value, the percentage change in value y%

is measured by equation8.1and recorded in table8.1. y(%) = x2− x1

x₂ ∗ 100

1 (8.1)

This help to visually illustrate the trend in plan quality of both the fixed and the con- trolled experiment. A decreasing (↓) trend in the value of y implies a good quality plan while a continuous increase (↑) in value of y means that the planner output is affected by the complexity of the problem in the domain. The more complex the problem becomes the more the challenge to generate quality plan at reasonable time. Moreover, when yis zero, it means the output plan is steady and stable despite an increase in problem complexity.