The fixed solution plan obtained in the previous section is not robust: any deviation during execution may result in an infeasible plan. In this section we discuss how we can extract a more flexible plan from the fully specified solution that can take advantage of the temporal and control flexibility of the problem.
As described in the Problem Statement, we use Qualita- tive State Plans (QSPs) to describe such flexible plans. A partial QSP extracted from the solution plan of the example problem is shown in Figure 7. This QSP is a temporal net- work of events (starts and ends of activities). These events no longer have associated precise execution times as in the fixed plan, but are connected by temporal constraints that come from the duration constraints of the activities and the problem temporal constraints (such as taking the volcano samples before the eruption finishes in 300 minutes). Start and end events of activities with continuous effects (such as navigate) are connected by state evolution constraints that express restrictions on how variables can vary while the ac- tivity is being executed and what the bounds of the control variables are. Finally, events can have state constraints (con- ditions at start or end of activities), such as being in the ren- dezvous region at the end of the mission, and events can be connected by state constraints, that specify the feasible tra-
jectories for the state variables between events (for example, being in the volcano region while the data is collected).
The QSP can be extracted from the fixed solution plan by traversing the fixed plan schedule and annotating the tempo- ral constraints (activity durations), state constraints (activity conditions) and state evolution constraints (continuous ef- fects). This requires maintaining the total order of the events so that the discrete conditions of activities hold. Lifting a partial order plan from the grounded solution or, even bet- ter, using POPF’s (Coles et al. 2010) approach of planning with partial order states is possible, and is considered for fu- ture work. In that case we would also have to annotate the discrete conditions as additional constraints in the resulting QSP.
Although we could extract a flexible plan like the one defined above from the solution of any temporal planner, the fact that Scotty operates with continuous control vari- ables makes this plan much more useful. The advantage of Scotty compared to other temporal generative planners is that Scotty can reason with continuous control variables, making this flexible plans much more useful. In effect, this gives the executive the flexibility to choose not only the ex- ecution times of the events, but also the values of the con- trol variables that modify the continuous effects. For exam- ple, if during execution it takes the AUV longer to take the sample than initially expected, the executive will be able to increase the navigation speed of the vehicle in order to ensure that the eruption data is collected before the event ends. In short, this provides the executive with two degrees of freedom to react to disturbances: the execution times and the control variables, as long as all state transition, activ- ity conditions and time constraints are satisfied. Algorithms exist that can execute QSPs (Hofmann and Williams 2015; 2006). These algorithms execute the plan activities online by choosing the execution times of the events and the control variables, while making sure that all constraints are propa- gated forward and satisfied at all times.
Empirical Evaluation
Scotty and Kongming can both reason with continuous ef- fects modified by bounded control variables. In a previ- ous section, we argued that Kongming’s time discretization can hurt performance as the complexity of the problem in- creases. In Figure 8 we present a simple AUV sampling mis- sion scenario that highlights this issue. The AUV needs to reach a certain depth range in order to take a sample. We parametrize this scenario in terms of the sampling depth that needs to be reached to take the sample. The AUV can use the action descend to modify its depth according to the bounded control variable descent-rate. Because Kongming discretizes time in constant time steps, increasing the tar- get sampling depth forces Kongming to create additional fact and action layers and, additionally, more variables that the ML(N)LP solver needs to consider. As a result, the per- formance of Kongming degrades very fast with the target sampling depth as shown in Figure 8. While Kongming’s performance degrades very fast with depth, Scotty’s per- formance is constant (and orders of magnitude better than Kongming’s). This is expected, as Scotty doesn’t discretize
0.1 1 10 100 0 40 80 120 160 Kongming Scotty
AeroAstro Doctoral Research Evaluation, January 2014 - Enrique Fernandez 19
Planning Time (seconds) 0.1
1 10 100 Sampling Depth 0 40 80 120 160 Kongming Scotty
Results are preliminary. Ongoing work.
performance due to it’s use of discrete time
sampling depth sampling X 30 m 80 m sampling area X = 0 depth = 0 descend1 descend2 descend1 descend2 descend3 Example scenario:
Higher depth (larger distance) requires more time steps (layers) to find a solution.
Kongming
Sampling Depth (m)
Planning T
ime
Figure 8: Sampling scenario that shows the problems of dis- cretizing time.
2D AUV 1 2D AUV 2 3D AUV Firefighting 1 Firefighting 2 Kongming 3.633 9.736 13.063 1.505 20.202 Scotty 0.054 0.025 0.192 0.03 0.372
Table 3: Comparison between Kongming and Scotty in sev- eral domains. Results show planning time in seconds for each problem.
time and, therefore, is solving the same problem regardless of the depth. Table 3 shows Scotty’s large performance ad- vantage in other domains. These domains typically show- case one or more mobile robots moving in a 2D or 3D envi- ronment and trying to complete objectives that involve vis- iting different locations.
On the other hand COLIN/POPF are efficient heuristic forward search planners that do not present the time dis- cretization problem. However, they are not meant for robot control and therefore do not support continuous control vari- ables, but fixed rates of change for continuous linear time evolving effects. Let’s consider an example that shows why this is not desirable for the type of problems we are inter- ested in (Figure 9). Imagine that the AUV in the example mission needs to reach the volcano region (xmin ≤ x ≤
xmax) some prudential time after the eruption has ended so
that it’s safe to be nearby but before too long has passed, so
t
x(t)
tmin tmax xmin xmax c*min c*max c0 c1 c2 c3 c4(control variable) cmin ≤ c ≤ cmax. Note that in order to
satisfy the constraints, the robot needs to move with some speed satisfying c∗min = xmin/tmax ≤ c ≤ c∗max =
xmax/tmin. Because COLIN does not support continuous
control variables, it would have to discretize the interval [cmin, cmax] into a set of discrete fixed rates of change (rep-
resented as c0...c4in Figure 9). The problem only becomes
feasible if one of the discretized values happens to be inside the valid bounds [c∗min, c∗max] (c2in the example). Each
discretized rate of change requires a new action added to the domain, making the problem harder to solve. The required number of discretized rates of change can become arbitrar- ily large as different time and state constraints can modify the valid speed interval arbitrarily. This problem becomes worse if the problem contains multiple goals with similar constraints but different values that require a different dis- cretization. If we expand this example to multiple dimen- sions (the robot can move in 3D space with different x, y and z velocities, for example) the problem exacerbates, as the discretization of the rate of change will need to happen si- multaneously across all dimensions, which would result in a large number of discretized actions. Because Scotty supports continuous control variables, no discretization is needed and only one activity is sufficient as long as the interval given by the control variable bounds and the interval that contains the problem valid speeds have a non-empty intersection.
Finally, we compare Scotty’s performance to COLIN and POPF in some of their benchmarking domains that use fixed rates of change and do not require continuous control vari- ables. The results are shown in table 4. Some of these do- mains were slightly modified to substitute discrete numeric effects with fixed-rate continuous time dependent effects. The reason for that is that Scotty’s current implementation doesn’t support discrete numeric effects yet. This is an im- plementation issue and not an algorithmic problem as dis- crete numeric effects could be added to the planner in ex- actly the same way as they are handled by COLIN.
COLIN and POPF perform better in general than Scotty (about an order of magnitude) in the domains they were de- signed for. This is expected as they are written in C++ and their code bases have matured for years in preparation for the International Planning Competitions. On the other hand Scotty has been written from scratch in Common Lisp to better integrate with our internal software and doesn’t in- clude common optimization techniques, such as detecting compressible temporal actions, that state of the art planners do. If the goal was to develop a high performance planner with Scotty’s new capabilities, integrating Scotty’s advance- ments in COLIN’s high performance code base would be the natural way to proceed.
Conclusion
We have presented Scotty, a mixed discrete-continuous gen- erative planner that fills the gap between high fidelity model predictive control approaches, that suffer from scalability problems, and temporal planners, that present an impressive
02 0.014 0.056 1.098 03 0.017 0.145 2.148 04 0.021 0.307 3.811 05 0.026 0.572 6.673 06 0.039 0.963 10.540 07 0.036 1.428 16.682 08 0.044 2.056 25.346 09 0.050 3.109 35.157 10 0.060 4.171 48.323 11 0.073 5.648 62.154 12 0.081 7.226 86.643 linear generator 01 0.013 0.015 0.543 rovers continuous 01 0.066 0.049 1.268 02 0.028 0.030 1.017 03 0.096 0.125 2.319 04 0.051 0.047 1.021 05 - 0.701 3.486 06 - - - 07 0.324 0.538 4.219 08 6.118 - 60.345 09 - - - 10 3.854 5.221 - 11 1.432 264.648 - 12 - - 5.863
Table 4: Scotty performance results in some of COLIN’s benchmark continuous domains. Table shows planning time in seconds with a timeout limit of 10 minutes. Results shown as “-” indicate that the planners couldn’t solve the problem in ten minutes.
performance on large problems at the cost of limited expres- sivity, as can only reason with behaviors with discretized control parameters. Scotty finds the middle ground between these approaches by using flow tubes to compactly encapsu- late continuous effects with continuous control variables, as Kongming, and using heuristic forward search and a contin- uous time formulation, as COLIN. By avoiding discretiza- tion of either time or control variables, Scotty can reason with more expressive problems than COLIN and perform at least two orders of magnitude better than Kongming. Fi- nally, Scotty produces flexible plans that are suitable for ro- bust execution, as executives can exploit both temporal and control flexibility due to the presence of continuous control variables.
Acknowledgments
The work was partially supported by the Boeing Company under grant number MIT-BA-GTA-1. The authors would also like to thank Andrew Coles for providing a PDDL grounder tool that was used for testing our planner.
References
Blum, A. L., and Furst, M. L. 1997. Fast planning through planning graph analysis. Artificial Intelligence 90(1):281– 300.
azzeni, D. 2014. AUV mission control via temporal plan- ning. In Robotics and Automation (ICRA), 2014 IEEE Inter- national Conference on, 6535–6541.
Coles, A.; Fox, M.; Long, D.; and Smith, A. 2008. Planning with Problems Requiring Temporal Coordination. In Pro- ceedings of the Twenty-Third AAAI Conference on Artificial Intelligence, AAAI 2008, Chicago, Illinois, USA, July 13-17, 2008, 892–897.
Coles, A. J.; Coles, A.; Fox, M.; and Long, D. 2009. Tem- poral Planning in Domains with Linear Processes. In IJCAI 2009, Proceedings of the 21st International Joint Confer- ence on Artificial Intelligence, Pasadena, California, USA, July 11-17, 2009, 1671–1676.
Coles, A. J.; Coles, A.; Fox, M.; and Long, D. 2010. Forward-Chaining Partial-Order Planning. In Proceedings of the 20th International Conference on Automated Planning and Scheduling, ICAPS 2010, Toronto, Ontario, Canada, May 12-16, 2010, 42–49.
Coles, A. J.; Coles, A.; Fox, M.; and Long, D. 2012. COLIN: Planning with continuous linear numeric change. Journal of Artificial Intelligence Research (JAIR)44:1–96.
Dechter, R.; Meiri, I.; and Pearl, J. 1991. Temporal con- straint networks. Artificial Intelligence.
Fox, M., and Long, D. 2011. PDDL2.1: An Extension to PDDL for Expressing Temporal Planning Domains. CoRR abs/1106.4561.
Hoffmann, J., and Nebel, B. 2001. The FF planning system: Fast plan generation through heuristic search. Journal of Artificial Intelligence Research253–302.
Hoffmann, J. 2003. The Metric-FF Planning System: Trans- lating ”Ignoring Delete Lists” to Numeric State Variables. J Artif Intell Res(JAIR)20:291–341.
Hofmann, A., and Williams, B. 2006. Robust Execution of Temporally Flexible Plans for Bipedal Walking Devices. In Proceedings of the Sixteenth International Conference on Automated Planning and Scheduling, ICAPS 2006, Cum- bria, UK, June 6-10, 2006, 386–389.
Hofmann, A. G., and Williams, B. C. 2015. Temporally and spatially flexible plan execution for dynamic hybrid systems. Artificial Intelligence(0 SP - EP - PY - T2 -):–.
Kautz, H. A., and Selman, B. 1999. Unifying SAT-based and Graph-based Planning. In Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence, IJ- CAI 99, Stockholm, Sweden, July 31 - August 6, 1999. 2 Vol- umes, 1450 pages, 318–325.
L´eaut´e, T., and Williams, B. C. 2005. Coordinating Ag- ile Systems through the Model-based Execution of Tempo- ral Plans. In Proceedings, The Twentieth National Confer- ence on Artificial Intelligence and the Seventeenth Innova- tive Applications of Artificial Intelligence Conference, July 9-13, 2005, Pittsburgh, Pennsylvania, USA, 114–120. Li, H. X., and Williams, B. C. 2008. Generative Planning for Hybrid Systems Based on Flow Tubes. In Proceedings of the Eighteenth International Conference on Automated Planning and Scheduling, ICAPS 2008, Sydney, Australia, September 14-18, 2008, 206–213.
Li, H., and Williams, B. 2011. Hybrid Planning with Tem- porally Extended Goals for Sustainable Ocean Observing. In Proceedings of the Twenty-Fifth AAAI Conference on Ar- tificial Intelligence, AAAI 2011, San Francisco, California, USA, August 7-11, 2011.
Li, H. X. 2010. Kongming: a generative planner for hybrid systems with temporally extended goals. Ph.D. Dissertation, Massachusetts Institute of Technology.
Long, D., and Fox, M. 2003. Exploiting a Graphplan Frame- work in Temporal Planning. In Proceedings of the Thir- teenth International Conference on Automated Planning and Scheduling (ICAPS 2003), June 9-13, 2003, Trento, Italy, 52–61.
Muscettola, N.; Nayak, P. P.; Pell, B.; and Williams, B. C. 1998. Remote Agent: to boldly go where no AI system has gone before. Artificial Intelligence 103(1-2):5–47.