For the first test case, the target image is as shown in Figure 12. The a priori belief state for the discrete wheel poses is shown in Figure 13 (values for car pose belief state are simi- lar). This indicates that the poses are largely unknown, with a slight bias to the zero pose.
Figure 12: Target image for test case 1.
The first control step iteration, based on this belief state, yields the initial plan shown below (plan 1). The Execu- tive performs the first of these actions, yielding a success- ful match, as shown in Figure 14. Based on this, wheel pose estimates are updated; the hypothesis for zero pose for the front wheel is strengthened. The second control step itera- tion, based on this updated belief state, yields plan 2. The Executive performs the first of these actions, yielding a suc- cessful match, as shown in Figure 15. Based on this, wheel and car pose estimates are updated to further strengthen the zero pose hypothesis. The third control step iteration, based on this updated belief state, yields plan 3. The Executive per- forms the first of these actions, yielding a successful match, as shown in Figure 16. In this case, because the pose hypoth- esis is pose zero (indicating that the camera is directly facing
(a) Wheel
Figure 13: Wheel pose, a priori belief state.
the car), the circle rather than ellipse variant of the Hough transform is used. The history of rear wheel pose belief state values over the control iterations is shown in Figure 17. The zero pose belief increases with successive iterations (obser- vations), whereas the pos and neg pose beliefs decrease. Plan 1 1. SURFMatch(front-wheel pose-zero) 2. SURFMatchOtherWheel(front-wheel pose-zero) 3. HoughEllipseMatch(rear-wheel pose-zero) Plan 2 1. SURFMatchOtherWheel(front-wheel pose-zero) 2. HoughEllipseMatch(rear-wheel pose-zero) Plan 3 1. HoughEllipseMatch(rear-wheel pose-zero)
Figure 14: Successful SURF match to front wheel.
Figure 15: Successful SURF match to other (rear) wheel.
For the second test case, the target image is as shown in Figure 18. As before, the planner generates a plan assum- ing pose zero, based on the a priori belief state. The SURF
Figure 16: Successful Hough ellipse match to rear wheel (match highlighted in green).
Figure 17: Evolution of belief state for rear wheel pose vari- able (neg, zero, and pos values).
matches succeed, even though the reference image for pose zero does not exactly match the wheels in the car due to its angle. After the SURF match other wheel action, the pose estimate is improved, resulting in a belief state where pose -1 is most likely. The effect of this new belief state is shown in Plan 3, which uses pose-neg-one, rather than pose-zero as the parameter to the Hough ellipse match action; the circle variation of the algorithm will not work, so it uses the ellipse variation, with bounds on aspect ratio and rotation informed by the car pose estimate. This results in a successful match, as shown in Figure 19. The history of rear wheel pose belief state values over the control iterations is shown in Figure 20. The zero pose belief is initally the highest, but after the SURF match other wheel action (iteration 2), it drops, along with the pos pose belief, while the neg pose belief increases. Plan 1
1. SURFMatch(front-wheel pose-zero)
2. SURFMatchOtherWheel(front-wheel pose-zero) 3. HoughEllipseMatch(rear-wheel pose-zero) Plan 2
1. SURFMatchOtherWheel(front-wheel pose-zero) 2. HoughEllipseMatch(rear-wheel pose-zero) Plan 3
1. HoughEllipseMatch(rear-wheel pose-neg-one)
Figure 18: Target image for test case 2.
Figure 19: Successful Hough ellipse match, using ellipse rather than circle variation of the algorithm.
Figure 20: Evolution of belief state for rear wheel pose vari- able, example test case 2.
Discussion
The focus of our efforts thus far has been on the sub-problem of finding a wheel in an image. This has led to an emphasis on “look” actions, corresponding to the different sensing al- gorithms (SURF and Hough), and parameterized by which wheel to focus on, and which reference image orientation to use. However, we have not incorporated “move” actions (actions that change the state of the agent or its environ- ment). We believe that the approach we have developed is well suited for incorporating move as well as look actions, with the generative planning component intelligently com- bining both types of actions. This would allow for testing with more general kinds of problems, where the goal is more than purely a perception goal, but rather, involves achieving an environment goal.
As a next step, we will use a quadcopter platform as a testbed for combining the existing look actions with move actions that move the quadcopter. We expect that this will allow for reliable navigation of the quadcopter around the vehicle, in indoor (garage) environments, while avoiding ob- stacles like pillars or people. It will also support movement of the quadcopter to close in on a wheel to inspect it more carefully.
Figure 21 shows a grid of navigation waypoints situated relative to a target vehicle. The quadcopter would move be- tween these waypoints in order to navigate, and also to get a better look at the vehicle and wheels. For example, sup- pose the quadcopter starts at waypoint wp1. A possible plan generated by the planner would mix look and move ac- tions, possibly with the goal of verifying that a wheel ac- tually has a flat tire. The move actions allow quadcopter to move to a more advantageous position, in order to de- termine with more certainty, the state of the wheel. The look actions determine the state of the wheel, and also keep the target vehicle anchored with respect to the quadcopter (Laporte and Arbel 2006; Coradeschi and Saffiotti 2002; Karlsson et al. 2008). This is important for indoor environ- ments, like garages, where GPS navigation is not available. Initial Plan 1. SURFMatch(front-wheel pose-neg-two) 2. Move(wp1, wp2) 3. SURFMatch(front-wheel pose-neg-one) 4. Move(wp2, wp3) 5. SURFMatch(front-wheel pose-zero) 6. HoughEllipseMatch(front-wheel pose-zero) The restriction to linear relations in the generative plan- ner we are currently using allows for a reasonable approxi- mation of belief state update, particularly for the belief state value associated with the look action. However, it does not allow for good normalization across all the values of a be- lief state variable during planning. Further testing and in- vestigation is needed to determine whether this is a serious shortcoming. In any case, we will investigate planners that allow for nonlinear relations, and therefore, more accurate belief state update during planning. Such planners include sampling-based planners, simple forward heuristic search planners, and SMT solvers.
Figure 21: Active perception by quadcopter navigating be- tween waypoints.
Thus far, we have avoided any attempt to learn from all the planning; we are computing point solutions, not learn- ing control policies. While learning comprehensive control policies is generally intractible, it would be interesting to investigate whether partial policies could be learned as a by-product of the planning. A related question is whether a planner, rather than generating a single, rigid plan, could generate a plan with some limited choices. The choices would be made quickly at execution time using a control policy associated with the flexible plan.
Acknowledgments.
This research was developed with funding from the Defense Advanced Research Projects Agency. The views, opinions, and/or findings contained in this article are those of the au- thors and should not be interpreted as representing the offi- cial views or policies of the Department of Defense or the U.S. Government. Distribution Statement ”A” (Approved for Public Release, Distribution Unlimited).
References
Bay, H.; Ess, A.; Tuytelaars, T.; and Van Gool, L. 2008. Speeded-up robust features (surf). Computer vision and im- age understanding110(3):346–359.
Bilton, N. 2012. Behind the google goggles, virtual reality. New York Times22.
Bonet, B., and Geffner, H. 2006. Learning depth-first search: A unified approach to heuristic search in deterministic and non-deterministic settings, and its application to mdps. In ICAPS, volume 6, 142–151.
Coradeschi, S., and Saffiotti, A. 2002. Perceptual anchor- ing: A key concept for plan execution in embedded sys- tems. In Advances in Plan-Based Control of Robotic Agents. Springer. 89–105.
Duda, R. O., and Hart, P. E. 1972. Use of the hough trans- formation to detect lines and curves in pictures. Communi- cations of the ACM15(1):11–15.
Fourman, M. P. 2000. Propositional planning. In Proceed- ings of AIPS-00 Workshop on Model-Theoretic Approaches to Planning, 10–17.
Intelligence129(1):35–62.
Hebert, P.; Hudson, N.; Ma, J.; Howard, T.; Fuchs, T.; Ba- jracharya, M.; and Burdick, J. 2012. Combined shape, ap- pearance and silhouette for simultaneous manipulator and object tracking. In Robotics and Automation (ICRA), 2012 IEEE International Conference on, 2405–2412. IEEE. Helmert, M. 2006. The fast downward planning system. J. Artif. Intell. Res.(JAIR)26:191–246.
Hofmann, A., and Robertson., P. 2015. Active perception: Improving perception robustness by reasoning about con- text. In Proceedings of the 10th International Conference on Computer Vision Theory and Applications.
Kaelbling, L. P., and Lozano-P´erez, T. 2013. Integrated task and motion planning in belief space. The International Journal of Robotics Research0278364913484072.
Karlsson, L.; Bouguerra, A.; Broxvall, M.; Coradeschi, S.; and Saffiotti, A. 2008. To secure an anchor–a recovery planning approach to ambiguity in perceptual anchoring. Ai Communications21(1):1–14.
Laporte, C., and Arbel, T. 2006. Efficient discriminant view- point selection for active bayesian recognition. International Journal of Computer Vision68(3):267–287.
Mausam, A. K. 2012. Planning with markov decision pro- cesses: an ai perspective. Synthesis Lectures on Artificial In- telligence and Machine Learning. Morgan & Claypool Pub- lishers.
McDermott, D.; Ghallab, M.; Howe, A.; Knoblock, C.; Ram, A.; Veloso, M.; Weld, D.; and Wilkins, D. 1998. Pddl-the planning domain definition language.
Monahan, G. E. 1982. State of the arta survey of partially observable markov decision processes: Theory, models, and algorithms. Management Science 28(1):1–16.
Pineau, J.; Montemerlo, M.; Pollack, M.; Roy, N.; and Thrun, S. 2003. Towards robotic assistants in nursing homes: Challenges and results. Robotics and Autonomous Systems42(3):271–281.
Prentice, S., and Roy, N. 2009. The belief roadmap: Efficient planning in belief space by factoring the covariance. The International Journal of Robotics Research.
Zhang, N. L., and Zhang, W. 2011. Speeding up the con- vergence of value iteration in partially observable markov decision processes. arXiv preprint arXiv:1106.0251.