This framework was tested with the use case of trying to detect whether a road or a lawn was in a picture from the camera in real time. Images from the camera were relayed to a ROS node that calculated the mean green color value in the image. Based on this mean value, a threshold test can be performed in order to decide whether there is a lawn or not in the picture.
Chapter 5
Conclusions and Recommendations
The scope of this MQP is to give an example of a design for a software system that can be used for an autonomous vehicle in a way that is extensible for further development, and is intended to be used in a robotic system. This is achieved by creating a system that shares video data for possessing among a network. This system is designed using tools that are designed for modular expansion of a single robot and flexible processing among multiple sources.
Implementation and Experimental Results
The implementation of this system is designed to provide the frame for a system that inte- grates vision processing from multiple sources moving forward, and has not currently been implemented on an autonomous vehicle. Provided in this MQP are recommendations for this systems implementation in autonomous vehicles that enable use vision.
The purpose of this system is to achieve a framework for an autonomous vehicle that uses computer vision to navigate. This system is designed to integrate vision from any IP enabled source, and is not limited by dedicated hardware stationed on a single vehicle.
Conclusions and Recommendations
Continuing with development in this framework should be in the pursuit of a system that can be an experimental test bed for using machine vision in vehicle navigation. While going forward, these recommendations that are concluded from this project:
Recommendation #1: An autonomous vehicle that uses machine vision for navigational pur-
poses should be agnostic to the source of its image data.
In order to take advantage of the scalability [3] strengths of computer vision, any vehicle that uses computer vision for autonomous navigation should not be locked into a single hard- ware setup. Since the system put forward in this project uses networking systems to control the flow of data throughout an autonomous vehicle, providing a system for agnostic data flow will make it possible to add more data sources or even dynamically integrate potential data sources. Recommendation #2: The design of an API to extend the usability of this framework
Figure 5.1: An example of a collaborative network consisting of ground and aerial vehicles. Data is relayed from/to the aerial vehicle to/from the ground vehicle to obtain more reliable information and greater accuracy about the vehicles’ surroundings [4]. This network is very well suited to using agnostic video sources because it opens up for much more flexibility in its configuration - multiple combinations of these vehicles cooperating together can be achieved without defining new standards for each new vehicles video stream.
Parallel to the system proposed in this MQP’s flexibility when it comes to hardware design on a single autonomous vehicle, a software framework must also be put in place in order to make software development under this framework possible. This should be achievable with the open source frameworks this project uses for image processing and for data coordination. A design that will allow future developers to employ dedicated functions and systems that en- capsulate various computer vision algorithms and abilities will make it possible for developers to completely leverage the advantages of this system. One of the major advantages that can be leveraged by providing a diverse development system around this framework is that it can take advantage of the internet. Developers can add improvements to an autonomous vehicle through software updates delivered online, rather than having to install new hardware in order to improve an autonomous vehicle.
Recommendation #3: More dedicated hardware should be used for critical real time perfor-
mance and safety functions.
While it is important for data to be shared across a network to take advantage of the higher level aspects that can be gained from computer vision, under this framework it is still possible to use data streams from local sources. Maintaining the flexibility of this setup is important for critical safety functions that could be optimized through dedicated hardware. While this dedicated hardware is not explored in this project to avoid premature optimization, the ap- proach of optimizing certain vision functions might be necessary for applications where real time performance, safety, and security is a risk.
Limitations of this Research
The scope of this project’s research is limited by the following factors:
1. While a current test bed for an autonomous vehicle is being developed through other MQPs at WPI, it is not currently ready yet - this research still needs to be tested on an
autonomous vehicle.
2. Requirements of passengers and future developers of autonomous vehicle should be taken into account as this project is expanded upon.
Potential Uses of the Recommendations
This project establishes a beginning framework for the software design and approach for an autonomous vehicle. It is a place to start at, and provides examples of possible test cases of the research in action. Further efforts in designing an usable framework should be expanded upon. The development, implementation, and maintenance of a software package that can be used to establish this framework in order to create an autonomous vehicle is a good next step for a future MQP.
References
[1] J. L. Kent, “Driverless van crosses from Europe to Asia,” 2010. [Online]. Available:
http://edition.cnn.com/2010/TECH/innovation/10/27/driverless.car/index.html?iref=allsearch [2] J. Fingas, “Self-driving vehicles and robotic clerks could take your job in 20 years,” 03
2015.[Online]. Available:
http://www.engadget.com/2015/03/08/robots-may-take-more-jobs/
[3] E. Cervera, “Integrating computer vision libraries in networked robotic systems,” in
Computational Intelligence in Robotics and Automation, 2005. CIRA 2005. Proceedings.
2005 IEEE International Symposium on, June 2005, pp. 267–272.
[4] J. H. Kim, “Multi-uav-based stereo vision system without gps for ground obstacle mapping to assist path planning of ugv read more at:
http://phys.org/news/2014-09-air-ground-based-robot-vehicles.html#jcp,” Electronics
Letters, vol. 50, no. 20, pp. 1431–1432, 09 2014.[Online]. Available: http://phys.org/news/2014-09-air-ground-based-robot-vehicles.html
[5] P. E. Ross, “Driverless Cars: Optional by 2024, Mandatory by 2044 - IEEE Spectrum,” 2014.[Online]. Available: http://spectrum.ieee.org/transportation/advanced- cars/driverless-cars-optional-by-2024-mandatory-by-2044
[6] J. Schmiduber, “Professor Schmidhuber’s Highlights of Robot Car History,” 2011. [Online]. Available: http://people.idsia.ch/ juergen/robotcars.html
[7] L. Martin, “Driving Forces: Lockheed Martin‘s Autonomous Land Vehicles,” 2012. [Online]. Available: http://www.lockheedmartin.com/us/100years/stories/alv.html [8] M. Novak, “DARPA Tried to Build Skynet in the 1980s,” 2013. [Online]. Available:
http://paleofuture.gizmodo.com/darpa-tried-to-build-skynet-in-the-1980s-1451000652 [9] D. Pomerleau, “RALPH: Rapidly Adapting Lateral Position Handler,” IEEE Symposium,
no. September 25-26, 1995, 1995.[Online]. Available: http://www.cs.cmu.edu/ tjochem/nhaa/ralph.html [10] J. Davis, “Say Hello to Stanley,” 2006. [Online]. Available:
http://archive.wired.com/wired/archive/14.01/stanley.html?pg=1&topic=stanley&topic_set= [11] DARPA, “DARPA Grand Challenge 2005,” 2005. [Online]. Available:
http://archive.darpa.mil/grandchallenge05/gcorg/
[12] C. Dobby, “Nevada state law paves the way for driverless cars,” 2011. [Online].
Available: http://business.financialpost.com/2011/06/24/nevada-state-law-paves-the- way-for-driverless-cars/?__lsa=e443-35b3
[13] U. H. of Representatives, “How Autonomous Vehicles Will Shape the Future of Surface Transportation,” p. 1, 2013.[Online]. Available:
http://transport.house.gov/calendar/eventsingle.aspx?EventID=357149
[14] Mobileye. (2014, 09) Artificial vision technology. Mobileye. [Online]. Available: http://www.mobileye.com/technology/
[15] M. Quigley, K. Conley, B. Gerkey, J. Faust, T. Foote, J. Leibs, R. Wheeler, and A. Y. Ng, “Ros: an open-source robot operating system,” in ICRA workshop on open source
[16] D. Vernon, “An Optical Device for Computation of Binocular Stereo Dispairity with a Single Static Camera,” in Opto-Ireland 2002: Optical Metrology, Imaging, and Machine
Vision, vol. 38, 2003.
[17] D. L. Baggio, Mastering OpenCV with Practical Computer Vision Projects, 2012.
[18] R. I. Hartley and P. Sturm, “Triangulation,” Computer Vision and Image Understanding, vol. 68, no. 2, pp. 146–157, 1997.[Online]. Available:
http://linkinghub.elsevier.com/retrieve/pii/S1077314297905476
[19] D. Lowe, “Object recognition from local scale-invariant features,” Proceedings of the
Seventh IEEE International Conference on Computer Vision, vol. 2, 1999.
[20] “Feature Matching — OpenCV 3.0.0-dev documentation.” [Online]. Available:
http://docs.opencv.org/trunk/doc/py_tutorials/py_feature2d/py_matcher/py_matcher.html [21] M. Bertozzi, A. Broggi, M. Cellario, A. Fascioli, P. Lombardi, and M. Porta, “Artificial
vision in road vehicles,” Proceedings of the IEEE, vol. 90, no. 7, 2002.
[22] H. Fountain, “Yes, Driverless Cars Know the Way to San Jose,” 2012. [Online]. Available: http://www.nytimes.com/2012/10/28/automobiles/yes-driverless-cars- know-the-way-to-san-jose.html?pagewanted=1&_r=0
[23] L. Gannes, “Here’s What It’s Like to Go for a Ride in Google’s Robot Car,” 2014.
[Online]. Available: http://recode.net/2014/05/13/googles-self-driving-car-a-smooth- test-ride-but-a-long-road-ahead/
[24] A. Goncalves and S. Joao, “Low Cost Sensing for Autonomous Car Driving in Highways.” [Online]. Available: http://welcome.isr.ist.utl.pt/img/pdfs/1663_hans_icinco07.pdf [25] E. Guizzo, “How Google’s Self-Driving Car Works,” 2011. [Online]. Available:
[26] L. Hardesty, “Think Fast, Robot,” 2014. [Online]. Available: http://newsoffice.mit.edu/2014/think-fast-robot-0530
[27] A. Heyden and M. Pollefeys, “Multiple view geometry,” in Emerging Topics in Computer
Vision, 2005, pp. 45–107.
[28] A. Iliafar, “LIDAR, Lasers, and Logic: Anatomy of an Autonomous Vehicle,” 2013. [Online]. Available: http://www.digitaltrends.com/cars/lidar-lasers-and-beefed-up- computers-the-intricate-anatomy-of-an-autonomous-vehicle/
[29] B. Kehoe, A. Matsukawa, S. Candido, J. Kuffner, and K. Goldberg, “Cloud-based robot grasping with the Google object recognition engine,” 2013 IEEE International Conference
on Robotics and Automation, pp. 4263–4270, May 2013.[Online]. Available: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=6631180 [30] T. Lassa, “The Beginning of the End of Driving,” 2013. [Online]. Available:
http://www.motortrend.com/features/auto_news/2012/1301_the_beginning_of_the_end_of_driving/ [31] D.-J. Lee, J. Archibald, X. Xu, and P. Zhan, “Using distance transform to solve real-time
machine vision inspection problems,” Machine Vision and Applications, vol. 18, no. 2, pp. 85–93, Nov. 2006.[Online]. Available:
http://link.springer.com/10.1007/s00138-006-0050-2
[32] M. de Paula, “Autonomous driving,” Popular Science, May 2014. [Online]. Available: http://www.popsci.com/blognetwork/tags/autonomous-driving
[33] S. J. D. Prince, “Computer vision : models , learning and inference,” 2012.
[34] P. Stenquist, “On the Road to Autonomous, a Pause at Extrasensory,” 2013. [Online]. Available: http://www.nytimes.com/2013/10/27/automobiles/on-the-road-to- autonomous-a-pause-at-extrasensory.html?pagewanted=all
[35] “Automated Driving Applications and Technologies for Intelligent Vehicles - AdaptIVe FP7 project- Automated Driving Applications and Technologies for Intelligent Vehicles.” [Online]. Available: http://www.adaptive-ip.eu/
[36] “Volvo Car Group‘s first self-driving Autopilot cars test on public roads around Gothenburg - Volvo Car Group Global Media Newsroom.”[Online]. Available:
https://www.media.volvocars.com/global/en-gb/media/pressreleases/145619/volvo- car-groups-first-self-driving-autopilot-cars-test-on-public-roads-around-gothenburg [37] A. Davies and A. Gallery, “Self-driving cars will make us want fewer cars,” 03 2015.
[Online]. Available:
http://www.wired.com/2015/03/the-economic-impact-of-autonomous-vehicles/ [38] A. Stoklosa, “Google shows off how its autonomous vehicles aren‘t killing cyclists or
hitting parked cars,” Car and Driver, 04 2014.[Online]. Available:
http://blog.caranddriver.com/google-shows-off-how-its-autonomous-vehicles-arent- killing-cyclists-or-hitting-parked-cars/
[39] ——, “California attempts to wade into the uncharted waters of autonomous-car regulation,” Car and Driver, 03 2014.[Online]. Available:
http://blog.caranddriver.com/california-attempts-to-wade-into-the-uncharted-waters- of-autonomous-car-regulation/
[40] J. Holloway, “Rinspeed releases details of micromax swarm car concept,” Gizmag, 02 2013.[Online]. Available: http://www.gizmag.com/rinspeed-micromax/26392/ [41] C. Weiss, “Rinspeed shows what the self-driving car will be like to ride in,” Gizmag, 12
2013.[Online]. Available:
http://www.gizmag.com/rinspeed-self-driving-concept/30104/
[43] C. Atiyeh, “European manufacturers leading r&d for autonomous cars we may actually want to drive,” Car and Driver, 02 2014.[Online]. Available:
http://blog.caranddriver.com/european-manufacturers-leading-rd-for-autonomous- cars-we-may-actually-want-to-drive/
[44] “Characteristics of Fifth-Wheel (Wagon Steer) Steering Page,” in Engineering Design
Handbook - Automotive Series - Automotive Suspensions: (AMCP 706-356). U.S. Army
Materiel Command, Nov. 2012.[Online]. Available:
http://app.knovel.com/web/view/swf/show.v/rcid:kpEDHASAS1/cid:kt00AC8JW7/viewerType:pdf/root_slug:engineering- design-handbook-18?cid=kt00AC8JW7&page=4&b-q=ackermann
steering&sort_on=default&b-subscription=TRUE&b-group-by=true&b-sort- on=default&q=ackermann
steering
[45] “Systems Engineering Management,” in Systems Engineering Fundamentals. U.S. Department of Defense, Jun. 2001.[Online]. Available:
http://app.knovel.com/web/view/swf/show.v/rcid:kpSEF00001/cid:kt00TYSBI1/viewerType:pdf/root_slug:systems- engineering-fundamentals?cid=kt00TYSBI1&page=2&b-q=system engineering&sort_on=default&b-subscription=TRUE&b-group-by=true&b-search- type=tech-reference&b-sort-on=default&b-toc-cid=kpSEF00001&b-toc-root- slug=systems-engineering-fundamentals&b-toc-url-slug=purpose&b-toc-title=Systems Engineering Fundamentals
[46] U. D. of Transportation. (2013, 08) Model systems engineering documents for adaptive signal control technology (asct) systems.[Online]. Available:
http://ops.fhwa.dot.gov/publications/fhwahop11027/sec_b.htm
[47] F. Brunet. (2011, 07) First definitions and concepts. [Online]. Available: http://www.brnt.eu/phd/node16.html
[48] J. V. Does. (2014, 08) Openmvg photo reconstruction. [Online]. Available: http://blog.htmlfusion.com/openmvg/
[49] Minidoodle. Gear ratio. [Online]. Available: http://jleibovitch.tripod.com/id111.htm [50] D. Bray. (2014, 09) Getting a raspberry pi on worcester polytechnic institute (wpi) wifi
Appendix A
Vision Code Repository
This Appendix serves as a repository for all code used throughout this MQP
A.1
ROS RTP Relay Initialize Bash Commands
The following is the bash command that initializes and launches the ROS node that performs a relay of RTP video stream data to a ROS topic.
> export ROCON_RTSP_CAMERA_RELAY_URL=r t s p ://YOUR IPCAM URL
> roslaunch rocon_rtsp_camera_relay rtsp_camera_relay . launch −−screen