The utility of this sensor suite could be increased by adding additional image processing algorithms for human recognition and object detection. In a first responder situation, human recognition could be used to provide potentially life-saving information about where persons of interest are located. This feature could be added through the use of Xilinx’s High Level Synthesis environment, which would allow for the creation of programmable hardware from a powerful C/C++ image processing library such as Open Computer Vision (OpenCV).
A simpler improvement could be to incorporate all available inertial displacement data from the PmodNAV IMU into the device’s processing pipeline. With inertial displacement data completely integrated, this sensor suite would have the capability to produce a more sophisticated 2D map showing the complete path traversed by the device. This functionality would allow for a more complete understanding of the environment around the device.
One other important step in the future of this design would be to combine all hardware used onto a single platform such as a printed circuit board. This board would contain a Zynq chip, an onboard IMU, and mounted stereo camera and rangefinder hardware. With the creation of a customized sensor board, the device could then be added to existing robotic platforms for field testing.
References
[1] Endeavor Robotics, 110 FirstLook R. Available from: http://endeavorrobotics.
com/products#110-firstlook.
[2] Bounce Imaging, Tactical Throwable Cameras. Available from: http://www.
bounceimaging.com.
[3] Serveball. Available from: http://www.serveball.com/.
[4] Davison, A.J., Real-Time Simultaneous Localisation and Mapping with a Single Cam- era. IEEE Computer Vision, 2003. 2(1).
[5] Stefano Mattoccia, M.P., A passive RGBD sensor for accurate and real-time depth sensing self-contained into an FPGA. in International Conference on Distributed Smart Cameras. 2015.
[6] Xilinx. ZedBoard Zynq-7000 ARM/FPGA SoC Development Board. 2017. Available from: https://www.xilinx.com/products/boards-and-kits/1-elhabt.html. [7] AVNET, ZedBoard Hadware User’s Guide. Version 2.2 datasheet, 2014.
[8] Hokuyo Automatic Co. Scanning Laser Range Finder URG-04LX Specifications. C-42- 3389 datasheet, 2012.
[9] Hokuyo Automatic Co. Scanning Laser Range Finder. Available from: https://www. hokuyo-aut.jp/02sensor/07scanner/urg_04lx.html. 2009.
[10] ARC Electronics, RS232 Data Interface. Available from: http://www.arcelect.com/ rs232.htm.
[11] Sparkfun Electronics, RS-232 vs. TTL Serial Communication. 2010. Available from: https://www.sparkfun.com/tutorials/215.
[12] Hokuyo Automatic Co. URG Series Communication Protocol Specification. C-42-3320- A datasheet, 2012.
[13] Xilinx. Processing System 7 v5.5 LogiCORE IP Product Guide. in Vivado Design Suite. PG082 datasheet, 2015.
[14] ASCII Table, ASCII Table - ASCII Character Codes. 2010. Available from: http: //www.asciitable.com.
[15] Wolfram MathWorld. Polar Coordinates. 2016. Available from: http://mathworld. wolfram.com/PolarCoordinates.html.
[16] STMicroelectronics. LSM9DS1 iNEMO inertial module: 3D accelerometer, 3D gyro- scope, 3D magnetometer. DocID025715 Rev 3 datasheet, 2015.
[17] STMicroelectronics. LPS25HD MEMS pressure sensor. DocID027112 Rev 4 datasheet, 2016.
[18] Digilent. Pmod NAV. Available from: https://reference.digilentinc.com/
reference/pmod/pmodnav/start.
[19] Thomas Finley. Two’s Complement. Available from: http://www.tfinley.net/
notes/cps104/twoscomp.html. 2000.
[20] Leopard Imaging Inc. LI-VM34LP Camera Board. 2009. Available from: http://www. leopardimaging.com/uploads/li-vm34lp_v1.1.pdf.
[21] On Semiconductor. MT9V034: 1/3-Inch Wide-VGA CMOS Digital Image Sensor. 2015. Available from: http://www.onsemi.com/pub_link/Collateral/MT9V034-D. PDF.
[22] Averlogic. AL422B Data Sheets. 2001. Available from: http://www.frc.ri.cmu.edu/ projects/buzzard/mve/HWSpecs-1/Documentation/AL422B_Data_Sheets.pdf. [23] Robert Collins, CSE/EE486 Computer Vision I Fall 2007 Lecture Notes. Available
from: http://www.cse.psu.edu/~rtc12/CSE486/.
[24] Stefano Mattoccia, Stereo Vision. Available from: http://www.slideshare.net/ DngNguyn43/stereo-vision-42147593.
[25] Chris McCormick, Stereo Vision Tutorial - Part 1. Available from: http://
mccormickml.com/2014/01/10/stereo-vision-tutorial-part-i/.
[26] Hokuyo Automatic Co. UrgBenri data viewing tool. 2014. Available from: https: //www.hokuyo-aut.jp/02sensor/07scanner/download/data/UrgBenri.htm.
[27] Wikipedia. USB On-The-Go. 2017. Available from: https://en.wikipedia.org/ wiki/USB_On-The-Go.
[28] Xilinx. AR# 47511: Zynq-7000 AP SoC, SPI - In Master Mode on MIO, the SPI Controller Resets Itself when the SS0 Signal Asserts. 2013. Available from: http: //www.xilinx.com/support/answers/47511.html.
[29] Analog Devices, ADIS16IMU1/PCBZ Breakout Board Wiki-Guide. 2016. Available from: https://wiki.analog.com/resources/eval/user-guides/inertial-mems/ imu/adis16imu1-pcb.
[30] On Semiconductor. MT9V032: 1/3-Inch Wide-VGA CMOS Digital Image Sensor. 2015. Available from: http://www.onsemi.com/pub_link/Collateral/MT9V032-D. PDF.
[31] Xilinx. 7 Series FPGAs Memory Resources User Guide. 2016. Available from: https://www.xilinx.com/support/documentation/user_guides/ug473_7Series_ Memory_Resources.pdf.
[32] Fatih Porikli, O.T., Human Body Tracking by Adaptive Background Models and Mean- Shift Analysis. in IEEE International Workshop on Performance Evaluation of Track- ing and Surveillance. 2003.
[33] Analog Devices, Low Profile, Low Noise Six Degrees of Freedom Inertial Sen- sor. Rev. C datasheet 2012. Available from: http://www.analog.com/media/en/ technical-documentation/data-sheets/ADIS16375.pdf.
[34] Dresser, L.H., A.W., Accelerating Augmented Reality Video Processing with FP-
GAs. 2016. Available from: https://www.wpi.edu/Pubs/E-project/Available/
E-project-042716-172155/unrestricted/armqp-final-report.pdf.
[35] Sebastian Thrun, D.H., David Ferguson, Michael Montemerlo, Rudolph Triebel, and C.B. Wolfram Burgard, Zachary Omohundro, Scott Thayer, William Whittaker. A System for Volumetric Robotic Mapping of Abandoned Mines. in IEEE International Conference on Robotics and Automation. 2003.
[36] Xilinx. MicroBlaze Soft Processor Core. 2015. Available from: https://www.xilinx. com/products/design-tools/microblaze.html.
Appendix
A
Component Selection
Component Part Number Supplier Cost
FPGA ZedBoard Borrowed N/A
IMU PmodNAV Digilent $45
Rangefinder URG-04LX Borrowed N/A
RS232 to TTL Converter MAX232CSE uxcell $7
RS232 Breakout Board Swellder DB9 VIKINS Tech $7
Stereo Cameras† MT9V034 Mouser $146
† Note that we originally planned to purchase a Flir Lepton thermal camera module and
accompanying breakout board to support two stereo OV7670 camera modules. After ex- perimenting with the OV7670 camera module on our FPGA board, we realized that these camera modules are highly limited due to their low frame rate and poor documentation, and decided to search for a different camera module. In addition, at a price of $223 for a thermal camera with an 80x60 degree resolution, 25 degree field of view, and 7-9Hz image sample rate, we believed that we were much better off spending our money on better camera modules that were more usable for our task. For more information see Section 4.8.1.