The above work has been published in prestigious and peer-reviewed journals and conference proceedings. The publications are listed below:
Peer-reviewed Published Journal Papers:
[J1] Youlu Wang, Senem Velipasalar, Mustafa Cenk Gursoy, “Distributed Wide-Area Multi- Object Tracking with Non-Overlapping Camera Views,” Springer Int’l Journal on Mul- timedia Tools and Applications, pp. 1–33, Nov. 2012 (DOI 10.1007/s11042-012-1267- x).
[J2] Youlu Wang, Senem Velipasalar, Mauricio Casares, “Cooperative Object Tracking and Composite Event Detection With Wireless Embedded Smart Cameras,” IEEE Trans. on Image Processing, vol. 19, no. 10, pp. 2614–2633, Oct. 2010.
Peer-reviewed Published Conference Papers:
[C1] Youlu Wang, Senem Velipasalar, Mustafa Cenk Gursoy, “Wide-area Multi-Object Tracking with Non-Overlapping Camera Views,” Proc. of the IEEE Int’l Conf. on Multimedia and Expo, pp. 1–6, July 2011.
[C2] Youlu Wang, Li He, Senem Velipasalar, “Real-time Distributed Tracking with Non- Overlapping Cameras,” Proc. of the IEEE Int’l Conf. on Image Processing, pp. 697– 700, Sept. 2010.
[C3] Youlu Wang, Mauricio Casares, Senem Velipasalar, “Cooperative Object Tracking and Event Detection with Wireless Smart Cameras,” Proc. of the IEEE Int’l Conf. on Advanced Video and Signal Based Surveillance, pp. 394–399, Sept. 2009.
[C4] Youlu Wang, Senem Velipasalar, Mauricio Casares, “Detection of Composite Events Spanning Multiple Camera Views with Wireless Embedded Smart Cameras,” Proc. of the ACM/IEEE Int’l Conf. on Distributed Smart Cameras, pp. 1–8, Aug. 2009. [C5] Youlu Wang, Senem Velipasalar, “Frame-level Temporal Calibration of Unsynchronized
Cameras by Using Longest Consecutive Common Subsequence,” Proc.of the IEEE Int’l Conf. on Acoustics, Speech and Signal Processing, pp. 813–816, Apr. 2009.
Our work on the wireless embedded smart camera system, described in Chapter 2, Chapter 3 and Chapter 4, is published in part in [J2], [C4] and [C3]. The real-time object tracking system with non-overalpping camera views, that is presented in Chapter 5, is published in [C2]. [J1] and [C1] include the multi-feature object matching algorithm and Petri-Net
based framework for object tracking across disjoint views, that are described in Chapter 6 and Chapter 7, respectively. [C5] presents the work on frame-level temporal calibration in Chapter 8.
Part II
Object Tracking and Event Detection
with Wireless Embedded Smart
Chapter 2
Embedded Smart Cameras and
Lightweight Vision Algorithms
Due to the limited processing power and limited memory of the embedded smart cameras, it is critical to design lightweight computer vision algorithms that require less computation and less memory, and consume less power. We designed and implemented lightweight algo- rithms on our smart camera boards. All the processing, which includes foreground detection, morphological operations, connected component labeling, blob forming, object tracking and event detection, is done onboard on the microprocessor of the smart camera unit. With the attached wireless motes, the camera nodes communicate with each other in a peer-to-peer manner, which removes the necessity of a central controller.
In this chapter, we firstly introduce the embedded camera boards and the attached wire- less motes that are employed in our system. Then, the algorithms running on each individual camera are described.
2.1
The Wireless Embedded Smart Camera Platform
The wireless embedded smart camera platform employed in our system is a CITRIC mote [6]. It consists of a camera board and a wireless mote, and is shown in Figure 2.1. The camera board captures video frames by a CMOS image sensor, and then processes them. An em- bedded Linux system runs on the camera board. Each camera board connects to a wireless mote via a serial port.(a) (b)
Figure 2.1: The wireless embedded smart camera platform employed in the proposed system.
2.1.1
CITRIC: The Camera Board
The camera board is composed of an image sensor, a fixed-point microprocessor, external memories and other supporting circuits. The camera is capable of operating at 15 frames per second (fps) in VGA and lower resolutions.
The image sensor of the camera board is an Omni Vision OV9655, which is a low voltage SXGA CMOS image sensor and designed to perform well in low-light conditions. It supports
image sizes SXGA (1280×1024), VGA (640×480), and any size scaling down from VGA. The microprocessor PXA270 is a fixed-point processor from Marvell with a maximum speed of 624 MHz, 256 KB of internal SRAM and a wireless MMX coprocessor to accelerate multimedia operations. It is capable of working in low voltage and low frequency, as low as 0.85 V and 13 MHz, to achieve low power consumption. The typical CPU frequencies that the CITRIC platform supports are 208, 312, 416, 520 MHz. Besides the internal memory of the microprocessor, the PXA270 is connected to 64 MB of SDRAM and 16 MB of NOR FLASH. 64 MB is the largest size of the Single Data Rate (SDR) mobile SDRAM components natively supported by the PXA270 currently available in the market [6].
All of our experiments were run in real-time with QVGA (320× 240) resolution. All the algorithms run on the embedded Linux system ported onto the PXA270 microprocessor. The embedded Linux system includes the JPEG compression library. Since we only store detected events of interest, with this compressing functionality, 64 MB SDRAM provides enough space for our experiments. All the programming data and saved results are transferred by the UART port of the PXA270. A USB-to-UART bridge controller is connected between the PXA270 UART port and USB port on a PC. The camera board can be powered by a USB port from a PC, or four AA batteries.
2.1.2
TelosB: The Wireless Mote
The wireless mote connected to the camera board is a TelosB mote from Crossbow Technol- ogy. The TelosB uses a Texas Instruments MSP430 microcontroller and Chipcon CC2420 IEEE 802.15.4-compliant radio, both for low-power operation [6].
The Texas Instruments MSP430 MCU operates at 8MHz with 10KB RAM. The TelosB is a commercial off-the-shelf mote loaded with TinyOS/NesC and multi-hopping commu- nication protocols. Thus, we can easily utilize them to perform wireless communication
and exchange data between camera nodes. Since the maximum data rate of the 802.15.4 is 250kbps, it is not viable to transfer whole video frames between camera nodes. Also, due to high power consumption of wireless communication and small buffer size of the mote, transferring large-sized packets should be avoided. We need to buffer and transfer as few and as small-sized packets as possible. We designed and implemented our algorithms and the communication protocol by taking this fact into account.
We focus on the lightweight algorithms, their energy requirement, P2P event detection and the application-layer protocol, and use the preloaded lower layer protocols in the TelosB mote. When TelosB is idle, no serial communication is performed between the camera board and the wireless mote. When the camera needs necessary information from other cameras, and needs to exchange data, only then it performs serial communication with the wireless mote to send and receive packets.