Why We Write This?
SPECTO provides accurate and fast detection of abnormalities faced in tunnels, highways and roads using cutting edge video processing technologies.
SPECTO is powered by INTEL acceleration Visual Processing Units hardware and OpenVINO software kit to obtain bottleneck performances. This paper provides an analysis of the delivered benefits, the performance measurements and its use cases. The paper also describes how SPECTO helps to reduce the risks of accidents that may occur.
ABOUT ISSD
ISSD is a R&D company that delivers Intelligent Transportation Systems (ITS). ISSD has specialties in several fields such as dynamic junction management systems and overall city junctions orchestration, sensor-based traffic analysis systems, enforcement systems and automatic incident detection systems.
ISSD products target sustainability related issues and public safety for better urban ecosystems.
Executive Summary
What is incident detection system?
There are millions of vehicular tunnels, bridges and highways in use worldwide, and that number is expected to continue to grow to alleviate the impact of congested traffic. These road structures mainly deploy CCTV
How is SPECTO critical to public safety?
The solution is the deployment of an Automatic Incident Detection (AID) system that gets its feed from the CCTV distributed throughout road networks to monitor traffic flow and detect incidents or traffic jams.
The system acts as a helper for the human operator in the monitoring room. Thus, road authorities will require a less number of operators with much more efficiency in incident responses, and as a result a more stable public safety ecosystem is achieved.
SPECTO is an Automatic Incident Detection (AID) system that translates ISSD view of traffic security and public safety. It involves the implementation of a systematic, planned and coordinated set of responses actions and deployment of resources to prevent accidents in potentially dangerous situations and the handling of incidents safely and quickly. The use of SPECTO improves road safety by enabling a more rapid response to the victims of an accident or even a potential one so authorities can interfere within what
Incident
Detection System
CONTENTS
About ISSD Executive Summary What is Specto Capable of?
Specto Architecture What is The Flow of The System?
Specto Non-Technical Design Aspect Specto Installation Specto Uniqueness What are Specto Key Benefits?
Conclusion
to ensure the safety of the tunnel and its users. A known problem is that human operators struggle to scrutinize large numbers of monitors over extended periods of time due to cognitive overload. These road structures mainly deploy CCTV cameras which are wired back to a central control room where human operators monitor the flow of vehicles to ensure the safety of the tunnel and its users.
What is SPECTO Capable of?
Where is it beneficial?
SPECTO’s main aim is to help operators working in a surveillance control room. It scrutinizes traffic flow in real time with the help of cutting-edge technologies such as artificial neural networks that relies at the system’s heart. The system relays alarms to the operators (less than 10 seconds) who will take appropriate measurements based on the detected abnormality risk potential.
SPECTO monitors severe risk environments such as bridges, highways and tunnels. It scans those road structure via real time video feed coming from installed CCTV throughout the road network.
Wrong direction driving vehicles:
In some tunnels or highways drivers may miss an exit.
Which causes them to take the road shoulder and try to revert. Although, it may save them time. However, it raises the risk of causing an accident. SPECTO detects that case and relays it in an alarm to the operators to take the required action.
Dropped objects or debris:
These objects pose a threat to passing vehicles and need to be removed quickly. Drivers might swerve to avoid debris, potentially leading to a secondary incident threat. SPECTO’s fast detection of debris or dropped objects leads to quicker response to remove them from the road.
Smoke and fire:
A serious vehicle fire can develop normally after a traffic stop (e.g. following an accident). It is desired to trigger an alarm when a smoke is at its early stages before even the fire develop and creates a catastrophe. SPECTO is capable to detect smoke in its early stages before the situation develops in severity.
What is SPECTO response capability?
SPECTO can detect an event visually anywhere in an image, not only in the driving lanes but also on hard shoulders and emergency refuge areas regardless of the road surface material (concrete, asphalt gravel or metal). SPECTO hastens the process of mitigating a potential incident for - not limited to - the following scenarios:
Pedestrian:
Authorities report that some pedestrians attempt to walk through inner-city tunnels or highways as a shortcut. Exposing their lives and others for risk. To prevent such risks, SPECTO alerts the operators if a pedestrian is detected while they are attempting to pass through.
Stationary/Stopped vehicles:
SPECTO identifies vehicles as car, bus, truck or even a motorcycle. A vehicle can become stationary due to a breakdown. However, it poses high risk for traffic and for that reason the system notifies the operators if a vehicle has stopped at a certain spot of the road.
Lowering speed vehicles:
SPECTO can detect vehicles slowing down in high speed roads. Low speeding in such environments can cause accident risks for the upcoming vehicles.
2
Specto Incident Detection System
Watch SPECTO in the Field
Make Roads Safer Fast And Accurate Detection Of Events
Integration With Existing Systems
Specto Incident Detection System
SPECTO Architecture
What is the combination that builds SPECTO
SPECTO is a computer-vision solution for incident detection. It implements complex image processing and machine learning algorithms (e.g. convolutional neural networks) to obtain the desired performance.
The core hardware consists of a centralized server acting as management instrument, and processing servers to handle the incoming video feed from cameras. The central server’s task is to relay the event to operators and other assets on the same network (Figure-1), and to record and archive them, it also integrates with other existing assets such as Network Video Recorder (NVR), Video Wall system and SCADA electro-mechanical systems. Thus, in case of an event occurring inside a road, it forwards the event stream to be popped on the video wall as well as the operator’s PC for human response, and triggers the SCADA controller for appropriate machine response such as popping a warning message on Variable Message/Traffic signs (VMS/VTS) for the incoming drivers to take caution. The processing servers get the CCTV feed as RTSP video stream from the cameras, where each of these servers have the objective of scanning the incoming frames in real time and check for an abnormality. Each server is configured to handle a cluster of CCTV cameras to permit discrete events to be detected.
4 Figure 1: SPECTO architecture overview
Figure 2: SPECTO System flow
Figure 3: Processor server configuration
What is the Flow of the System?
At the core of the processing servers, ISSD deploys the most recent software algorithms combined with as described in Figure-2. SPECTO processing servers fetch RTSP video streams from CCTV cameras. The stream is processed and scanned for abnormality. In case an abnormality occurs, it will inform the central SPECTO server about the event with supporting information (timestamp, location, type of event). The central server in turn will trigger an alarm on:
a. Operator’s PC via a web page popup, b. Video wall,
c. SCADA electro-mechanical system.
The human operators will take the required measurements to prevent potential risks.
What is the technological breakthrough in SPECTO
At the core of the processing servers, ISSD deploys the most recent software algorithms combined with INTEL recent hardware technologies in computer vision:
Software & Algorithm
In research, Neural networks have shown excellent performance in computer vision related tasks. The mentioned performance is related to robustness against different environments, accuracy and response time compared with legacy or traditional vision solutions. Thus, ISSD took the challenge to implement SPECTO system with neural networks and achieve the desired goals.
SPECTO utilizes INTEL OpenVINO toolkit to run the algorithms on INTEL based architecture, and to obtain bottleneck performances. OpenVINO came in handy with the fact that ISSD researchers found out that they can use their frameworks of interest to prepare an algorithm then deploy it into INTEL hardware with the same systematic process.
Hardware
To achieve aimed software configuration. It is best to use INTEL Visual Processing Units (VPU) that are dedicated to video image processing instead of CPUs.
This approach contributed in having SPECTO easily scalable against road network sizes, different detection contexts and customer requirements.
In SPECTO the CPU decodes incoming streams and forwards them to the VPU arrays through the HDDL (PCI) daemon. With this CPU and VPU orchestration, the system is scalable and dispatchable into several working maintainable elements in a cost-effective manner.
The VPU returns the inference result to CPU which takes a logical based decision whether an event should be relayed to the central server to trigger an alarm. The communication protocol between the processing server and central is standard TCP.
SPECTO Non-Technical Design Aspects
What about user experience (UX)?
SPECTO is carefully built to provide zero learning time for users to understand and adapt to its workflow. The operators are provided with a web-based application which they can access from a web browser. This application communicates with the central server. The web app implements W3C standards in its user interfaces and UX design.
SPECTO has a dashboard showing of the processing servers’ health status with an overview of the deployment (cameras, servers …etc.), as in Figure-4.
In addition to that, the camera listing interface is provided. So, the operators are informed when a camera from CCTV network gets disconnected due to a reason (hardware failure, electricity …etc.), as in Figure-5. This interface provides the operators with the option of limiting the observation scene to a specific area. Thus, enabling more customization to the end user, as in Figure-6.
6 Figure 4: Dashboard overview
Figure 5: CCTV camera listing
Figure 6: User interface to select Region of Interest for detection
When it comes to alarm, in a pop-up window, SPECTO provides the operator with a live stream from the alarm location and recorded video segment of moments before the alarm. And to ensure the operator attention, a sound alarm accompanying the pop-up is issued on the webpage, as in Figure-7.
Figure 7: SPECTO event alarming pop-up on the web interface
How does it integrate with pre-installed instruments?
As mentioned before, SPECTO is integrable with different industrial systems in the field to create a heterogenous monitoring system.
In case of an event, SPECTO will forward the detected event video from related camera to the Video wall and the SCADA system. Thanks to SCADA integration, SPECTO can show of warning messages over VMS/VTS. In addition to that SPECTO is compatible with many Video management systems such as PELCO VideoXpert. So, the operators can be informed in various ways about anomaly to take the required measurements.
SPECTO Installation
What type of cameras does SPECTO support?
One of the key design aspects of SPECTO is its independence from the camera brand. As far as the camera provides RTSP stream, the system can work with it.
SPECTO has variation of preferred choices for camera type. In tunnels the suggested use type of camera is the fixed or static ones. Where cameras are fixed at the ceil of the tunnel tube (5-7 meters height) with around 70 meters in between to obtain a full view of the whole tunnel.
SPECTO also supports Pan-Tilt-Zoom (PTZ) cameras.
With PTZ cameras the system can auto-zoom towards the event area in a highway and save rich informative data about the event occurred.
The required servers to run SPECTO system are one central server and a group of processing servers. The number of needed processing servers is dependent on the customer requirement and the future scaling horizon, which makes each deployment design a unique customer story.
How does it vary between tunnel and highway environments?
This configuration can also run well to monitor highways or bridges. However, here at ISSD we find that supporting Pan-Tilt-Zoom (PTZ) cameras can deliver a more cost-effective solution to the customer, since a single PTZ camera can replace 8 static cameras at a highway, as in Figure-8.
Fixed cameras on highways are not considered feasible due to their limitation in such open and uncontrolled environment in comparison to tunnel environments.
A PTZ camera can scan 8 different zones within a 15 seconds interval.
A B
Figure 8: Comparison between fixed camera and PTZ camera setup on highways
The field of view (FOV) of the PTZ camera is set 80 meters for each zone to come up with a total FOV of 480 meters.
However, as a tradeoff for their cost efficiency, PTZ cameras introduce a new dimensionality to the challenge of video processing. The scene does not have a fixed background model as with static cameras in a tunnel, as was mentioned before every 15 seconds the scene view would change. Moreover, rain, snow, fog or haze on the road may cause clarity degradation apart from the challenges introduced by the light variations.
Hence, SPECTO supported with INTEL VPU chips made it possible to run more sophisticated Convolutional Neural Networks (CNN) algorithms to process the dynamic scene being fed to the system.
SPECTO Uniqueness
What are SPECTO key features?
Rapid alert on incident detection:
Operators are alerted within 10 seconds of incident occurrence. This allows minimum time for them to take the required safety measurements.
Accurate incident detection:
The system utilizes machine learning algorithms that were trained on data ISSD has collected from the target environment. Thus, creating a specialized machine learning solution for the detection context.
Exact incident location:
SPECTO triggers an alarm with the exact location of the incident place. It locates the target camera location on the road segment with a highlight of the incident location on the viewed scene.
Low false alarm rate:
SPECTO is accurate in its detection since it utilizes cutting edge technologies. It reports less than 1 false detection per day in complex open environments.
8
Incident archiving:
SPECTO provides a data storage utility for incidents.
Operators can query the archive of events easily via the web application provided. Each incident information is indexed with its timestamp, location and a video of the event.
Camera brand independent:
SPECTO does require RTSP video stream from any brand of deployed camera in the field to work with.
Ease of use:
SPECTO provides a simplified user interface for the operators to use and interact with. It allows them to configure their region of interest, and notifies them about any problem in the CCTV network that may prevent a camera stream to be fetched and processed.
They can also easily add the camera groups to the system via this interface.
Ease of installation:
The only requirement to install SPECTO® system is an active CCTV network. So, the streams would be forwarded to the processing servers easily and the surveillance processes can start.
User story customization:
SPECTO can detect the aforementioned incidents.
However, it is not limited to them. The customer requirements are analyzed and based upon that the best fitting hardware and software configuration is proposed to be installed. Thus, SPECTO architecture allows to have a tailored customer story.
Robustness:
In tunnels SPECTO utilizes static cameras to cover a specified area. However, on highways ISSD tends to use PTZ cameras in order to eliminate blind spots in expansive areas. In addition to that, the use of CNN with specialized vision processing units aided to deliver a robust solution against different weather conditions.
Integrable solution:
SPECTO has proven its integrability with other industrial systems under the surveillance umbrella such as SCADA, Video wall and Network Video Manager systems by different vendors.
24/7:
SPECTO works in an around the clock manner, and in case of a software upgrade or failure, ISSD provides a 24/7 direct customer support to interfere with the situation.
What are SPECTO Key Benefits?
Quick and dependable detection of incidents:
Due to its utilization of Visual Processing Units that accelerate CNN algorithms, SPECTO significantly accelerates the emergency response, giving the operators the ability to interfere with incidents and take safety measurements in real time. Incidents are detected reported and archived for any future security usage.
Consistent surveillance:
The processing of visual information and detection of irregularities within varying conditions of weather causes any surveillance system to have performance degradation. However, SPECTO CNN algorithms are trained on ISSD collected datasets targeting the environment of interest which creates a consistent performance on the overall run.
Cost effective & sustainable solution:
SPECTO design and deployment takes several factors into consideration. The usage of VPU to accelerate CNN benefits in hardware scalability configuration at a better cost for the end customer. The support of different camera types (fixed, PTZ) enables SPECTO to have better impact in terms of being an adaptable and sustainable solution.
Conclusion
ISSD aims to increase the effectiveness of traffic surveillance and hasten the response against traffic related emergencies. SPECTO is a translation of ISSD’s views of public safety.
Supported with recent technologies, SPECTO can detect various incidents and report them to the operators instantly. It acts as a helper system for them by digesting huge amounts of data coming from cameras distributed throughout road networks.
SPECTO design takes into consideration being an integrable solution with industrial systems in the field.
