Deliverable 5.9
System Integration Report
Grant
Agreement:
225663
Project
Title:
Integrated
System
for
Transport
Infrastructures
surveillance
and
Monitoring
by
Electromagnetic
Sensing
Project
Acronym:
ISTIMES
Start
date
of
project:
July
1
st,
2009
Duration:
36
months
Project
website:
http://www.istimes.eu
Organization name of lead beneficiary
TERN
Work package contributing to the
deliverable
WP 5
Contractual Date of Delivery
Month 33
Actual Date of Delivery
Month 36
Nature
R
Dissemination Level
PU
Document
properties
Document Responsible WP5 Author(s) R. Bernini, TERN‐CNR M. Argenti,TELESPAZIO V. Giannini, TERN‐CREATEC J. Dumoulin, IFSTTAR Contributors Version 4.0
Contents
List of Figures ...4 i. Preface...5 ii. Editor contact points...5 iii. Revision history ...5 1 Introduction...6 1.1 Purpose and Scope...6 1.2 Document Organization...6 1.3 Reference documents ...6 1.4 Glossary...7 1.5 Conventions ...7 1.5.1 Symbols and abbreviations ...7 2 Dynamic Sensors Integration... 10 2.1 Overview ... 10 2.2 Distributed Optic Fiber Sensor ... 12 2.2.1 Integration in the Portal ... 14 2.3 Uncooled Infrared Camera ... 17 2.3.1 Integration in the Portal ... 18 3 Static Sensors Integration... 21 4 Conclusions ... 23 5 References ... 24
List
of
Figures
Figure 1 Layout of the distributed optic fiber sensor web interface 11 Figure 2 Schematic layout of the DOF communication protocol... 12 Figure 3 Sensor Commanding Selection... 13 Figure 4 DOF sensor details... 14 Figure 5 DOF sensor commanding ... 14 Figure 6 DOF sensor result ... 15 Figure 7 DOFsensor raw result ... 15 Figure 8 IR camera main components ... 16 Figure 9 IR camera OGS‐SWE integration ... 17 Figure 10IR camera commanding sensor selection 17 Figure 11 IR camera details... 18 Figure 12 IR camera Quick Look View... 19 Figure 13 IR camera raw data 19
i.
Preface
This document describes the software components that make up D5.9.
ii.
Editor
contact
points
Name Email Organization
Argenti, Massimo <[email protected]> TELESPAZIO
Bernini, Romeo <[email protected]> TERN‐CNR
Giannini, Vincenzo <[email protected]> TERN‐CREATEC Dumoulin, Jean <[email protected]> IFSTTAR
iii.
Revision
history
Date Version Author Paragraph
modified Description 15/03/12 0.1 M. Argenti R. Bernini J. Dumoulin V. Giannini All Document structure and initial content
25/06/12 0.2 M. Argenti All Finalization
03/07/12 0.3 F. Soldovieri All Revision from
Scientific Coordinator 13/07/2012 0.4 M. Argenti R. Bernini F. Soldovieri All Set up of the final version
1
Introduction
1.1 Purpose and Scope
This document describes the integration of the system architecture, with a specific focus tosoftware components, with the sensing technologies. The purpose of the software components is:
• to demonstrate the feasibility of the integration of thefinal ICT architecture and sensors; • to promote discussion within the project about the results. In ISTIMES architecture we split the sensors deployed in two main categories: • Dynamic sensors (online sensors); • Static Sensors(offline sensors);
As dynamic sensor, we refer to a sensor that can be remotely commanded using web (online sensors), whereas for static sensor, we refer to a sensor that cannot or it is not useful, for the purposes of the ISTIME system, to be remotely commanded using web (offline sensors). In the following we describe and demonstrate the feasibility of the integration of thefinal architecture and sensors by considering separately the cases of dynamic and static sensors. 1.2 Document Organization Chapter 1 gives an overview of the system integration process.
Chapter 2 describes in details the integration of “dynamic sensors” providing two assessed examples
Chapter 3 describes the integration of “static sensors” providing also the information about the .format of the outcomes of the sensor data acquisition and processing. Chapter 4gives the conclusions of the deliverable . 1.3 Reference documents The main reference documents for this release of the ISTIMES architecture are: [R.1] OWS‐6 Secure Sensor Web Engineering Report 08‐176r1 [R.2] OWS‐6 Security ER 09‐035 [R.3] Observations and Measurements – Part 1 – Observation schema 07‐022r1
[R.4] Sensor Model Language SensorML Implementation Specification 07‐000 [R.5] Report on User Requirements D1.2 [R.6] SWE Common Data Model 2.0 08‐094 [R.7] Installation GuideforSensor Observation Serviceversion 3.1.1 [R.8] Installation GuideforSensor Planning Serviceversion 1‐00‐00 [R.9] OpenGIS_Sensor_Planning_Service_Implementatio n_Specification 07‐014r3 [R.10] OpenGISSensor_Observation_Service 06‐009r6 [R.11] OpenGIS Web Map Service 06‐042 [R.12] OpenGISWeb Coverage Service 07‐067r5 [R.13] OpenGIS Web Feature Service 06‐027r1 [R.14] OpenGIS Catalogue Service 07‐006r1 [R.15] OpenGIS Web Processing Service 08‐091r6 [R.16] OpenGISWeb Notification Service 06‐095 1.4 Glossary
Client –software component with the ability to invoke operations from a
server.
Dataset – coherent set of data and metadata.
Dataset Collection – a collection of datasets and/or other dataset collections, optionally containing metadata.
Interface – named set of operations that characterize the behavior of an entity.
Metadata – set of data ancillary information, supporting spatial/temporal location, classification, etc.
Operation – specification of a procedure that an entity may be called to execute.
Properties file – text file containing key‐value pairs according to the appropriate syntax.
Server – software component implementing a service, on which a client can invoke an operation.
Service – distinct part of the functionality provided by an entity through
interfaces.
Spatial Data Infrastructure – hardware/software system with specific support for spatial information management.
1.5 Conventions
1.5.1 Symbols and abbreviations
API – Application Programming Interface AM – Asset Manager
CDI – Common Data Index
CMS – Content Management System
Observation CRS – Coordinate Reference System CSV – Comma Separated Value CSW –Web Catalogue Service DAO – Data Access Object DBMS – Data Base Management System DDL – Data Definition Language DCP – Distributed Computing Platform DML – Data Manipulation Language DQL – Data Query Language DSS – Decision Support System ECT – Electrical Capacity Tomography ENVISAT – Environmental Satellite EOS – Earth Observation System ESB – Enterprise Service Bus FOI – Observed Features of Interested GBIF – Global Biodiversity Information Facility GIS – Geographic Information System GML – Geography Markup Language GPR – Ground Penetrating Radar GUI – Graphic User Interface HTTP – Hypertext Transfer protocol ICT – Information and Communication Technology ID – Identifier IEEE – Institute of Electrical and Electronics Engineers INSPIRE – Infrastructure for Spatial Information in Europe IP – Identity provider ISB – Internet Service Bus ISO – International Organization for Standardization JAXB – Java Architecture for XML Binding JAX‐WS – Java API for XML Web Service JPEG – Join Photographic Expert Group JSON – JavaScript Object Notation KML –Keyhole Markup Language KVP – Key‐value pair ML – Model Language NXD – Native XML Database OGC – Open Geospatial Consortium O&M – Observation and Measurements OS – Operating System OSI – Open System Interconnections OWS – OGC Web Service PDF – Portable Document Format PNG – Portable Network Graphic PM – Profile Manager REST – Representational State Transfer RMI – Remote Method Invocation
RM‐ODP – Reference Model of Open Distributed Process RPC – Remote Procedure Call RSS – Really Simple Syndication SANY – Sensor Anywhere SAR – Synthetic Aperture Radar SAS – Sensor Alert Service SDI – Spatial Data Infrastructure SES –Sensor Event Service SMS – Short Message Service SMTP – Simple Mail Transfer Protocol SOA – Service‐Oriented Architecture SOAP – Simple Object Access Protocol SOS – Sensor Observation Services SPS – Sensor Planning Service SSO – Single Sign On SVG – Scalar Vector Graphic SWE – Sensor Web Enablement SWG – Standard Working Group TIFF – Tagged Image File Format UC – Use Case UML – Unified Modeling Language WAYF – Where Are You From WCS – Web Coverage Services WFS – Web Feature Service WMS –Web Map Service WNS – Web Notification Service WP – Work Package WSP – Web Service Provider XML – Extensible Markup Language
2
Dynamic
Sensors
Integration
2.1 Overview
For the dynamic Sensors Integration, two sensors have been specifically considered as: • the uncooled InfraRed (IR) camera (available at IFSTTAR); • the distributed optic fiber (DOF) sensor (available at TERN‐CNR). The choice of these two sensors is related both to technical and methodological reasons. In fact, both the sensors are self‐assembled prototype that can be easily modified and customized. Furthermore, to solve the specific problem of web based interface for the infrared camera will provide the general guidelines for a more general class of sensors built up of cameras based on different sensing principles (video, microwave,..), where the main problem to be addressed is to manage a large amount of data.
For the second case regarding the optic fiber sensor, we are concerned with the concept of a “spatially distributed monitoring” that is a quite new concept but at the same time, the implemented solutions are of interest even for the more traditional case of network of “point‐sensors”.
Two different communication approaches have been implemented for the infrared camera and fiber optic sensor.
In the case of the uncooled IR camera, a dedicated so called “SENSORBOX” has been developed. The SENSORBOX consists mainly of a small PC that permits interactions of several different sensors (IR camera, whether station, GPS) with SWE (SPS + SOS) and WNS.In the SENSORBOX case, the user sends a task request to SENSORBOX, for instance by using SPS formalism. The SENSORBOX receives the tasking messages, translates them and makes the related operation. After, it sends a message to SOS regarding the storage of the data.
The approach implemented for the DOF is different requires a task storage service to see if there is an operation to be done or scheduled. This service is running on the application server (where there is also the SPS service). This action is done in a loop mode.
The two approaches exhibit different advantages and disadvantages.
For theSENSORBOX solution presented, it is not possible to know in advance what physical support will be used to connect the sensor to WAN, the necessity of a simplified solution arises.
The second solution (the DOF one) solves the problem of the statically allocated the IP, but it requires a loop for the request toask if a task is planned by the user. Traffic raise continuously; this is not a problem for ADSL modality, but few problems could arise when it is available only a low data flux line (for instance
2.2 Distributed OpticFiber Sensor
The DOF sensor is a self‐assembledprototype that permits distributed strain/temperature measurements using a standard optic fiber. The sensor is fully managed by an internal mini personal computer, which controls the data acquisition, data processing and sensor interfacing. A dedicated web‐based interface has been developed for distributed fiber optic sensor. An OGC sensor web services have been implemented; in particular, the following service have been implemented: 1) SOS (Sensor Observation Service) that provides access to observations from sensor.; 2) SPS (Sensor Planning Service) that has the aim to provide a standard interface to collection assets (i.e., sensors, and other information gathering assets) and to the support systems that surround them. In figure1, a schematic layout of the fiber optic sensor web interface is depicted.
In particular, the communication approach implemented between the ISTIMES SDI and the DOF consists of the following steps.
• The Sensor requires a task storage service to see if there is an operation to be done or scheduled. This service is running on the application server (where there is also the SPS service). This action is done in a loop mode.
• User sends a request to a sensor using the SPS request formalism. The sensor plugin puts the task on the application server. • This step can be achieved after several loop when a task was required by user and answer was yes « there it is ». • Sensor makes the task and sends an SOS message for data storage. Figure 2 depicts a schematic layout of the DOF communication protocol .
Figure 2Schematic layout of the DOF communication protocol
The implemented solution does not require to know in advance what physical support will be used to connect the sensor to WAN; this permits a flexible approach to interface the sensors to the ISTIMES SDI. In fact, the sensor only needs to know the server IP, which must be statically allocated; conversely, the server does not need to know the IP of the sensors, which could be dynamically allocated. The system requires a loop for the request to ask if a task is planned by the user, this entail a possible problem of traffic that is not significant due to the low data amount transferred by the sensor.From a security point of view, improvements by HTTPS, SSL or TLS, and certificates could be easily implemented in future.
2.2.1 Integration in the Portal
This section describes the integration of the components created for the fiber optic sensor in the ISTIMES user interface: the web portal.
The user after the log in the portal, with the correct credential, should access to the runtime management of the session.
Figure 3. Sensor Commanding Selection
Figure 3 shows the test bed selections. By selecting the Musmeci test bed, the list of the sensors deployed at the Musmecitest bed appears with the GIS map of the selected zone. The information about the sensor position is used to map the sensor icon over the GIS map. If the user select the icon with label “i” will access to the sensor details pop‐up window (see Figure 4).
In the left side corner, the user can select among the list of runtime sensor allocated to the selected test bed. For Musmecibridge,DOF sensor and IR camera should be selected.
After the sensor selection, the fiber optic for the case at hand, the parameter list to be filled as input for the SPS command appears. The parameter available at this stageis the time schedule, i.e., when the user wants the acquisition being
performed, an activity is in course to add other optional parameters
After the command has been sent, the status of the command appears in the list just below the map. The command should be accepted or not. In case of the “acceptance” command, the entry will enter in a pending status and wait for the acquisition result. As soon as the acknowledge will be triggered, a show button appears allowing the data access.
Figure 4DOF sensor details
The user will access a quick data acquisition view; for fiber‐optic case, the acquisition view modality concerns with a 2‐axis diagram,whichshows the acquired value as function of the position acquisition, for a fixed acquisition time.
Figure 6. DOF sensor result
The user can also access the raw data message received from the SOS DB, which contains details of the acquisition and the acquisition value.
Figure 7. DOF sensor raw result
The user will be able also to download the raw data message for a further analysis.
2.3 Uncooled Infrared Camera
IrLaW is an OGC compliant infrared thermography measurement system, which has been developed on a mini PC with real time computing capabilities for long term monitoring of transport infrastructures. As for the fiber optic sensor, the infrared sensor is fully managed by an internal mini personal computer, which controls the data acquisition, data processing and sensor interfacing.
An OGC sensor web services has been implemented. In particular the following services have been implemented (see Figure 8).
1. SOS (Sensor Observation Service) that provides access to observations from sensor.
2. SPS (Sensor Planning Service) that is meant to provide a standard interface to collection assets (i.e., sensors, and other information gathering assets) and to the support systems that surround them. 3. A dedicated FTP server to locate the performed acquisition.
Figure 8. IR camera main components
The solution developed, by means of with patches added to the 52North, has been performed at IrLaWand at measurement platform levels with the aim to be compliant with OGC (SWE) requirements . Therefore, such a solution has been different in details compared to the fiber optic one, due to the constraints in SOS definition and different parameter needs.
The insertion‐Message used has been adapted so to not integrate directly the measurement in the exchanged XML file, but a detailed link to an FTP server where the acquisition should be downloaded.
The final integration steps are shown in
sensor.
Figure 9. Infrared camera OGS-SWE integration
2.3.1 Integration in the Portal
Figure 10 IR camera commanding sensor selection
Figure 10 shows the test‐bed selections. By selecting the Musmeci test bed, the list of the Musmeci used sensor appears with the GIS map of the selected zone. The sensor position has been used to map the sensor icon over the GIS map. Pushing the icon I close to the sensor, a pop up has been created to show sensor details to the user (see Figure 11).
allocated to the selected test bed. For Musmeci fiber optic sensor and IFFSTR camera can be selected. After the sensor selection, in our case the Infrared Camera the parameters list to be filled as input to the SPS command appears. No parameter has been used in this integration and only the acquisition is commanded. After the command has been sent, the status of the command appears in the list just below the map. The command should be accepted or not. In case of the “acceptance” command, the entry will be in a pending status and will wait for the acquisition result. As soon as the acknowledge will be triggered a show button appears allowing the data access.
Figure 11.IRCamera details
Figure 12. IR CameraQuick Look View The user will access a quick data acquisition view, for the infrared camera, we havean image viewer (Figure 12). It has been used a .png viewer, performing an on‐the‐fly conversion between TIFF produced and stored on the ftp server and the png available viewer. The full detailed image should be download from the FTP server using the proper button selection.
Figure 13. IR Camera Raw Data
The user should also access the raw data message received from the SOS DB, which contains detail of the acquisition and the acquisition value numeric (figure 13).
3
Static
Sensors
Integration
This section is devoted to give a frame of the finalization logic of the ISTIMES staticsensor integration
ISTIMES architecture homogenizes the engine, by extending the logic of OGS SWE also to the static sensors. The static sensors will continue to nor be commanded directly via web, but will insert the acquisition results according to the same modalities of the online sensor. The SOS has been centralized and extended in order to be ready to receive the acquisition of each static sensor.
ISTIMES allow the insertion of static sensor acquisition, providing an interface to inject acquired data in the architecture. The portal will be the virtual SOS client of the static sensor and allow the injection of the acquisition in the SOS DB. In this way, the Catalogue will be able with the same harvesting procedure, to retrieve back the acquisition of both satellite categories.
This approach allows static sensor not provided with native OGC‐SWE capabilities to be integrated in the ISTIMES infrastructure.
In the following, for sake of clearness, we report a table with the main kindsand formats of the outcomes of the acquisition and processing relative to all the ISTIMES sensors.
Synthetic Aperture radar ‐ 2D spatial image (global vision) of the deformation of the structure
‐ Time series of the deformation of the single pixel of the structure (plot)
Distributed Optic Fiber ‐ Strain/temperature distribution of the monitored area as function of the spatial location (plot)
‐ Time series of the strain/temperature distribution in a single point of the structure (plot)
Cooled and Uncooled
Infrared Thermography ‐ Thermal images of the structure (series of frames) ‐ Time thermal behavior of an image pixel
pertaining to the structure. Hyperspectral
spectroscopy ‐ RGB images of the surface of the structure Mobile Ground Penetrating
Radar ‐ 2D profiles of the shallower layers of the road (images) ‐ Image depicting the asphalt thickness
Manual Ground Penetrating
Radar ‐‐ 2D profiles of the inner of the structure Constant depth slices of the inner of the structure (this is the representation also for the holographic radar)
Electrical Resistivity
Tomography ‐ Image of the inner of the structure
Ground Based SAR ‐ Time series of the fast displacements of the structure (plot)
‐ Fourier transform of the time domain displacement (plot)
Optical Displacement
Monitoring ‐ Time series of the fast displacements of the structure (plot) ‐ Fourier transform of the time domain
displacement (plot)
AIRBORNE LASER
SCANNER, DIGITAL
CAMERA, THERMAL
CAMERA AND HYPER‐
SPECTRAL SENSORS
‐ Images of the structure (DSM, DTM, thermal images)
4
Conclusions
This deliverable has reported about the integration of the ICT system architecture with the sensing technologies. The integration has been first carried out for two online sensors such as the optic fiber sensor and the infrared camera. The choice of these two sensors is methodological, since the solutions designed for these two sensors are of general interest for larger classes as the cameras of different kind (video, thermal, microwave) and the spatially distributed and point sensors.
After, the integration of the sensor outcomes in the system architecture has been also sketched out by focusing to the different format/kind of the outcomes of all the expected ISTIMES sensors.
5
References
Deliverable 5.8. Diagnostic results for the railway tunnel during the monitoring
period, PU.
