Data Collection, LoggerNet and Reporting System

Monitoring and Technical Services

(

MATS

)

Systems for

Data Collection,

Management

and Reporting

FOR SMALL APPLICATION

SYMBOL

Environmental

Accurate and reliable data is an essential component in the

operation and management of many government, industrial

and scientific organisations. ALS provides a wide range of data

management systems and services to complement its range of

Measurement and Control Platforms (MCPs) for the automated,

long term, continuous management of data from nearly any

system, including weather, air quality, water, wastewater,

engineering, agricultural and geotechnical systems.

The range of options for data retrieval, both on-site and with the use of modern telecommunications, allows systems to be customised to meet exact customer needs. When properly managed by either the customer and/or ALS, they enable customers to optimise their processes while still meeting their environmental requirements.

ALS can design and supply data collection, management and reporting products and systems to suit many different applications. These range from the supply of simple stand-alone software applications through to the design and implementation of large and sophisticated automatic data collection and monitoring networks. They are based on the use of quality equipment and software with a proven track record, are versatile and reliable, and are used by government, business and industrial organisations within Australia and around the world.

Systems for Data Collection,

Management and Reporting

MAIN LOGO

Typical Monitoring Network

The diagram overleaf shows a typical Environmental

Monitoring Network. Monitoring networks may comprise

any combination of monitoring stations (weather, rainfall,

water level/flow/quality, air quality, noise etc) and may be

either stand alone (no telemetry) or connected together via a

telemetry system to provide data for operational purposes.

The monitoring network may be divided into several regions, depending on the telemetry connections being used. Stations located around a process plant, office or industrial area may be connected directly into the IT network via a dedicated radio base station and/or wired/wireless IT connections. More remote stations may be connected into the IT network via a system of radio repeater(s) or phone, cellular or even satellite connections.

Operation

Most monitoring stations supplied by ALS are typically based around the use of Campbell Scientific dataloggers and software. Each monitoring station is programmed to collect data at a predetermined rate.

Two sorts of data are generated by each station – instantaneous (most recent or ‘real time’ data) and historical (or ‘logged’) data. For a weather station, instantaneous data is generally measured every 1-10 seconds. Logged data may include 15-minute averages, daily minimums and maximums, and daily battery voltages. For stream or dam stations, instantaneous data is usually measured every 10 seconds (low powered sensors) or every 5 minutes (high powered sensors) to minimise power consumption. Water Level and Quality data may be logged every 5 minutes.

Telemetry

A manual connection is the cheapest and simplest way to retrieve data from our stations. This is commonly achieved using the manufacturer’s supplied communication cable (RS232, USB, infrared or Bluetooth) and the equipment specific software. This software may also include options for viewing, processing and exporting the data.

A wide range of off-the-shelf telecommunications options may be used to retrieve data, including phone (PSTN, CDMA, GSM, 3G/Next G), radio (Short Haul, VHF/UHF, Spread Spectrum), satellite (Globalstar, Iridium, Thuraya, GOES, ARGOS, INMARSAT etc) or via the internet (Fixed and Wireless Ethernet, GPRS, 3G/Next G).

Data Collection

Data collection is generally carried out under the control of specialist computer software, running on a dedicated computer that has been configured specifically for this purpose. As data is collected, it is stored or forwarded for further processing and/or use by data display systems (for example SCADA/intranet/internet displays). Operation of these systems is usually fully automatic (once configured).

A typical telemetry system uses Campbell Scientific LoggerNet or LoggerNetAdmin software for Data Collection, Monitoring and Control. LoggerNetAdmin is a special version of the standard LoggerNet software that, amongst other things, allows it to be installed and run on a server as a service. LoggerNet is the ‘heart’ of the

monitoring network software suite and allows the user to configure the frequency at which stations are polled and data is collected (both ‘real time’ and historical). LoggerNet provides many maintenance features, such as the Status screen, and can also be used for ‘manual’ data collection.

When each station is polled by LoggerNet, the most recent instantaneous (or ‘real time’) values will be collected in addition to any data that has been logged since the last poll. If data is ‘missed’, for example during a telemetry or IT network outage, LoggerNet will collect the most recent instantaneous values and all logged data since the last data collection.

The Campbell Scientific dataloggers used by ALS are capable of storing many months of data. In the event of a long outage or similar period of inactivity, LoggerNet may take some time to collect all the data. In these circumstances, LoggerNet will usually collect the data over several attempts, first collecting all historical data before collecting the instantaneous values.

Data is collected as long as LoggerNet maintains its configuration and is running on the dedicated computer/ server. This data is raw and unprocessed and is generally collected in CSV format and placed in a suitable location on the customer’s computer network in a folder structure to permit ease of access. The data may be retrieved manually or automatically for processing.

In addition, LoggerNet maintains a data cache (essentially a mirror image of the raw data files), which is used, amongst other things, for generating the intranet displays and alarms. The data cache is stored in binary format and is of a limited size, and is only used by LoggerNet and its associated software.

Data may also be collected manually at any time via the telemetry system or directly using a suitable field computer, PDA, etc and the appropriate software. It may even be collected using third party data collection software and imported manually or automatically into the data collection system.

MAIN LOGO

Typical Environmental Monitoring Network

Monitoring and Control

Real time or historical data can be displayed with a variety of software. LoggerNet includes a component called Real Time Monitoring & Control (RTMC), which may be operated in several different ways. It can be used to generate ‘screens’ or displays that show the current and historical data as it is collected. In RTMC Run Time mode (RTMC-RT), it allows a local user to monitor system operation and effect a control operation, from either the base station computer or from a remote location. In RTMC Web Server mode (RTMC-WEBS), it can be used to generate web pages of the RTMC screens, which can be published to the intranet/internet for viewing by all users.

Alarming

Depending on the telemetry system fitted, our systems can be programmed to send alarms or report site conditions by calling out to computers, phones, radios or pagers.

Email and SMS alarms may be generated directly from the monitoring station if it is fitted with cellular or satellite telemetry. Stations equipped with Campbell Scientific dataloggers (not CR200 Series) and any form of telemetry are capable of ‘Callback’ alarms via LoggerNet, which can be used to trigger audio/visual, Email or SMS alarms.

Software such as RTMC, VDV and HYDSTRA can also be used to generate alarms to warn of dangerous conditions and/or maintenance issues. Alarm conditions are continually monitored and, in the event of an out-of-limits condition, emails and/or SMS alarms may be generated via the local email system. These emails may be delivered to any number of staff. Alarm acknowledgement may be configured so as to require a user to acknowledge any/all alarms.

Data Exchange

Data may be forwarded or accessed by third party software, such as SCADA and data processing systems, using a number of LoggerNet Clients. These include Baler software (to ‘bale’ the data into packets) or OPC software (to permit data access by SCADA systems). Data can also be exported as ASCII files for further processing by spreadsheets, databases or analysis programs.

Data Archiving and Reporting

There are many third party software packages available to process, archive and/or report on the data collected from our systems. Vista Data Vision software can be supplied so that customers may carry out data processing, reporting and even web page generation themselves. HYDSTRA Data Management software can be set up to carry out a high degree of automated data processing and reporting, with data being available to the customer via a secure web site.

Remote Access

In many instances, ALS has remote access rights to the environment computer for the purpose of station and software maintenance. In many cases, problems can be resolved remotely, and thus saving expensive site visits from overseas based technicians. Remote access may require special software to be loaded onto the accessing computer and the user to have an active local IT account.

MAIN LOGO

LoggerNet / LoggerNetAdmin

LoggerNet is the premier software for Campbell Scientific dataloggers. It supports all forms of operational and maintenance activities associated with our stations, including monitoring station performance, datalogger configuration, manual data collection and data display. It is especially adept in applications that require telecommunications or scheduled data retrieval used in large datalogger networks. LoggerNet supports a variety of telemetry options, including direct connect, phone modems, GSM, satellite, radio and Ethernet systems.

LoggerNet includes an easy-to-use station configuration wizard. This station-oriented process will walk the user through setting up the communication path, configuring the datalogger settings, setting the datalogger clock, sending and retrieving datalogger programs (if required) and finally carrying out a communication test. Other screens are included for monitoring station performance and carrying out troubleshooting.

Programming of the datalogger can be carried out using several different

methods. The Shortcut program generator permits setting up the datalogger with simple straightforward programs, while two full-featured program editors provide more sophisticated programming capabilities. The Reporting feature Split can be used to post process data files and create reports. A View tab enables the user to view tabular displays or plot line graphs of any logged data.

Data Management Software

Vista Data Vision

Vista Data Vision (VDV) Data Management software comprises three main applications that work together to create a complete data management system.

The db.robot.c application allows the user to import data from a .dat file collected by the Loggernet software

into a database. The application monitors the .dat files, and when any new data is added, it is automatically copied over to the database. The user can easily label each variable with descriptive names and engineering units or use naming labels from the header information of the table-based .dat files from the datalogger. Old .dat files can also be manually imported into the database and experienced users can use SQL (Structured Query Language) commands to access the data in the database.

The db.data.browser and db.web.browser applications provide ways to edit and view the data either locally

or hosted as a web page. These applications allow reporting in both table and graphical formats and perform statistical calculations on the data in the database as part of the reporting process. These applications support multiple languages. The db.web.browser application includes a built-in access control to configure user

accounts and passwords and to set each user’s access to the data from various sites.

MAIN LOGO

HYDSTRA Data Management Software

The HYDSTRA Data Management Suite provides a complete solution to environmental data management, incorporating components to manage all forms of time series data, groundwater bore information and water quality data, as well as providing a mapping interface and modelling tools.

HYDSTRA is aimed at organisations that need to manage large amounts of data and provides unrivalled capabilities for:

• Data Acquisition

• Data Importing and Exporting

• Data Management

• Data Analysis

• Modelling and Simulation

• Automated Task Scheduling and Processing

• Integration with Water Quality and Groundwater databases, GIS functions and Modelling packages

• User Configuration.

HYDSTRA is widely accepted within the water resources industry as the standard software package for the processing, editing, validating and archiving of time-stamped environmental data. This is due to its unrivalled capability in data acquisition, data importing and exporting, data management, data analysis, modelling and simulation, automatic scheduling and task processing, user configurability and integration with other databases, such as Groundwater, GIS and Modelling packages.

The Time Series module is used predominately for data retrieved from dataloggers. Most Time Series collected data is at least partly wrong when it is collected, and blatantly so. Sensors drift, clocks lose time, sensors are affected by silt and algal growth, batteries run flat, water level sensors are affected by atmospheric conditions, electronic instrumentation generates spikes etc. It is vital that what is stored in the archive is as close as possible a representation of what really happened, and not immutable evidence of failure. In many cases, the data is improved by the appropriate correction and editing. When it can’t be improved, the appropriate quality code is assigned to each point of data with a pre-determined quality coding structure to highlight the accuracy of the data so that its quality and usability is considered when decision making. The goal is to ensure that the measurements are scientifically valid, legally defensible and of known acceptable precision and accuracy. Data is stored permanently in the HYDSTRA system, with full data security and daily offsite back-ups, and is stored indefinitely. Data is supplied to clients in a vast array of formats, tailored specifically to meet an individual client’s requirements.

The ALS Data Management Group combines

years of experience and expertise to provide

the best possible data management and

reporting options for its clients. A variety of

data management services are offered, with

solutions available for both in-house and

on-site, using individually customised HYDSTRA

software and tailored configurations.

Data services include:

• On-site specialised HYDSTRA and data management

• Optional centralised in-house data management

• Support for additional applications

• Secure client data web reporting

• Specialised tailored services

• Telemetered data retrieval and processing

• Other datasource data handling.

On-site Specialised HYDSTRA and Data Management

The Data Management Group is able to provide customer assistance with the installation, configuration and operation of all data management systems, including:

• Data management assistance

• Easy to use daily read import routines

• SCADA and other automated HYDSTRA imports

• HYDSTRA customisation with PERL

• Automated seasonal and group reporting with HYDSTRA HYBATCH and HYBATCHW (Word format outputs)

• Local operator exception notification and automated report provision

• Automated and semi automated laboratory results import to HYDSTRA-WQ (Water Quality) or HYDSTRA-GW (Ground Water) Modules

• HYDSTRA training

• Bureau of Meteorology Water Act reporting assistance.

ALS Data

Management Group

MAIN LOGO

Centralised Data Management

The Data Management Group offers, as an option, a full hydrometric data management service for clients, including (refer figure below):

• Permanent storage and secure backup of all data on HYDSTRA

• Optional data verification and quality coding with expert data analysis

• Rating table creation and ongoing verification

• Optional automated data sanity checking using HYDSTRA tools, including:

• Data present

• Data current

• Data within range

• Flat lining test

• Rate of change limits

• Data within rating

• Maximum intensity limits

• Nearby site comparisons

• Optional client access to ALS Citrix HYDSTRA subset with appropriate licensing and security.

Support for Additional Applications

Besides the range of services offered by HYDSTRA, the Data Management Group is able to provide a full range of support for additional applications, such as:

• ENVIROSYS data provision with automated data cleansing and tagging

• XML outputs

• Water Act meta data WDTF and HYDSTRA outputs

• Vista Data Vision

• RTMC verified data provision

• Tideda

• IQuest

• Modelling specific outputs.

Client Data Web Reporting

The Data Management Group has a number of client web reporting options, including:

• Secure password protected login

• Telemetered data retrieval and web updates at up to 5 minute intervals, or any period greater than this (hourly, daily, monthly etc)

• Individually tailored, precomputed client reporting to client specifications

• On change trigger actions – when a dataset changes, a web update runs automatically

• Optional automated Sanity Checking and action taken

• Data recorded

• Automated SMS or email triggers on condition being met (at client specified frequency, for example, daily or hourly notification)

• Optional ability to trigger data unloads and web updates via SMS by client.

The Data Management Group can configure its Web Portal to suit the individual needs of each client. On accessing the Web Portal each client will be directed to his/her individual Home page. This generally takes the form of a map front end and/or drop down lists. From here the client can choose to access information in many ways, including precomputed reports, custom reports, tabular data, etc. Reports and data files may be

downloaded by the client for use in other applications.

MAIN LOGO

OpTiOn 1

AuTOmATeD DATA ReTRievAl viA pSTn, GSm, neXT G OR SATelliTe TelemeTRY (TRADiTiOnAl DiAl up cOnnecTiOn) wiTH pRe-cOmpuTeD RepORTS DeliveReD TO SecuRe web SiTe

OpTiOn 2

AuTOmATeD DATA ReTRievAl viA GpRS, neXT G OR SATelliTe ip TelemeTRY (neAR ReAl Time pAckeT DATA) wiTH pRe-cOmpuTeD RepORTS DeliveReD TO SecuRe web SiTe

OpTiOn 3

AuTOmATeD DATA ReTRievAl fROm (TYpicAllY) clienT RADiO bASe STATiOn cOmpuTeR viA fTp wiTH pRe-cOmpuTeD RepORTS DeliveReD TO SecuRe web SiTe

TYpicAl STATiOn lATeST vAlueS web pAGe

MAIN LOGO

TYpicAl STATiOn DeTAilS web pAGe

TYpicAl STATiOn pRepAReD OuTpuTS web pAGe

Specialised Tailored Services

The Data Management Group can provide in-house development to build specific client reporting requirements, such as the following:

SpecialiSt RepoRting

THeRmiSTOR DATA GRApHicAl DiSplAYS AGAinST Time

DuST plume mAp OveRlAYS

SMS lateSt Update tRiggeRing

The Data Management Group can provide customers an optional SMS Latest Update service. This enables customers to trigger a data retrieval and processing request via SMS. The latest station parameters will then be sent by return SMS.

MAIN LOGO

Telemetered Data Retrieval and Processing

Incoming telemetered data can be automatically handled in most formats:

• FTP • GSM • GPRS • Satellite • SBD • Radio • Email • SCADA.

Other Datasource Data Handling

• LIMS to HYDSTRA Water Quality automated updates

• Automated and semi automated laboratory result import to HYDSTRA Water Quality or HYDSTRA Ground Water Modules.

Why Use ALS for Your Data Management?

we provide quick and easy access to all of your data.

ALS can store your data efficiently, reducing the amount of software you require and improving the speed at which it is processed and displayed. With HYDSTRA, we can keep tens of thousands of ‘station years’ of data safe, and analyse years of continuous data very quickly.

we can consolidate data from different sources.

ALS can import data from a variety of sources, including any datalogger, digitised charts, telemetry systems and spreadsheets. We can easily import data on a one-off basis or set up a system to routinely

import data.

we can consolidate different types of data.

ALS can combine your time-series data, water quality data, groundwater bore information and geographical information into a single integrated archive.

we provide facilities for editing and reviewing data.

In a perfect world, data would always be clean and accurate but in reality data needs to be reviewed after it is collected and often edited. With HYDSTRA, ALS provides a powerful tool for editing and reviewing data. Facilities include freehand editing, rescaling, zooming, cut and paste, and stretching. Raw data is also preserved in original format.

we allow you to assess the quality of your data.

When you are making multi-million dollar decisions based on your data, you need to know how good the data is. ALS is able to guarantee the quality of your data by storing quality information with each data point. Our systems provide facilities for routinely performing data audits and highlighting potential problems in your archive.

we provide efficient storage.

Using very efficient compression techniques, the ALS HYDSTRA Time Series software can store tens of thousands of ‘station years’ of data. All data is stored in a central archive that can be accessed by all users.

we make it easy to publish data from your archive.

Our HYDSTRA software provides a wide range of outputs that can be automatically assembled to publish in Microsoft Word format or on the internet.

we can store other information related to your data.

We can store supporting information with your data, including Station Details, Rating Tables and Shifts, Gauging Measurements, Instruments, Cross-Sections, Variable Descriptions, Variable Conversions and Data Quality Descriptions.

we can easily integrate with other systems.

We make it easy for you to get data out of your archive to use with other packages. With HYDSTRA, we can extract data from your archive in text format, dBase III database format, CSV format for spreadsheets, as well as through a DLL.

MAIN LOGO

Adelaide

Unit 2, 1 Burma Road Pooraka, SA, 5095 Phone: +61 8 8359 0890 Email: [email protected] brisbane 32 Shand Street Stafford, QLD, 4053 Phone: +61 7 3243 7222 Email: [email protected] Gladstone 46 Callemondah Drive Clinton, QLD, 4680 Phone: +61 7 4971 5600 Email: [email protected] melbourne (Springvale) 2-4 Westall Road Springvale, VIC, 3171 Phone: +61 3 8549 9600 Email: [email protected] mudgee 27 Sydney Road Mudgee, NSW, 2850 Phone: +61 2 6372 6735 Email: [email protected] newcastle 5 Rosegum Road Warabrook, NSW, 2541 Phone: +61 2 4968 9433 Email: [email protected] nowra 4/13 Greary Place North Nowra, NSW, 2541 Phone: +61 2 4423 2063 Email: [email protected] perth 10 Hod Way Malaga, WA, 6090 Phone: +61 8 9203 7655 Email: [email protected] wollongong 99 Kenny Street Wollongong, NSW, 2500 Phone: +61 2 4225 3125 Email: [email protected] Sydney 277-289 Woodpark Road Smithfield, NSW, 2164 Phone: +61 2 8784 8555 Email: [email protected] Townsville 14-15 Desma Court Bohle, QLD, 4818 Phone: +61 7 4796 0600 Email: [email protected] melbourne (Scoresby)

Caribbean Business Park 22 Dalmore Drive Scoresby, VIC, 3179 Phone : +61 3 8756 8000 Email: [email protected] Geelong 49 Carr Street

Geelong South, VIC, 3220 Phone : +61 3 5226 9249 Email: [email protected] wangaratta 48 Faithful Street Wangaratta, VIC, 3676 Phone : +61 3 5722 2688 Email: [email protected] bendigo

Gate 6, Sharon Street La Trobe University, Bendigo (PO Box 106) Strathdale, VIC, 3550 Phone : +61 3 5441 0700 Email: [email protected] brisbane 32 Shand Street Stafford, QLD, 4053 Phone: +61 7 3243 7222 Email: [email protected] Sydney

277-289 Wood Park Road Smithfield, NSW, 2164 Phone: +61 2 8784 8555 Email: [email protected] Traralgon Gippsland Water Hazelwood Road Traralgon, VIC, 3844 Phone : +61 3 5176 4170 Email: [email protected] canberra 16B Lithgow Street

(PO Box 1834) Fyshwick, ACT, 2609 Phone : +61 2 6202 5400

Email: [email protected]

brisbane

Suite 10, Yeerongpilly Corporate Park 44 Station Road

Yeerongpilly, QLD, 4105 Phone : +61 7 3859 7800

Email: [email protected]

Sydney

24A Lemko Place Penrith, NSW, 2750 Phone : +61 2 4721 3477 Email: [email protected] canberra 16B Lithgow Street (PO Box 1834) Fyshwick, ACT, 2609 Phone : +61 2 6202 5400 Email: [email protected] melbourne (Scoresby)

Caribbean Business Park 22 Dalmore Drive Scoresby, VIC, 3179 Phone : +61 3 8756 8000

Email: [email protected]

GeneRAl enviROnmenTAl GROup - lAbORATORieS

wATeR ReSOuRceS GROup - lAbORATORieS

wATeR ReSOuRceS GROup - wATeR ScienceS GROup

canberra 16B Lithgow Street (PO Box 1834) FYSHWICK ACT 2609 Australia Phone: +61 2 6202 5400 Fax: +61 2 6202 5455 Email: [email protected] Sydney

24A Lemko Place PENRITH NSW 2750 Australia Phone: +61 2 4721 3477 Fax: +61 2 4721 5699 Email: [email protected] mackay 78 Harbour Road (PO Box 2009) MACKAY QLD 4740 Australia Phone: +61 7 4944 0177 Fax: +61 7 4944 0722 Email: [email protected] Rockhampton Unit 2/11 Sturt Street (PO Box 1622) ROCKHAMPTON QLD 4700 Australia Phone: +61 7 4921 2599 Fax: +61 7 4921 4297 Email: [email protected] melbourne (Scoresby)

Caribbean Business Park 22 Dalmore Drive SCORESBY VIC 3179 Australia Phone: +61 3 8756 8000 Fax: +61 3 9763 1862 Email: [email protected]

fOR All inSTRumenT

SAleS AnD SeRvice

fOR All DATA mAnAGemenT

enquiRieS

Data management Group

24A Lemko Place PENRITH NSW 2750 Australia

Phone: +61 2 4721 3477 Fax: +61 2 4721 5699

Email: [email protected]

wATeR ReSOuRceS GROup - mOniTORinG AnD TecHnicAl SeRviceS

Brisbane 2 Blyth Street STAFFORD QLD 4053 Australia Phone: +61 7 3243 7222 Fax: +61 7 3552 8629