The software design required sampling of sensors, scheduling of function executions and ow events, and storage of the recorded data. The three aspects are interlinked and are thus di cult to explain in isolation. Experiments were designed and executed (using scheduling) to generate speci c, useful data. The data generated during an experiment performed using the experimental unit created a Dataset. The generated Datasets are
1Thanks to Lowku Leeuwner who was provided with the design speci cations and requirements and
CHAPTER 4. EXPERIMENTAL UNIT DESIGN 56 discussed further in Section 4.5. Further information relating to sampling, scheduling and storage can be found in Section B.2.
The data sampling and storage script, or "acquisition system", was executed on the RPi. The acquisition system was coded using Python 2.7 to ensure that all the required python packages were compatible. Python packages which were heavily used in the acquisition system were Pandas to store and manipulate local data, APScheduler ('Advanced Python Scheduler') to control when functions were executed, and SQLAlchemy to connect to the MySQL database to store recorded data. A library was available to setup LSM303 accelerometer which reduced the development time.
The acquisition system was used to sample all the sensors, performed the scheduled ope- ning and closing of SV to induce ow events according to a pre-determined schedule and stored the recorded data to a local MySQL database. The data stored in the MySQL da- tabases generated Datasets 1, 2 and 3 which were post-processed as described in Chapter 5 to create and evaluate the non-invasive ow estimation system.
4.4.1 Sampling
The sampling was executed using an APScheduler scheduler object which had a job to control the execution time of the sampling function (discussed in Section 4.4.2). The sampling can be divided into two sections for clarity: the primary sweep (time, tempe- ratures, ow rate, state of SV) and vibration burst sampling (high frequency sampling of accelerometer data for N samples). The primary sweep was performed at 1 s intervals. Vibration burst sampling generated large amounts of data, therefore vibration priority ags were used to determine how often a burst sample was made. During low priority times, a burst sample was performed once per minute, and during high priority times a burst sample was performed once per second. Both the primary sweep and vibration burst were executed in the same function, but the execution or skipping of the vibration burst was controlled using ags.
4.4.1.1 Primary Sweep
The primary sweep was executed at a sampling frequency of 1 Hz. Once per second the following was executed:
Store DateTime and Microsecond of current sample
Store and clear GPIO interrupt incremented FFM and CFM pulse count values Store state of SV to know if water was owing
Perform I2C sweep of 8 temperature sensors, store the integer representation If vibration sampling is required: execute vibration burst sampling
Append the locally stored values of current sample to global 'acquisition' list The sampled points were stored locally until the completion of the sample and then appended to the global 'acquistion' list storing the samples. GPIO interrupts on the FFM and CFM pins were enabled and the number of pulses between samples were incremented
using these GPIO interrupt handlers. The primary sweep thus recorded the samples per second value for each ow meter. The integer representation of the temperature values refers to the fact that the raw output of the TMP275 was stored directly. The output must be divided by 16 to obtain the actual temperature (to 0.0625◦C resolution). The raw values were stored as integers and not as oating point numbers to minimize storage requirements because the conversion in post processing is simple.
4.4.1.2 Vibration Burst Sampling
As stated in Section 4.4.1, burst sampling of the LSM303 accelerometer was performed in the same function as the primary sweep but during low priority times then the burst sample was not performed. Vibration sampling was performed as burst sweep of 500 samples and a sampling frequency of ≈ 1 kHz was achieved. The high frequency vibration sampling of the 3 accelerometer axes meant that a large amount of data was generated, much of which was not required for the ow estimation algorithm to function. The minimum number of samples required to produce a useful standard deviation value was unknown until after Dataset 1 was post-processed. The largest number of vibration samples which could be captured and stored each second, without aecting the primary sweep, was found to be 500. During low priority times (when no ow events were scheduled to occur) a vibration burst sample was performed once every minute. During high priority times (i.e. 5 min prior to a scheduled test start until 5 min after the completion of a scheduled test) a vibration burst sample was performed every second. Data size was thus minimized while maintaining high temporal resolution vibration data during times where water ow was known to occur for the initial data acquisition. High resolution vibration data was stored in a separate Pandas DataFrame and a separate MySQL database. The vibration data was immediately inserted into the 'vibration' MySQL database when the burst sample was completed.
The vibration priority sampling was used during the initial development of the system. When the minimum required vibration data format was established (after post-processing Dataset 1) then vibration data was gathered in real time. A high frequency sweep of 250 samples per accelerometer axis was found to be su cient, and the standard deviation of each 250 sample burst sweep could be stored directly. This means that 1 Hz sampling is possible for the acquisition system without requiring vibration priority control, and that the generated data is not impractically large. Datasets 2 and 3 recorded the required vibration data at 1 Hz.
4.4.2 Scheduling
The scheduling was performed using the APScheduler Python module. An APScheduler scheduler object was created, the required jobs were added to the scheduler. A blocking scheduler object was used which ran in the foreground. A job makes a call to a function at speci c times when trigger conditions are met.
APSchduler has 3 types of triggers (which are used to execute jobs): date, interval and cron. Date is used when the job must run only once at a certain time. Interval is used when the job must run at xed time intervals. Cron is used when a job must run periodically according to date and time conditions. The following important APScheduler jobs were used in the acquisition system:
CHAPTER 4. EXPERIMENTAL UNIT DESIGN 58 Interval triggering was used to execute the primary sweep at 1 Hz for temperature,
ow and SV state data
Cron triggering was used to open/close SV to control scheduled ow events
Cron triggering was used to toggle the vibration burst sampling intervals using an internal ag (enabling high resolution vibration sampling only for scheduled ow events)
Cron triggering was used to write local Pandas DataFrames containing acquired data to the relevant MySQL database once hourly
It was found that certain temperature sensors occasionally reset to the default 9 −bit re- solution after SV was toggled during scheduled water ow control. This was rst observed during the preliminary thermal tests and was shown in Figure 3.7. Shielded 4-conductor wire was used for all I2C devices in the experimental, unit but further steps were re- quired. To ensure that maximum resolution was maintained during times of ow; the primary sweep job was paused after SV toggles were completed, the resolution of each TMP275 sensor was con rmed, and the primary sweep was resumed. This ensured that all temperature sensors were set to 12 −bit resolution during ow tests.
The automated control of the SV required scheduling to ensure repeatable experiments. The sampling of the data also required scheduling. The vibration priority control was performed using scheduling when relevant. The 1 Hz sample rate of the primary sweep was also controlled using scheduling and interval triggering. Data storage to the 'acquisition' MySQL database was also controlled using an APScheduler job.
4.4.3 Storage
A MySQL 5.5 server was hosted locally on the RPi. The Pandas Python module, which was used to store the samples prior to being inserted into the MySQL database, has many convenient features which are available when using the SQLAlchemy SQL toolkit. SQLAlchemy was thus used to connect to the MySQL server and connect the sampled data to a MySQL Database. Two separate MySQL databases were required for the rst experimental unit dataset gathered (Dataset 1). The minimum vibration requirements were then determined and subsequent experiments performed vibration sampling at the same frequency as the temperature and ow rate measurements, therefore a single MySQL Database was required to store the 1 Hz data.
4.4.3.1 Acquisition Database
A Pandas DataFrame was created from acquisition samples list containing 1 Hz tempera- ture, ow and valve state information was inserted into the 'acquisition' MySQL database at 50 min of each hour and then cleared locally. This kept the row depth of the Pandas DataFrame to under 3600 samples and MySQL insert operations on the RPi were observed to complete in an acceptable time for this interval.
The acquisition database was used to store all measured values which were sampled at a frequency of 1 Hz. The sensor values contained within each stored sample can be found in Table B.5 in Section B.2.
4.4.3.2 Vibration Database
The burst vibration sweep had a much higher sample rate than the primary sweep and thus a large amount of data was generated each second. The raw vibration data was inserted into a separate database, the 'vibration' MySQL database, after the desired number of burst sampled accelerometer readings were gathered.
The vibration MySQL database contained the accelerometer data which consisted of 500 accelerometer g-force samples per accelerometer axis for each entry (for Dataset 1). Each entry in the vibration MySQL database thus contained a DateTimeIndex value, an integer Microsecond value, and 3 integer g-force readings for x0, y0 and z0 axes. 500 entries were stored for each burst sample. The burst sweep was performed once per second during scheduled ow events and once per minute when no ow events were scheduled to reduce the data size.
The vibration database for Dataset 1 was used to determine the minimum accelerome- ter samples required to calculate the standard deviation values, σ, which were used in the ow estimation algorithm discussed in Chapter 5. After the minimum vibration re- quirements were established then the acquisition script was altered to perform real-time vibration sampling. This improvement resulted in two bene ts: vibration sampling could be performed in real time, and the amount of vibration data per second was reduced to 3 oating point values. A separate vibration database was therefore not required for any data obtained after Dataset 1 had been processed.