7.2 Demonstrating monitoring and analysis capabilities
7.2.1 Cloud Application Platform provider’s perspective
Primarily, the EXCLAIM framework is intended to support and enhance the CAP provider’s capabilities for platform governance, and therefore the provider is seen as the main stakeholder. We have already raised several issues the CAP provider might be concerned with. Among other things, one of the key challenges in the presence of numerous virtual tenants simultaneously accessing a common shared pool of cloud resources is to ensure that the amount of the underlying physical resources is sufficient. This underpins stable and undisrupted operation of the cloud services.
Given the motivation of the provider to minimise the amount of idle physical resources (for economic and ecological reasons), it is typically expected that phys- ical servers are turned on and off dynamically in response to the ever-changing demands of the cloud platform consumers. In these circumstances, it is crucial for the cloud provider to be provisioned with a most recent overview of the overall resource consumption resulting from resource consumption of individual tenants. More specifically, let us focus on Heroku’s PostgreSQL data storage service, used by Destinator. When subscribing to this service, each subscriber is allocated a personal virtual space on a physical database server. Heroku aims at reduc- ing operational costs by minimising the total number of physical database server instances running at the same time. This is typically achieved by increasing the ‘tenant density’ (i.e., number of virtual tenants on each of the physical servers), and, following the principle of elasticity, launching additional database servers only in case the existing capacity is not enough. Accordingly, the CAP provider needs to measure disk space utilisation of individual virtual tenants with respect to the total available physical space in order to launch additional server if needed or ‘merge’ several already running, but idle servers.
First, the framework needs to measure disk space utilised by an individual instance of the PostgreSQL service couple with Destinator. The next step is to sum up the collected individual values to get an overall occupied amount of Post-
greSQL disk space by Destinator. Finally, this value needs to be evaluated against the rest of the disk space occupied by other instances of PostgreSQL connected with other applications (i.e., not Destinator) on Heroku. Thus, the framework will be in a position to decide whether there is still enough space for the PostgreSQL service or not.
Below, we illustrate how this kind of use case scenarios is handled by the EX- CLAIM framework to support cloud platform self-governance. The explanation is aligned with the three main steps, through which raw collected data is trans- formed as it passes through the EXCLAIM framework.
Triplification
The code snippet below (Listing 7.1) represents a stream, associated with the particular user instance of Destinator, to which all related RDF triples are sent. The snippet demonstrates how the disk space occupied by a PostgreSQL instance postgres-service-10changes with the time from 7,000 rows to 9,000 rows. Listing 7.1: RDF triples on the stream indicate database space, occupied by an individual service.
cso:postgres-service-10 rdf:type cso: HerokuPostgresService cso:postgres-service-10 cso:hasDatabaseSize cso:database-size-122 cso:database-size-122 rdf:type cso:DatabaseSize
cso:database-size-122 cso:hasValue "7000"^xsd:int
cso:postgres-service-10 rdf:type cso: HerokuPostgresService cso:postgres-service-10 cso:hasDatabaseSize cso:database-size-122 cso:database-size-122 rdf:type cso:DatabaseSize
cso:database-size-122 cso:hasValue "7500"^xsd:int
cso:postgres-service-10 rdf:type cso: HerokuPostgresService cso:postgres-service-10 cso:hasDatabaseSize cso:database-size-122 cso:database-size-122 rdf:type cso:DatabaseSize
cso:database-size-122 cso:hasValue "8000"^xsd:int
cso:postgres-service-10 rdf:type cso: HerokuPostgresService cso:postgres-service-10 cso:hasDatabaseSize cso:database-size-122 cso:database-size-122 rdf:type cso:DatabaseSize
cso:database-size-122 cso:hasValue "8500"^xsd:int
cso:postgres-service-10 rdf:type cso: HerokuPostgresService cso:postgres-service-10 cso:hasDatabaseSize cso:database-size-122 cso:database-size-122 rdf:type cso:DatabaseSize
C-SPARQL querying
The next step is extract triples representing disk space occupation from the RDF stream. This task can be achieved by registering the following C-SPARQL query (Listing 7.2). The query only fetches real values, which are greater than zero, thus filtering out values coming from applications, which are staying idle, and thereofre should not be taken into consideration. The resuts of the query are sent forward to CriticalStream, which serves to deliver potentially critical triples to the OWL/SWRL reasoning engine.
Listing 7.2: A C-SPARQL query fetching the current value of the disk space utili- sation by the HerokuPostgres service.
REGISTER STREAM CriticalStream
AS PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> PREFIX cso: <http://seerc.org/ontology.owl#>
CONSTRUCT { ?service rdf:type cso:HerokuPostgresService . ?service cso:hasDatabaseSize ?size .
?size rdf:type cso:DatabaseSize . ?size cso:hasValue ?value }
FROM STREAM <http://seerc.org/rdf/destinator-stream/> [RANGE 1m STEP 1s] WHERE { ?service rdf:type cso:HerokuPostgresService .
?service cso:hasDatabaseSize ?size . ?size rdf:type cso:DatabaseSize . ?size cso:hasValue ?value . ?value > 0 }
AGGREGATE { (?totalvalue, SUM, {?value}) FILTER (?totalvalue > 1000000) }
OWL/SWRL reasoning
At the previous step, we assumed that all instances of Destinator (and therefore all instances of the HerokuPostgres service associated with it) send its measured val- ues to a dedicated data stream (i.e., in the examples above – http://seerc.org/- rdf/destinator-stream/). In these circumstances, an individual C-SPARQL query registered with this stream can only process values coming from this par- ticular application and is incapable of providing a more global view on the overall disk space consumption by the HerokuPostgres service connected to other Heroku applications. In these circumstances, it is important to gather information coming from various data streams in one place, and see if the overall utilisation is still within limits. The SWRL reasoning engine is intended to play the role of such a central component. In this sense, it is akin to a central component of a sensor network, which collects data coming from distributed sources.
All values appearing on CriticalStream are added to the CSO by the EX- CLAIM framework, and the reasoning process is initiated. Listing 7.3 depicts an SWRL policy which sums up the total amount of disk space occupied (for demonstration purposes we assume the presence of only five applications using the HerokuPostgres service).1
Listing 7.3: New SWRL rule detecting situations when the overall database size dedicated to the HerokuPostgres service (shared by five different applications) is critical.
HerokuPostgresService(?ser1) ^ HerokuPostgresService(?ser2) ^ HerokuPostgresService(?ser3) ^ HerokuPostgresService(?ser4) ^ HerokuPostgresService(?ser5) ^
DatabaseSize(?size1) ^ DatabaseSize(?size2) ^ DatabaseSize(?size3) ^ DatabaseSize(?size4) ^ DatabaseSize(?size5) ^
hasDatabasSize(?ser1,?size1) ^ hasDatabasSize(?ser1,?size1) ^ hasDatabasSize(?ser1,?size1) ^ hasDatabasSize(?ser1,?size1) ^ hasDatabasSize(?ser1,?size1) ^
hasValue(?size1, ?v1) ^ hasValue(?size2, ?v2) ^ hasValue(?size3, ?v3) ^ hasValue(?size4, ?v4) ^ hasValue(?size5, ?v5) ^
swrlb:greaterThan(1000000000, op:numeric-add(?v1, ?v2, ?v3, ?v4, ?v5)) -> CriticalTotalDatabaseSize(?ser1) ^ CriticalTotalDatabaseSize(?ser2) ^
CriticalTotalDatabaseSize(?ser3) ^ CriticalTotalDatabaseSize(?ser4) ^ CriticalTotalDatabaseSize(?ser5)
With SWRL it is also possible to combine dynamic values coming from the monitoring stream with a more static facts. We can modify the previous SWRL example with an assumption that the total amount of disk space occupied by HerokuPostgres services being greater than the dangerous threshold is only criti- cal if there are no additional physical servers available to the CAP provider. Other- wise, it would be possible to elastically provision extra storage using the additional server. To represent this kind of policy with SWRL we assume the presence of a ContextObjectinstance PhysicalServer in the CSO – this instance does not come from the monitored stream, but is part of the background static knowledge. Accordingly, the modified SWRL rules is depicted in Listing 7.4.
Listing 7.4: A critical situation is detected only when there does not exist an addi- tonal server.
HerokuPostgresService(?ser1) ^ HerokuPostgresService(?ser2) ^ HerokuPostgresService(?ser3) ^ HerokuPostgresService(?ser4) ^ HerokuPostgresService(?ser5) ^
1Please note how we make use of the SWRL built-in operators op:numeric-add and swrlb:greaterThan, which are used to calculate a sum of several values and check if it is greater than some other value respectively. The number 1,000,000,000 indicatively represents a dangerous threshold for the HerokuPostgres service.
DatabaseSize(?size1) ^ DatabaseSize(?size2) ^ DatabaseSize(?size3) ^ DatabaseSize(?size4) ^ DatabaseSize(?size5) ^
hasDatabasSize(?ser1,?size1) ^ hasDatabasSize(?ser1,?size1) ^ hasDatabasSize(?ser1,?size1) ^ hasDatabasSize(?ser1,?size1) ^ hasDatabasSize(?ser1,?size1) ^
hasValue(?size1, ?v1) ^ hasValue(?size2, ?v2) ^ hasValue(?size3, ?v3) ^ hasValue(?size4, ?v4) ^ hasValue(?size5, ?v5) ^
swrlb:greaterThan(1000000000, op:numeric-add(?v1, ?v2, ?v3, ?v4, ?v5)) ^ DatabaseServer(?dbs) ^ exists(?dbs, false) ->
CriticalTotalDatabaseSize(?ser1) ^ CriticalTotalDatabaseSize(?ser2) ^ CriticalTotalDatabaseSize(?ser3) ^ CriticalTotalDatabaseSize(?ser4) ^ CriticalTotalDatabaseSize(?ser5)