Systems
Architecture and Data Model
We are seeking stakeholders’ views on the questions set out in this consultation document. If you have any comments on the paper please contact: [email protected]
Publication Date: 2 January 2014
Response deadline: 14 February 2014
Contents
1. Executive summary 4
1.1 Required market operator IT systems 4
1.2 Industry data exchange hub interface approach 4
1.3 Market data model 4
2. Introduction 5
2.1 Programme background 5
2.2 Document purpose and context 5
3. Market participants and market facilitation 8
4. Required IT systems 9
4.1 Approach to data exchange and required interfaces 10
5. Services offered by the market operator 12
5.1 Types of IDEX interface 12
5.2 Transmission 14
5.3 Storage 15
5.4 Validation 16
5.5 Indicative event and data volumes 17
6. High-level market conceptual data models 20
6.1 Conceptual model for registration data 21
6.2 Conceptual model for service request data 22
6.3 Conceptual model for consumption and financial settlement data 22
Appendix A: Consultation questions and approach 25
Appendix B: Glossary of terms 27
Appendix C: Customer expectations assessment 28
Appendix D: Working group comments 29
1. Executive summary
This document sets out our high-level design proposals for the systems architecture and data model and the scope of message services that the market operator (MO) offers within the new retail market.
1.1 Required market operator IT systems
To enable the MO to deliver services in the areas of registration and switching, financial settlement, market governance, and industry data exchange, IT systems will be required, most notably:
a registration system and database;
a meter reading preparation system, charge calculation system, and associated databases;
a management information/business intelligence system; and
an industry data exchange hub, including a message validation system and message database.
1.2 Industry data exchange hub interface approach
The MO should be responsible for co-ordinating data exchange within the industry. To enable this, an industry data exchange (IDEX) hub will be created, through which market participants will send and receive data. Reflecting the different frequencies with which different market processes will be employed, and the different scale of market participants, the following three types of interface will be implemented.
An automated interface, expected to be for machine-to-machine communication and used for transferring large volumes of data at low cost.
A semi-manual interface for participants to upload data files to and download data files from manually, to be used as a contingency or as a low-cost alternative for new or small market entrants with lower volumes of data to be transferred.
A manual interface such as set of secure web forms to enter data into manually, to be used as a contingency or as a very low-cost alternative for new or small market entrants with lower volumes of data to be transferred.
The transmission of all standard messages between market participants should be through the IDEX hub and mandatory for all participants, with the MO validating and storing the data contained in such
messages. The MO should also enable the transmission of all non-standard messages through the IDEX hub. Use of this will be optional for participants, but where it is used, the MO will store, but not validate, message data.
The IDEX is only intended to support the electronic transfer of information between participants and/or the MO. It does not cover verbal communications.
1.3 Market data model
The data model for the market, and for the MO in particular, includes:
for registration and switching, a centrally held record of premises, service points, and associated meters and market participants. The central data model will not hold customer-level data;
for operational services, a centrally held record of service requests and notifications; and
for metering and financial settlement, a centrally held record of meter readings, wholesale charging schemes, and derived wholesale charges.
2. Introduction
2.1 Programme background
The UK Government’s Water Bill1 was introduced into Parliament and published on 27th June 2013. The Water Bill is designed to address the current and future challenges faced by the water and sewerage sector, which were described in the Water White Paper2.
Among other things, the Water Bill is designed to:
increase customer choice;
improve service provision;
stimulate innovation; and
drive more sustainable approaches to managing our scarce resources.
The Government’s key reforms are:
the introduction of retail competition for water and sewerage services to non-household customers in England, which will be in place from April 2017; and
the introduction of competition in the upstream sector, which will take place at a later date (after 2019).
The Open Water programme has been created to facilitate the implementation of the proposed reforms.
2.2 Document purpose and context
This document is part of a suite of materials being published throughout 2014. They set out the Open Water programme’s recommendations for the high-level design for the new competitive water and sewerage retail market for non-household customers in England. These materials are:
a market blueprint, which describes the present and future market arrangements and the different roles in the new market arrangements, and summarises the programme’s recommendations for the high-level market design;
a series of documents on strategy and high-level design (of which this is one), which present in more detail the programme’s recommendations in areas such as registration and switching, financial settlement, and industry governance and performance management; and
supporting discussion papers and option analyses, which have informed the documents listed above.
The intended audiences for the high-level design documents are:
the strategy, regulation and change teams within incumbent and new entrant water companies in England;
Ofwat;
potential providers of services and systems to a new central MO and/or to water companies; and
anyone with an interest in the reform of the water and sewerage sector.
1 Water Bill 2013-14. http://services.parliament.uk/bills/2013-14/water/documents.html
2 Water for Life – Market reform proposals. https://www.gov.uk/government/publications/water-for-life-market- reform-proposals
This document sets out the Open Water programme’s recommendations for how the systems
architecture and data Model should be configured in the new retail market – in particular, by explaining:
market participants and market facilitation activities (chapter 2);
the MO and market participants’ required IT systems and how data exchange is to be managed (chapter 3);
services that the MO offers in relation to data exchange, message transmission, validation and storage; including indicative event and data flow volumes (chapter 4); and
the conceptual data models for registration, financial settlement and operational services (chapter 5).
The design recommendations set out in this document have been developed with consideration of their impact on and response to wider issues, including how they would:
ensure a level playing field for market participants;
support market consistency both within England and between the English market and the markets in Wales and Scotland;
reflect customers’ expectations of how they hope to see the retail market operate;
align with the later introduction of upstream markets; and
strike an appropriate balance between scale and complexity, deliverability and the benefits they will generate.
In creating this document, the Open Water team has reviewed and considered the market designs and associated codes for, and met market participants from the:
Irish and British electricity markets;
British gas market; and
Scottish water market.
We have also met with market operators and experts in these markets such as Ofgem, the Central Market Authority (CMA), the Water Industry Commission for Scotland (WICS), Elexon and Electralink.
The recommendations set out in this document have been discussed by:
an industry working group;
Ofwat’s Choice and Trading Arrangements Programme Board members;
Open Water’s Programme Delivery Board; and
Open Water’s High Level Group.
Feedback from these groups has been considered and reflected in the proposals made.
Throughout this document we ask a number of questions and seek stakeholders’ views on these. We list all of the questions in appendix A. Please provide your responses to the consultation questions and any other comments or queries you may have regarding this paper by 14th February 2014, to
[email protected]. We provide an accompanying template for responses on the Open Water website to help this process, which we strongly encourage respondents to use. We will, however, accept responses in other formats if necessary.
We will be running a workshop on 29th January for representatives from water companies to discuss the content presented in all of the high-level design papers. Details of this session have been shared with water companies, and for more information please contact [email protected].
A second iteration of this document will be issued in early summer 2014. This will include any changes necessary to align with additional strategy and high-level design papers which will be produced in early summer 2014, and will also reflect any changes made in response to consultation responses received.
The recommendations set out in this document are intended to facilitate wider discussion about the changes. Following the consultation, the updated recommendations will act as a recommendation to Ofwat, for the relevant regulatory decisions in due course.
3. Market participants and market facilitation
From 1st April 2017, a new retail market for water and sewerage services will be introduced in England.
Market participants that will operate in the new retail market will comprise the following.
Wholesalers:
o Incumbent water and sewerage companies (WaSCs), water only companies (WOCs) and new appointments and variations (NAVs) will evolve to provide wholesale services to their own and new entrant retailers on a non-discriminatory basis.
Retailers:
o Incumbent retailers, including the existing WaSCs, WOCs and NAVs, acting as the default retailer for customers within their area of appointment.
o New entrant retailers, including the competitive retail arms of any existing water companies operating anywhere in the country, and any other new market entrants.
Market operator:
o A new MO providing services in the areas of registration and switching, financial settlement, market governance, and industry data exchange.
Further details about types and roles of future market participants can be found in section 4.3 of the market blueprint.
To enable the new retail market to function effectively, new market facilitation activities are required to enable:
registration and switching – maintaining a record of service points and the registered parties for providing different services to each service point, and enabling the registered parties to be switched;
financial settlement – determining and processing the financial payments between companies to pay for services provided; and
industry data exchange – passing relevant data between market participants to enable market processes to be executed.
For the full list of services provided, more detail on those stated above, and information supporting the decisions can be found in the market blueprint.
We are seeking your views on the recommended scope of services provided by the MO related to registration and switching through consultation questions in the registration and switching strategy.
We are seeking your views on the recommended scope of services provided by the MO related to financial settlement through consultation questions in the financial settlement strategy.
4. Required IT systems
Key recommendations made in this chapter
To enable the MO to deliver services in the areas of registration and switching, financial settlement, market governance, and industry data exchange, IT systems will be required, most notably:
a registration system and database;
a meter reading preparation system, charge calculation system, and associated databases;
a management information/business intelligence system; and
an industry data exchange hub, including a message validation system and message database.
To deliver the activities described in chapter 3. wholesalers, retailers and the MO will require IT systems.
The types of system/the high-level system functionality required are shown as the green items in Figure 1and described below.
Figure 1: IT systems and interfaces
Wholesalers and retailers will both require systems to support their business activities. For example, a retailer will require billing and collection systems. It is the responsibility of such companies to define their own IT plans, but it might be reasonably expected that:
most of the required systems will already exist in the IT estate of an incumbent water company, but that some separation or modification may be required, for example to support processes for
customers changing Retailer.
some new IT systems may be required where Wholesalers and Retailers have to provide input to or take action based on new market processes, for example to validate financial settlement charge calculations made by the Market Operator.
retailers may at their discretion choose to make additional IT investments as part of a competitive market strategy, for example investing in enhanced marketing systems to support proactive customer retention.
To deliver registration and switching services to the market, the MO will require systems that provide the following functionality.
A registration database containing the relevant information regarding sites and the registered parties for providing different services to each site.
A registration system for managing registration processes – for example, updating data in the registration database and performing reviews and audits of registration data.
A business/market intelligence (BI/MI) system for capturing market process performance data to provide visibility of performance to enable market monitoring activities.
To deliver financial settlement services to the market, the MO will require systems that provide the following functionality.
A meter read database containing all required meter readings for performing financial settlement calculations.
A meter data preparation system to perform any validation, editing, estimating and/or aggregation of meter reads that may be required to determine the appropriate consumption values to be used in financial settlement.
A charge calculation system to apply charging rules to processed meter read data to determine the financial charges due between wholesalers and retailers.
A financial transaction database to provide a record of all financial charges calculated in the market and to store all other relevant data.
It is expected that as and when systems are procured or built to provide the MO functionality outlined above, it may be the case that particular systems can perform a number of functions.
4.1 Approach to data exchange and required interfaces
For the market to function successfully, the IT systems of wholesalers, retailers and the MO will need to exchange data. The approach for data exchange in the market is that data which supports market processes will be in a standard format and communicated between parties through a single IDEX hub.
This, together with the key flows of data, are shown as the pink items in Figure 1and described below.
The MO will be responsible for co-ordinating data exchange within the industry. To enable this, the following systems and functions are envisaged.
Data exchange interfaces with which wholesalers and retailers send and receive data. These
interfaces would then push that data on to the intended recipient(s). It is expected that there will be up to three types of interface:
o an automated interface, expected to be for machine-to-machine communication and used for transferring large volumes of data at low cost;
o a semi-manual interface for participants to upload data files to and download data files from manually, to be used as a contingency or as a low-cost alternative for new or small market entrants with lower volumes of data to be transferred; and
o a manual interface such as set of secure web forms to enter data into manually, to be used as a contingency or as a very low-cost alternative for new or small market entrants with lower volumes of data to be transferred.
A message verification and validation system to confirm and accept or reject transmitted messages based on predefined rules.
A message database to maintain a central record of some or all of the data exchanged between market participants.
The MO’s functional IT systems for registration and financial settlement would be integrated with the industry data exchange hub. It would be through the hub that data associated with these activities is exchanged with wholesalers and retailers.
Performance of processes captured within the BI/MI statistics system will be integrated with the industry data exchange hub, and made available to market participants through the use of a web form interface
To exchange data with the IDEX hub, wholesalers and retailers will need to implement changes that facilitate the passing of data to/from their functional IT systems. While it is for these companies to develop and deliver their own plans, it may be reasonably expected that these changes could include:
data preparation systems, which produce market message files, transforming the data from the format used in functional IT systems into the format required by market processes (and vice versa); and
an industry data exchange interface, which manages the transmission of data from the functional systems to the industry data hub (and vice versa).
If a wholesaler or retailer confirmed they required it, the capability outlined above might be:
delivered in stand-alone systems;
embedded in functional IT systems; or
carried out in an alternative manner.
Consultation questions:
SD 4a: Do you think that the proposed systems architecture is appropriate for enabling the services that are being recommended for delivery by the MO and market participants?
SD 4b: Do you agree that the services within the IDEX (automated, semi-manual and manual) sufficiently covers the data transmission requirements of market participants?
5. Services offered by the market operator
Key recommendations made in this chapter
The MO will be responsible for co-ordinating data exchange within the industry. To enable this, an industry data exchange (IDEX) hub will be created, through which market participants will send and receive data. Reflecting the differing frequencies with which different market processes will be employed, and the different scale of market participants, three types of interface will be implemented.
An automated interface, expected to be for machine-to-machine communication and used for transferring large volumes of data at low cost.
A semi-manual interface for participants to upload data files to and download data files from manually, to be used as a contingency or as a low-cost alternative for new or small market entrants with lower volumes of data to be transferred; and
A manual interface such as set of secure web forms to enter data into manually, to be used as a contingency or as a very low-cost alternative for new or small market entrants with lower volumes of data to be transferred.
The MO should be responsible for transmitting, validating and storing standard market messages between participants. It should be mandatory to use this MO service.
The MO should provide the capability for transmitting and storing (but not validating) non-standard messages, and use of this service should be optional.
5.1 Types of IDEX interface
As we explained in section 4.1 , it is intended there may be up to three interfaces through which a
wholesaler or retailer may be required to, or may choose to, send or receive market data. We explain the current thinking behind such an approach below, and give examples of when the different interfaces may be most appropriate.
Figure 2: IDEX hub interface choices
5.1.1 The automated IDEX Interface
For many common events, such as managing a customer changing their retailer, it is important for the data transfer approach to be fast and cost effective. In this scenario, it is envisaged that most
wholesalers and retailers would make use of automated data exchange, and that the process to produce and pass data to the market could be as marked in Figure 2 as route (A-A-A) and described below.
The underlying functional system (for example, the retailer’s Customer Relationship Management (CRM) system, manages the workflow associated with the activity and produces data that needs to be passed to another market participant.
A data preparation system (which may be part of the functional system) translates the data into agreed market data formats and produces a market data file.
An IDEX Interface automatically passes the market data file through to the MO’s automated IDEX Interface.
5.1.2 The semi-manual IDEX Interface
For events that are less frequent, or for all events for smaller wholesalers and retailers, it may not be cost effective to develop or purchase an IDEX Interface. In these scenarios, it is envisaged that wholesalers and retailers may use the semi-manual data exchange. In addition, wholesalers and retailers that use the automated IDEX Interface may choose to be able to use the semi-manual IDEX Interface as a contingency. The process to produce and pass data to the market could be as marked in Figure 2 as route (B-B) and described below.
The underlying functional system (for example, the retailer’s CRM system) manages the workflow associated with the activity and produces data that needs to be passed to another market participant.
A data preparation system (which may be part of the functional system) translates the data into agreed market data formats and produce a market data file.
A user uploads the produced market data file to the MO’s semi-manual IDEX interface (for example, through a secure market website).
5.1.3 The manual IDEX interface
For events that are even less frequent, or for all events for smaller wholesalers and retailers, it may not be cost effective to develop or purchase both an IDEX Interface and a data preparation system. In these scenarios, it is envisaged that wholesalers and retailers may use the manual data exchange. In addition, wholesalers and retailers that use the automated and/or semi-manual IDEX interfaces may choose to be able to use the manual IDEX interface as a contingency. The process to produce and pass data to the market could be as marked in Figure 2 as route (C) and described below.
The underlying functional system (for example, the retailer’s CRM system) manages the workflow associated with the activity and produces data that needs to be passed to another market participant.
A user manually enters the required data into a secure web form (or similar product) that is part of the MO’s manual IDEX interface.
5.2 Transmission
5.2.1 Transmission decision
The table below sets out the recommendation for message types requiring transmission through a standardised IDEX mechanism that the MO offers. The justification for each message type will be described within subsequent sections.
Data service Market operator messages
Standard participant to
participant
Non-standard participant to participant
Transmission Mandatory Mandatory Optional
5.2.2 Market operator messages
By definition, MO messages have the MO involved in the communication – either as the sender or the recipient, of a message. Therefore, the MO must transmit all messages of this type.
5.2.2.1 Market consequence of not implementing
It would not be possible to implement centralised registration, switching and financial settlement processes without providing transmission services of MO messages.
5.2.3 Standard participant to participant messages
Standard participant to participant messages represent standard interactions between market participants. Therefore, it is through this type of message that performance within the market can be monitored.
As the MO will already perform message transmission services in support of MO messages,
implementation costs for further exploitation of these services for this message type will be significantly reduced.
5.2.3.1 Market consequence of not implementing
Failure to implement standardised MO transmission of standard participant to participant messages would result in the creation of participant-specific and non-standardised interfaces. Establishing such interfaces requires all participants within the market to develop integration systems specifically for
integrating with each participant. Spiralling participant development costs and increasingly complex integration within the market would become a significant barrier for new entrants into the market.
5.2.4 Non-standard participant to participant messages
It is not necessary for the MO to provide transmission services for non-standard participant to participant messages to facilitate market communication, standard communication mechanisms, such as email, could be used.
However, there is a real opportunity for the MO to provide ease of communication to multiple participants – for example, where a wholesaler must communicate the details of an operational incident to the
retailers of customers within an affected area.
Although this is not demanded by the market, transmitting non-standard participant to participant messages would provide a consistent communication mechanism that participants could exploit. This would reduce barriers to market entry for new entrants and provide participants a mechanism for communicating with a number of other participants in a single action.
As the MO will already perform message transmission services in support of market operator messages, implementation costs for further exploitation of these services for this message type will be significantly reduced.
5.2.4.1 Consequence of not implementing
Failure to implement transmission of these messages through the market operator would not be detrimental to the market, but opportunities to improve communication within the market would be lost.
5.3 Storage
5.3.1 Storage decision
The table below sets out the decision for message types requiring storage within a standardised IDEX mechanism that the MO offers. The justification for each message type will be described within
subsequent sections.
Data service Market operator messages
Standard participant to
participant
Non-standard participant to participant
Storage Mandatory Mandatory Mandatory where
transmitted
5.3.2 Market operator messages
MO messages effectively provide two functions within the market. Inbound messages (to the MO from market participants) provide the ability to maintain a current and correct representation of the market within the centralised registration system through registration, switching and financial settlement processes. Outbound messages (to market participants from the MO) provide visibility to the market participants of progress through the registration, switching and financial settlement processes.
Therefore, it is a fundamental requirement that messages and their content are stored within the MO.
5.3.2.1 Market consequence of not implementing
Failure for the MO to store such message content would result in the data loaded in preparation for market opening becoming stale and untrustworthy. The processes of registration, switching and financial settlement all provide updates to data held centrally within the MO. Therefore failure to store the content of these messages would result in a centralised registration system that no longer represents the market.
5.3.3 Standard participant to participant messages
To facilitate market monitoring and management of default service levels, all standard participant to participant messages must be stored within the MO. This also gives the added benefit of providing resilience to the whole market. Should any market participant experience system failure that results in data loss, standard participant to participant messages may be re-sent following system recovery.
As message storage will be delivered for MO messages, the implementation costs for further storage of messages will be significantly reduced.
5.3.3.1 Market consequence of not implementing
Failure to store standard participant to participant messages within the MO would leave the MO blind to the performance of participants within the market. This would result in the MO being unable to perform market monitoring and service level agreement (SLA) management of standard industry processes such as operational services to ensure a level playing field.
5.3.4 Non-standard participant to participant messages
It is not necessary for the MO to store non-standard participant to participant messages, but retaining an accurate record of market communication of this type would provide insight into activity within the market.
This also provides the added benefit of providing resilience to the whole market. Should any market participant experience system failure that results in data loss, non-standard participant to participant messages may be re-sent following system recovery.
As message storage will be delivered for MO messages, the implementation costs for further storage of messages will be significantly reduced.
5.3.4.1 Market consequence of not implementing
Failure to store non-standard participant to participant messages within the MO would remove the ability to replay those messages should a participant experience system outage.
5.4 Validation 5.4.1 Validation
The table below sets out the decision for message types requiring validation within a standardised IDEX mechanism that the MO offers. The justification for each message type will be described within
subsequent sections.
Data Service Market operator messages
Standard participant to
participant
Non-standard participant to participant
Validation Mandatory Mandatory N/A
5.4.2 Market operator messages
Because of the critical nature of the MO messages and their support of data held within the centralised registration system, messages presented to the MO must be validated to ensure suitability of market data update into the data representation of the market. Messages that the MO generates expose registration data and therefore the validation can be assumed. Recipients of MO messages may also choose to validate the message content before processing,
5.4.2.1 Market consequence of not implementing
If validation against the centralised registration is not performed before updates take place, then
inaccurate or incomplete data updates could be performed, causing ‘downstream’ process problems for participants. Below, we set out two examples of such problems.
A wholesaler installs a new meter at customer premises, but registers it with the MO without including location details. This causes problems for the retailer when they attempt to read a meter for which they have no location details. It also causes problems for the wholesaler in the future when replacing the meter as part of site works is required.
A retailer uploads an invalid meter reading that is of a lower value than previous readings
(1021…1220…1431…1042), thus causing data-related issues within the financial settlement process between the retailer and wholesaler.
5.4.3 Standard participant to participant messages
Message content is validated against the centralised registration data. Therefore, all standard messages passed through the market will be proven as valid before being forwarded to the recipient, thus reducing the validation development work that market participants require.
5.4.3.1 Market consequence of not implementing
Failure to implement validation of standard participant to participant messages within the MO would allow for processes to be invoked incorrectly. Examples of these would include the following.
A retailer requests a meter exchange at premises where the meter has previously been removed.
A retailer requests any operational services at customer premises for which they do not provide retail services.
5.4.4 Non-standard participant to participant messages
Non-standard messages between participants are freeform in nature. Therefore, it is not possible to implement any validation mechanisms.
5.5 Indicative event and data volumes
To specify the required IT systems it is necessary to understand non-functional requirements, including the volume of data being exchanged. In this section, we set out very rough estimates of the volume of data messages which that would be produced in a given year in steady state market operations. We have provided estimates are provided for the following areas.
Managing registration updates.
Managing service requests.
Managing consumption data.
Managing financial settlement.
For each case the estimate is derived by considering the:
(a) volume of applicable service points or participants;
(b) number of events per year or participant per site (for example, the number of service requests per site); and
(c) number of data messages exchanged to complete an event in each area (for example, the number of messages exchanged to complete a service request). These values are then multiplied to give an
indicative data volume for:
(i) a small water retailer with 2,000 customers;
(ii) a large water retailer with 250,000 customers;
(iii) a large water wholesaler with 250,000 customers; and (iv) the total market/the MO with an assumed 1.5 customers.
These estimates are provided to give an order-of-magnitude to aid design.
Managing registration updates
Managing service requests
Managing registration updates Value
Min Max
Events per supply point per year 0.23 0.65
e.g. Change of Retailer, New supply added, supply removed, demolition, disconnections/reconnections, Newly Eligible premises and Loss of eligibility premises, increase/decrease size of supply,
Messages per event between retailers & MO 2 2 From the Retailer to the MO, MO confirms Messages per event between wholesalers & MO 2 2 From the Wholesaler to the MO, MO confirms
Supply points per customer 3 3
Assumption Potable Water, Surface Water and Foul Sewerage/Trade Effluent
Messages per customer per year 2.76984 7.80552
Annual Totals Annual Totals No. of customes
Small Retailer; 2k customers 5540 15611 2000
Larger Retailer 250k customers 692460 1951380 250000
Large Wholesaler 250k customers 692460 1951380 250000
Market Operator 1.5mill customers 4154760 11708280 1500000
Managing Operational Service requests (site works) Value
Min Max
Events per supply point per year 0.05 0.11
Based on a companies data for total non-hh Operational Service Request contacts in 2012 into total non-hh premises for all supply points. This is divided by the expecetd no. of supply points
Messages per event between retailers & MO 6 6
Request to/from MO, MO confirms received, MO notifies Retailer of update, Retailer confirms received, MO notifies completion, Retailer confirms received
Messages per event between wholesalers & MO 6 6
Request to/from MO, MO confirms received, MO notifies of update, Wholesaler confirms received, MO notifies completion, Wholesaler confirms received
Supply points per customer 3 3
Assumption Potable Water, Surface Water and Foul Sewerage/Trade Effluent
Messages per customer per year 1.68 4.08
Annual Totals Annual Totals No. of customes
Small Retailer; 2k customers 3360 8160 2000
Larger Retailer 250k customers 420000 1020000 250000
Large Wholesaler 250k customers 420000 1020000 250000
Market Operator 1.5mill customers 2520000 6120000 1500000
Managing consumption data
Managing financial settlement
Consultation questions:
SD 5a: Do you agree with the recommendation that the transmission of standard participant to participant messages should be via the IDEX, and mandatory?
SD 5b: Do you agree with the recommendation that the transmission of non-standard participant to participant messages should be via the IDEX, and optional?
SD 5c: Do you agree with the recommendation that the MO should store all standard participant to participant messages?
SD 5d: Do you agree with the recommendation that the MO should store all non-standard participant to participant messages, where the IDEX is used?
SD 5e: Do you agree with the recommendation that the MO should validate all MO messages received e.g. meter readings?
SD 5f: Do you agree with the recommendation that the MO should validate all standard participant to participant messages?
SD 5g: Do you agree with the recommendation that the MO should not validate any non-standard participant to participant messages, where the IDEX is used?
Managing consumption data
Proportion of premises metered 89%
2010/11 figures from OFWAT (WaSCs and WoCs combined water forecast figures)
http://www.ofwat.gov.uk/regulating/reporting/rpt_tar_2010- 11nonhouseholddata
Events per supply point per year (Reads per meter per year) 2 No. of meter readings taken for non-hh for billing purposes
No. of meters per premises 1
Messages per event between Retailers & MO 2
Messages per event between Wholesaler & MO 2
Messages per water service point per year 7.152
Annual Totals No. of customes
Small Retailer; 2k customers 14304 2000
Larger Retailer 250k customers 1788000 250000
Large Wholesaler 250k customers 1788000 250000
Market Operator 1.5mill customers 10728000 1500000
Managing financial settlement
Settlement periods per year 365
Setlement runs per settlement period 4 As per Scotland Standard
No. of retailers 60
10 Incumbent Retailer WaScs , 12 Incumbent Retailer WoCs, 21 Incumbent WSL Retailers, plus expectation of 7 more new retailers coming in to the market, 5 Incumbent NAVs, 5 Incumbent WSL NAVS
No. of wholesalers 26 21 WaSc/WoC and 5 NAVs
Messages per settlement run between MO & Retailer 2 Messages per settlement run between MO & Wholesaler 2
Annual Totals
Small Retailer; 2k customers 12480
Larger Retailer 250k customers 12480
Large Wholesaler 250k customers 12480
Market Operator 1.5mill customers 24960
6. High-level market conceptual data models
Key recommendations made in this chapter
The data model for the market, and particularly for the MO, includes:
for registration and switching, a centrally held record of premises, service points, and associated meters and market participants. The central data model will not hold customer-level data;
for operational services, a centrally held record of service requests and notifications; and
for metering and financial settlement, a centrally held record of meter readings, wholesale charging schemes, and derived wholesale charges.
To enable the market to function successfully, a common data model is required. This chapter sets out early thinking regarding:
a conceptual model for registration data;
a conceptual model for service request data;
a conceptual data model for consumption and financial settlement data; and
indicative event and data volumes.
The conceptual data models show the data entities and examples of data items within those entities. It is not intended that the models shown represent all data items.
The data models shown, especially the consumption and settlement model, are highly dependent on decisions still to be made about how the market shall operate. As such, they should be considered as giving indication of the possible data models, and are highly likely to change.
6.1 Conceptual model for registration data
The conceptual data model for registration data is shown in Figure 3 and is described below.
Figure 3: Registration conceptual data model
At the heart of the registration data model are service points – these are uniquely identified points where a service is provided. Data items related with these may be a unique service point ID and information such as a service point effective date.
Each service point would be associated with a premises, with data items such as the:
unique property reference number;
house number;
street name;
city name; and
post code.
One premises may have multiple service points associated with it, but one service point would only be associated with one premises.
Each service point may be associated with none, one or more present meter, as well as with historical meters. Meter data items may include the:
meter type;
meter serial number; and
meter install date.
A Meter would only be associated with one service point.
Each service point would be associated with responsible parties (both present and past). Responsible parties data items may include a:
retailer ID;
retailer effective date; and
wholesaler ID.
All responsible parties would associate with a market participant, with related data items such as the participant name and participant type.
6.2 Conceptual model for service request data
The conceptual data model for service requests is shown in Figure 4 and is described below. In the diagram, registration data entities that also apply here are shown in blue and are not explained further.
New data entities related to service requests are shown in green.
Figure 4: Service request conceptual data model
Service requests would be created with data items related to where the work is required (such as the premises ID, service point ID and/or meter serial number), what work is required (indicated through a job type ID), who is required to perform the work (such as wholesaler ID), and other relevant data items such as a job urgency flag.
Service requests would be categorised by and therefore associated with service request types.
6.3 Conceptual model for consumption and financial settlement data
The conceptual data model for consumption and settlement data is shown in Figure 5 and is described below. Registration data entities that also apply here are shown in blue (or grey if repeated) and are not explained further. New data entities for consumption and financial settlement are shown in orange.
Note that this data model relates to financial settlement for water and sewerage, and not execution of service requests or other event-based activities.
Indicates data sets that are assumed but would need to be re-evaluated once more detail on the structure of wholesale charging becomes known
Figure 5: Consumption and settlement conceptual data model
Meter readings are associated with meters and may have data items such as meter reading date and meter reading value. One meter would have multiple meter readings, and one meter reading would be associated with only one meter.
A series of settlement adjustment factors may need to be defined that describe how meter readings are adjusted or, in absence, estimated for use in settlement. The relevant data items may be settlement uplift percentage, factor effective date and such like. Settlement adjustment factors would likely be associated with one market participant and service points would be associated with one set of settlement
adjustment factors.
Depending on the agreed settlement approach, settlement readings would be determined from adjusting or estimating meter readings using settlement adjustment factors to produce aggregate consumption values for market participants in a given settlement period. Data items would be those to link to these other entities – for example, the participant ID and trading period, and the settlement reading.
Wholesale charging frameworks would define the financial settlement variables, with data items such as the charge rate for a given consumption level. Service points would likely be associated to a relevant Wholesale charging scheme.
Wholesale charges would be determined through combination of settlement readings and wholesale charging schemes to produce financial charge due values for market participants in a given settlement period. Data items would be those to link to these other entities – for example, the participant ID and trading period, and the financial value.
Consultation questions:
SD 6a: Do you agree with the recommended data model for registration?
SD 6b: Do you agree with the recommended data model for operational services?
SD 6c: Do you agree with the recommended data model for financial settlement?
Appendix A: Consultation questions and approach
Consultation questions
We are seeking views in relation to the following questions posed throughout this document.
SD 4a: Do you think that the proposed systems architecture is appropriate for enabling the services that are being recommended for delivery by the MO and market participants?
SD 4b: Do you agree that the services within the IDEX (automated, semi-manual and manual) sufficiently covers the data transmission requirements of market participants?
SD 5a: Do you agree with the recommendation that the transmission of standard participant to participant messages should be via the IDEX, and mandatory?
SD 5b: Do you agree with the recommendation that the transmission of non-standard participant to participant messages should be via the IDEX, and optional?
SD 5c: Do you agree with the recommendation that the MO should store all standard participant to participant messages?
SD 5d: Do you agree with the recommendation that the MO should store all non-standard participant to participant messages, where the IDEX is used?
SD 5e: Do you agree with the recommendation that the MO should validate all MO messages received e.g. meter readings?
SD 5f: Do you agree with the recommendation that the MO should validate all standard participant to participant messages?
SD 5g: Do you agree with the recommendation that the MO should not validate any non-standard participant to participant messages, where the IDEX is used?
SD 6a: Do you agree with the recommended data model for registration?
SD 6b: Do you agree with the recommended data model for operational services?
SD 6c: Do you agree with the recommended data model for financial settlement? And in particular what complexities do you see in this area of design?
Consultation approach
Please provide your responses to the above consultation questions and any other comments you may have regarding this paper by 14th February 2014, to [email protected]. We provide an accompanying template for responses on the Open Water website to help this process, which we strongly encourage respondents to use. We will, however, accept responses in other formats if necessary.
In addition, we will be running a workshop on 29th January for representatives from water companies to discuss the content presented in all of the high-level design papers. Details of this session have been shared with water companies, and for more information please contact [email protected].
Appendix B: Glossary of terms
For a full glossary of terms used in this document please refer to the Open Water programme Glossary of Terms, available at www.open-water.org.uk
Appendix C: Customer
expectations assessment
Open Water has previously identified that to achieve the objectives of water market reform it is critical to ensure that customers’ expectations of a competitive water retail market are understood and considered in the market design. To achieve this, work has been carried out3 with the objectives of:
“Understanding what customers’ expectations are of a competitive water market;
Identifying whether customer expectations will be met through market design, or left to market competition and innovation; and
Where the need for market design has been identified, agreeing what the Open Water programme will deliver, what will be delivered elsewhere, and what expectations will not be met.”
The customer expectations defined through this work have been considered in the relevant strategy and high-level design documents. As no customer expectations have been mapped to the market operator target operating model, accordingly no assessment has been carried out.
Customer expectation Assessment Commentary
None N/A N/A
3 Document available at: http://www.open-water.org.uk/documents/.
Appendix D: Working group comments
Open Water held a working group on 21 October 2013 to provide water industry experts with the chance to provide early feedback on high-level principles, processes and switching scenarios outlined in this strategy document.
The group’s key comments and questions were captured and we have set them out in the table below.
The accompanying commentary identifies where these are referenced in the strategy document and/or where they will inform future detailed design work.
In particular, the group endorsed the proposed switching process and the Open Water programme’s proposal to take a ‘top-down’ approach to developing a central register of the contestable market. The group agreed that this strategy document had identified all potential switching scenarios.
It was agreed that outstanding debt was an appropriate grounds for the current retailer to object to a customer switch request, but it was noted that there was more work required to establish:
a clear definition of what ‘debt’ is;
whether the current retailer may want to have discretion to write off small debts; and
the exact requirements of what needs to be settled and when in order for switch to be able to progress.
Incumbents in particular noted the need for sight of data specifications for the information they will need to provide to the central register as soon as possible. This has been noted and will inform the Open Water programme’s data work stream.
Working group comments and questions Commentary
The aim of a seamless experience was discussed. Would this place a constraint on design re how Scotland currently works?
It was clarified that Open Water is designing for England with a view to Scottish arrangements. Aiming for consistency but not bound by it.
After the OJEU is issued will we need to revisit how services are cut?
It was confirmed the OJEU is just high-level call for interest.
It was asked how companies had been selected for data pilot?
The pilot wanted to ensure there was coverage of the borders with Wales and Scotland, regions with lots of multi-premises sites, a water only company wholly enclosed, and a new entrant. Scoping document will be published on website. The pilot is also looking to engage with those who have previous experience.
It was pointed out that companies will need to do a lot of work.
Open Water will provide the required outputs but will not do the work.
It was confirmed that the MO is a given and B3 option confirmed. The MO will do all wholesale charging.
It was raised that there will be complexities A standard catalogue of services is needed. This was
around settlement if there is not a consistent charging arrangement.
parked during the workshop but will be identified as part of the data, systems and processes workstream.
It was suggested to change title of settlement if not tracking monies, just making charges?
Do we change the title to ‘Charge calculation’?
It was commented that the biggest challenge in Scotland was metering arrangements.
To be considered in the blueprint and detailed design.
It was asked how do you work out how someone has defaulted?
Voluntary/involuntary exits also raised.
It was agreed there is work to do on triggers and processes in this area.
It was stated that governance decides a lot on how MO looks.
Current approach is to get to level 3 process map by April next year – then talk to vendors to help construct the fine detail.
It was asked what is mandatory data and what is optional data to exchange?
This depends on boundaries on what the market covers. Unstructured data is going to be very hard to manage (non-standard comms level). Materiality should possibly be a factor? It was argued that non- standard comms should not drive design, and would not add any value. Do not overcomplicate things – makes no sense to add costs for once a year events.
It was questioned how useful analytics tools to give early warning signs would be if the MO is not getting all information.
All data/interactions must be performed through the MO to ensure full visibility.
It was mentioned that the experience in Scotland is that the wholesaler will request customer details to fulfil jobs. Is this a non- standard message?
Not currently in MO design but recognised
wholesalers will need it for particular events. There needs to be a requirement for ‘on demand’ customer data requests.
It was asked if Open Water intended to publish a standard list of use cases and data flows for different scenarios?
It was confirmed that this would be part of the level 3 process design in June 2014.
The question of performance monitoring was raised – who does it? MO can do it.
What are the SLAs?
To be determined as part of the market governance paper due out early next year.
How can the system allow wholesalers to offer different levels of service – for example, basic or premium service depending on willingness to pay? It was argued that the MO cannot support some kinds of bilateral agreements but can facilitate them.
To be considered in the blueprint and detailed design.
It was stated that in the gas market the MO failed to provide non-standard messages. If you do not standardise then there are competition issues – if some messages are non-standard there is a risk the MO will
Market messages will be standardised where possible.
miss them.
It was raised that we need to ensure the MO does not become over-complicated.
There is no incentive for incumbents to make it less complex, and the MO to some extent – scope creep. Lots of incentives for some parties to over complicate things.
To be considered in the blueprint and detailed design.
Design by contract is a key concept.
The design needs to accommodate large incumbents – which may be fully automated – to small new entrants. Thrust is to give choice.
It was confirmed the pilot will use semi-automated data.
There was a question about the sequencing of messages.
It was suggested this was more delivery than high- level design.
It was asked if MO involvement in non- standard transmissions contradicted the participant to participant role if this was mandatory?
It was clarified that transmissions would go through the MO by default, with bilateral by exception. Where possible, the use of non-standard messages will be reduced and will be standardised.
It was again mentioned that there are competition issues around non-standard messages.
It was suggested that storage of messages was not necessary to prove delivery. Any user could access an audit database to check delivery.
To be considered in the blueprint and detailed design.
It was commented that electricity is different as no single standard database. The
question was raised what do stakeholders want? Just an audit trail?
In Scotland, a single standard database is very useful – can see what’s going on, very easily. Water more suitable for central storage – smaller.
It was asked what are the requirements that drive this?
It was clarified that this was from the MO scope level 3 paper. There is not a single requirements catalogue – but can see threads in market blueprint. This lays out principles, not a detailed requirements catalogue as such.
Volume of stuff will be a key driver.
Transmission may be a problem if high volume.
Electricity – lots of messages, but not big. Do not need massive storage/capacity: 2.5 million messages a month, 250,000 switch events,150 market
participants. Water – 1/20 of that size. The
assumption is that we will not limited by infrastructure.
Non-standard was raised again – still needs some kind of framework to define what it means. It was commented that it shouldn’t be a free for all. The terms infrequent/ad hoc were suggested as alternatives.
To be considered in the blueprint and detailed design.
It was argued that if you cannot standardise it, do not put it in the MO function.
To be considered in the blueprint and detailed design.
It was asked if there will be a robust change process to add things?
Transparency of communication is behind this – level playing field. It is not a question of nice to have, or adding incidental
services.
To be considered in the blueprint and detailed design.
It was argued that service point should be top of diagram as this is the key element.
But it was also argued that premises dictate the switchable market.
The diagram drills down from premises to service points. Service point drives design – should be top for these purposes? Premises are identified first, then determine service points. Service points are the primary market unit.
It was asked how WOCs billing on behalf of WaSCs for sewerage would work. Will WOCs still do it? Under the new system WOCs would get notification from MO of change of retail liability. There is a need to break down the data flow when designing scenarios. Premises may have different services at individual service point level.
To be considered in the blueprint and detailed design.
Will the MO have to interpret 21 different wholesale charges schemes? Or will the MO just require fixed outputs. Confidence is key? Everyone will double check
calculations anyway. Perception is
important. A common methodology is also important.
In the current design the MO applies price against quantity. It was argued there is a requirement on Ofwat to ensure charging schemes are not complex.
To be considered in the blueprint and detailed design.
It was commented that it is essential for switching process that data is supplied.
To be considered in the blueprint and detailed design.
For an operational services request, the wholesaler makes appointment with the customer, not the premises. This kind of action fits in retailer/wholesaler space not MO space. This is about logging a request between participants – not tracking every aspect of the job.
The MO does not want to store customer data. It will not hold billing or contact addresses. The customer name is part of the data set in Scotland, but it is not proposed to do this.
The issue of void properties was raised. The data pilot is looking to tease out these kinds of issues.
Data integrity will form part of the market readiness testing.
The data pilot is about specification, processes, etc, not data quality. The output of the pilot could define data sets. Open Water intends to share requirements with companies as soon as we have them.
All participants go through MO unless exception agreed with Ofwat. Internal
All communication will pass through the MO to provide market visibility.
messages need to be exposed.
Look at standardising outputs – not standardising individual systems.
Perceptions of Level Playing Field are very important.
If you do not have intra-company visibility, any performance data/analytics is useless.
All participant interactions will be mandated through the MO.
There was general agreement with the broad principles. Keep complexity low, standardise as much as possible.
Recognise the process is a journey – need to be aware of need to fix problems that come along later.
It was argued that the market will react to sort out the interfaces with MO. It is important to consider what an innovative new entrant will need.
Non-standard should be avoided. All inter-company data exchange mechanisms will be transmitted through the MO. Non-standard
interactions will be minimised through standardisation where possible.
Retailers will have to mimic wholesalers’
charging schemes – so keep it simple.
Wholesale charging mechanism simplicity will be dictated by the whole market participants. The MO will support the demands of the market
Engagement is key. Get water company IT change programs kicked off early sight of data sets welcome.
Working groups and data pilot will inform industry knowledge and should stimulate the initiation of IT programs of work
Where does the customer data sit? The retailer maintains customer data but discussions must be had to establish how this can be made available to wholesalers. It may be possible to offer this data on demand from the retailer to support operational services; alternatively, the MO could offer a view of this data.
There is an implicit assumption that customers know which services they receive – not true! This sometimes only exposed when a new retailer comes in.
