HiPerDNO
High Performance Computing Technologies for Smart Distribution Network Operation FP7 - 248135
Project coordinator: Dr Gareth Taylor (BU)
Consortium Members: Brunel University (BU), Electricite de France (EF), IBM Israel, University of Oxford, UK Power Networks plc (UKPN), Union Fenosa Distribution (Union Fenosa), Indra Sistemas, GTD Systems de Information (GTD), Korona Inzeniting (Korona), Elektro Gorenjska Podjetje za Distribucijo Elektricne Energije (EG) and Fraunhofer IWES
Document Title Prototype deployment of selected architecture and necessary documentation, D1.2.2
Document Identifier HiPerDNO/2011/D.1.2.2
Version Version 1.0
Work package number WP1
Sub-Work package Number WP1.2
Distribution Public
Reporting consortium member Oxford
Prototype deployment of selected architecture and necessary
documentation, D1.2.2
Executive Summary
This document presents the HiPerDNO HPC Platform prototype developed in WP1.2. The prototype covers all major components of the HiPerDNO HPC platform, and it was presented during the HiPerDNO Quarterly Progress Meeting in Clamart on 29th – 30th June 2011. The prototype can be downloaded from the FTP server hosted by Brunel University. The prototype can be installed and run following the instruction and tutorial in this document.
This document also lists the requirements for the HPC scheduler. These requirements are met by the HPC prototype, and represent our current understanding, formed through interaction with the HiPerDNO partners, of the needs of DNO applications. Obviously, as collaborations with partners progress so will the specification and deployment of their needs within the HiPerDNO HPC Platform.
Document Information
HiPerDNOProject Number FP7 - 248135 Acronym HiPerDNO
Full Title
Prototype deployment of selected architecture and necessary documentation, D1.2.2
Project URL http://www.hiperdno.eu
Document URL N/A
Deliverable Number
D.1.2.2 Title
Prototype deployment of selected architecture and necessary documentation (M18)
N/A Work Package
Number WP1.2 Title
Investigation and evaluation of new HPC platforms and architectures.
Date of Delivery Work Plan Actual
Status Version 1.0
Nature Prototype Internal Report External Report Dissemination Dissemination Level Public Consortium
Author(s) (Partners) Stef Salvini, David Wallom
Lead Author Name Piotr Lopatka E-mail [email protected]
Partner Oxford Phone 44 1865 610607
Abstract
This document and the prototype of the HPC System constitute deliverable D1.2.2. The prototype covers all major components of the HiPerDNO HPC platform, and it was presented during the HiPerDNO Quarterly Progress Meeting in Clamart on 29th – 30th June 2011. The prototype can be downloaded from the FTP server hosted by Brunel University. Access details can be found on Basecamp. This document describes the prototype, its installation and use.
Table of Contents
1. Introduction ... 5
2. Prototype Description ... 6
Purpose of the Prototype ... 6
Functional Description of HPC Prototype ... 6
HPC Interface to the Data System ... 7
Applications scheduling ... 7
3. Prototype Deployment – Installation Tutorial ... 9
Prerequisites ... 9
Creation of VM ... 9
Virtual network setup. ... 10
Head-Node: hpc1 ... 11
Compute-Nodes: hpc1, hpc2, hpc3, hpc4 ... 12
Fast Storage node: storage ... 12
Data System nodes: hdfs1, hdfs2 ... 13
4. Testing the Prototype ... 14
DNO-like model applications ... 14
Running the example ... 14
Other key commands (HDFS, Torque, MAUI) ... 16
Editing schedules ... 17
5. From Prototype to System Deployment ... 18
1. Introduction
This document describes the prototype of the HiPerDNO HPC Platform that was described and demonstrated at the HiPerDNO Quarterly Progress Meeting in Clamart on 29th -30th June 2011. The prototype implemented the design concepts introduced in the Deliverable 1.2.1.
The chapters of this document discuss the prototype implementation, deployment and the tests that were conducted. The HPC prototype software package has been made available to all partners through the FTP server hosted by Brunel University. Installation instructions are included in this deliverable.
2. Prototype Description
Essential reading : HiPerDNO deliverables 1.2.1 [1], 3.1.1 (particularly chapters 2 and 3) [7].
The prototype consists of the HPC Engine (HPCE) and the Data Subsystem (DSS). The HPCE schedules and executes the DNO applications making optimal usage of resources while ensuring appropriate balance and priority between mission critical and non-critical applications. The Data System is a resilient and secure repository for the data necessary to execute DNO applications. A fast storage system is an essential component of the HPCE: it is a local storage system for compute nodes, where applications as well as their current operational data (scratch data) are stored.
Purpose of the Prototype
The purpose of this prototype is two-fold. In the first place, it allowed both feasibility and implementation studies of the system architecture presented in [1], as it covers all major components of the HiPerDNO HPC Platform as defined in that document. In the second place, the prototype allows the testing of the DNO application scenarios to run on the full HPC system. Test applications of appropriate scheduling requirements and characteristics were employed during the demonstration (see chapter 4). It also allows the integration and test deployment of real DNO applications, which is the subject of on-going work.
Functional Description of HPC Prototype
The HPC prototype consists of the HPCE and the DSS and it is depicted in Figure 1. One of the nodes in the HPCE is the head-node. In the prototype it plays also the role of the DMS, responsible for requesting execution of jobs in the HPC and it includes the scheduler and its required mechanisms. The HPC nodes require the presence of a common storage space (Fast Storage in figure 1).
HPC Interface to the Data System
All data are stored in the DSS. The DSS is not visible to the applications or indeed to either the DMS or the HPCE: data can only be accessed through appropriate client-server mechanisms. Both retrieval and storage of data are carried out through the client-server interface.
In the prototype, data retrieval and storage results for the model applications are greatly simplified. We expect that in the HiperDNO HPC Platform appropriate data design, including metadata, will be carried out, but this is application-dependent and details will emerge in due course through the collaborations already started.
The prototype relies on meta-data protocol that allows identification of the types of the application and of data access. Exchange of meta-data precedes exchange of content.
The client-approach also allows changing the DSS technology (currently HDFS), without affecting the rest of the system and can be security hardened.
Applications scheduling
The scheduler ensures that applications are executed according to the execution priorities set by the DNOs. Based on collaboration with partners, we (WP1.2) have laid out the preliminary scheduler requirements for HPC applications, and are further discussed in chapter 4 below, and in [7]. We would like to stress that these requirements are not final: they corresponds to our current understanding, and no doubt will be modified following the DNOs feedback and the emergence of fully defined requirements as the project progresses.
High-level requirements that must be supported by the scheduler are pre-emption and partitioning. Pre-emption allows the HPC system to stop a running task in order to make resources available for a high-priority task (kill-and-requeue). Low priority jobs are killed and re-added to the appropriate scheduler queue, without altering their job id, hence their place in the appropriate queue. Pre-empted tasks can then be restarted as soon as HPC Engine resources become available.
Partitioning allows limiting the amount of resources that can be dedicated to jobs of a specific priority. The scheduler requirements are listed in greater detail in table 1. We have employed the Maui scheduler in the prototype, sitting on the Torque resource manager. Maui is a freely available, open-source package, and it supports all the requirements in table 1, with the exception of the optional no 6. Maui does not allow medium priority jobs to be both pre-emptable by higher priority jobs and at the same time pre-emptying lower priority ones: roles of pre-emptee and pre-emptor, so to speak, are incompatible, although this might be further studied. In any case, we do not believe that this has any major impact at this stage of the HiPerDNO project. The suggested partitioning of computing resources is adequate for current needs. Notice that we had to create a patch to ensure that Maui satisfies requirements no. 5 in table 1. This patch is of course part of this deliverable and is made available together will other sources, scripts and model applications.
No.
Type
Requirement
1.
Resources
Must
The schedulersupports execution of jobs requiring 1...n
compute nodes.
2.
Pre-emption
Must
The scheduler supports pre-emption.
3.
Pre-emption
Must
Jobs can be classified as critical and non-critical (critical jobs
always pre-empt non-critical jobs).
4.
Pre-emption
Must
Requeueing mechanism is supported, i.e. killing jobs and
putting them at the top of their queue for execution.
5.
Pre-emption
Must
Critical job can pre-empt as many low-priority jobs as
necessary (but no more) to get the required resources for
execution.
6.
Pre-emption
Nice
to
have
Another class of jobs can be defined with intermediate
priority level “between critical and non-critical”. These jobs
can empt non-critical jobs, however they can be
pre-empted by critical jobs at the same time (pre-emption
cascade).
7.
Partitioning
Must
If no. 6 is not possible, the scheduler must allow the
partitioning of resources. The amount of compute resources
allocated to a certain class of critical jobs can be limited.
8.
Scheduling
latency
Must
To satisfy mission-critical requirements, high priority jobs
must be run when they are submitted.
Pre-emption and scheduling frequency must be sufficiently
fine-grained to allow the stopping of low priority jobs and
starting of high-priority jobs quickly and effectively. (Order
of 1 second?)
3. Prototype Deployment – Installation Tutorial
The following sections discuss the installation of the HPC prototype. The prototype is to be deployed in a virtualized environment. We would like to stress that the target deployment infrastructure for the HPC system will be the real hardware. This is however the part of on-going work and not a part of this tutorial (see also Chapter 5).
A number of prerequisites must be met before starting the HPC prototype installation. The deployment consists of two stages. The first stage involves the creation of a virtual cluster (nodes and network), the installation of the operating system and the network configuration. The second stage involves the installation and configuration of software which is central to the prototype: resource manager, scheduler, fast storage and data system software.
Prerequisites
The following list presents all necessary components for the deployment of the prototype. We suggest using VirtualBox as virtualization software and Ubuntu Server 10.04 as an operating system for the nodes.
Host
o Hardware: Desktop PC (we used: Intel Core 2 quad, 8 GB RAM) o OS: Linux (we used: Ubuntu 10.04)
Virtual Machines (VM)
o Virtualisation software: Oracle VirtualBox (www.virtualbox.org) o Guest OS: Ubuntu Server 10.04 (www.ubuntu.com) (or Ubuntu 10.04) Prototype software (Archive: D1.2.2.tar.gz from Basecamp), including
o Cluster Resources MAUI v.3.3.1 (patched version) o Cluster Resources Torque v. 3.0.0
o MAUI configuration file: maui.cfg o Torque configuration file: torque.cfg o demo scripts and applications o a video of running prototype
o HDFS (Hadoop Yahoo distribution 20.2, hadoop.apache.org or www.cloudera.com)
Creation of VM
Seven virtual machines should be created. The names listed below are fixed, as they are part of the prototype configuration: hpc1, hpc2, hpc3, hpc4, storage, hdfs1, hdfs2.
The OS on the VMs do not require a GUI, and therefore have modest requirements: one CPU core and 384MB of RAM per virtual machine are sufficient. VirtualBox virtualization software provides graphical interface which makes it easy to create VM’s quickly. During the creation most parameters can be left unchanged.
During the VM creation the network interfaces must be set correctly. We chose to use two network interfaces per virtual machine. The first one gives access to the Internet (security updates,
software repositories). The second interface connects the VMs together using a virtual network and makes them accessible from the host. In terms of VirtualBox configuration, two network interfaces should be enabled during the VM generation: the first one will be a NAT (Network Address Translation)for Internet access, the second is a Host-Only-Adapter for the virtual network connection with other VMs and the host.
The OS must then be installed. The installation of the Ubuntu Server 10.04 is very straightforward, and it takes only few minutes. During installation, all the default options can be accepted. When asked, create the default username “proto”, and mark the OpenSSH package for installation. It is possible to speed up the whole process by cloning the virtual hard drive of the first VM and plugging it into the other six. It is important to remember to change their hostnames. For more information on cloning, please refer to the VirtualBox documentation [5, chapter 5].
Virtual network setup.
This step assumes that all the VMs have been implemented, the OS has been installed in each of them and that the user “proto” is enabled and has rights to execute administrator’s tasks (sudo). The network configuration requires editing two configuration files. The IP addresses are static for the virtual cluster network, and dynamic for the access to the Internet. The static addresses were chosen arbitrarily and they can be changed if necessary. The following section can be appended to each VM /etc/hosts file.
// Prototype hostnames mapping 192.168.56.10 hpc1 192.168.56.11 hpc2 192.168.56.12 hpc3 192.168.56.13 hpc4 192.168.56.80 hdfs1 192.168.56.81 hdfs2 192.168.56.100 storage
The VirtualBox settings of each VM define the first network interface as NAT and the second as Host-Only-Adapter and the Linux kernel in the guest OS should assign to them eth0 and eth1 descriptors, respectively. The following section can be added to the /etc/network/interfaces file of every VM operating system if required. It may be necessary to substitute the IP address field for each VM to avoid any address conflicts.
// The primary (NAT) network interface auto eth0
iface eth0 inet dhcp
// The secondary (virtual network) interface auto eth1
After VMs are rebooted, the network should be up and running. In order to verify both interfaces, try to ping a machine on the internet and the host virtual network interface.
It is suggested to enable SSH access on all nodes for the user “proto”, so that the nodes can be accessed from the host. By default the SSH service is enabled if the OpenSSH software was marked for installation during OS installation and we suggest to enable password-less SSH access from the host to all nodes. Instructions on how to do it can be found in the last section of this chapter, which discusses the installation of Data System software.
This step completes the first stage of the installation. The Virtual cluster can be started and it is possible to connect to any VM machine from the host machine using SSH. In the next step, we are going to install the required software packages on the virtual nodes.
Head-Node: hpc1
The hpc1 node is the head-node. It requires the following software packages to be installed: MAUI scheduler
Torque server module
Please use the MAUI installation package provided with this deliverable and not the one available on the website above: we have applied a required patch to the package we provided.
MAUI
Copy, unpack and install the file: maui-{version}.tar.gz, using the following commands: >tar –xzvf maui-*.tar.gz
>cd maui-* >./configure >make
>make install
Copy the file maui.cfg from the installation directory to the following location on the hpc1 node: maui.cfg -> (hpc1) /var/spool/maui/maui.cfg
The MAUI installation and troubleshooting documentation can be found in [2]. Torque
Copy, unpack and install the file: torque-{version}.tar.gz, using the following commands: >tar –xzvf torque-*.tar.gz
>cd torque-* >./configure >make
>sudo make install
The installation of Torque server module creates /var/spool/torque directory where configuration files are placed. From the installation folder torque files copy the following file:
nodes -> (hpc1) /var/spool/torque/server_priv/nodes Set up Torque submission queues by copying and installing torque configuration file: torque.cfg -> (hpc1) ~/torque.cfg
>sudo qmgr < torque.cfg
Torque installation and troubleshooting documentation can be found in [3].
Compute-Nodes: hpc1, hpc2, hpc3, hpc4
Torque
The nodes hpc1, …, hpc4 are compute nodes. Notice that hpc1 node is also the head-node. The compute nodes require the following software installed:
Torque compute node module NFS client
The Torque module will allow compute nodes to communicate with the head-node.
To install the Torque compute node module login in the hpc1 node, enter the Torque installation director and create compute module installation packages:
>cd torque-* >make packages
It will generate a list of Torque packages. Copy and install the following packages on all four compute nodes:
>sudo torque-package-mom-linux-i686.sh --install >sudo torque-package-clients-linux-i686.sh --install To enable Torque to run as a service, execute the following commands: >sudo cp contrib/init.d/debian.pbs_mom /etc/init.d/pbs_mom >update-rc.d pbs_mom defaults
From the installation folder torque files copy the following file:
server_name -> (hpc1,…, hpc4) /var/spool/torque/server_name config -> (hpc1,…, hpc4) /var/spool/torque/mom_priv/config
NFS Client
Compute nodes require the following software: NFS Client
To install NFS Client on Ubuntu virtual machine, follow the following tutorial [6].
https://help.ubuntu.com/community/SettingUpNFSHowTo
On each compute node: hpc1, …, hpc4 create ~/shared/demo folder for the user proto. This folder should be mapped to the exported folder on the Fast Storage System (see next section).
Fast Storage node: storage
Create shared folder:
>mkdir -p ~/storage/demo
Export the folder as a shared folder, by adding the following entry to the /etc/fstab file: >/home/proto/storage/ /export/storage none bind 0 0
Verify whether exported folder can be accessed from all compute nodes.
Data System nodes: hdfs1, hdfs2
The Data System requires installation of: HDFS (Hadoop Distributed File System)
The HDFS package can be obtained from the Cloudera website under the following link:
https://ccp.cloudera.com/display/CDHDOC/CDH3+Installation
The HDFS has few prerequisites. It requires the JDK6 and SSH packages. The details of installing JDK6 are not addressed here as they are described in the link above. SSH setup requires password-less type of access between two machines. Setting up password-less access requires two steps: generating the key for the user, and copying this key to a remote machine for subsequent password-less access.
To generate an encryption key, execute on both machines hdfs1 and hdfs2 the following command: >ssh-keygen –t rsa
When asked, confirm default parameters. To enable password-less SSH access between hdfs1 and hdfs2 machines, run the following commands on the nodes:
On hdfs1 node:
>ssh-copy-id –I ~/.ssh/id_rsa.pub proto@hdfs2 On hdfs2 node:
>ssh-copy-id –I ~/.ssh/id_rsa.pub proto@hdfs1
Before the HDFS cluster can be used, it should be formatted. A list of useful HDFS commands, including the command to format the HDFS system is presented in the next chapter.
4. Testing the Prototype
DNO-like model applications
The prototype contains three “model” DNO applications as they are studied in the HiPerDNO project: distributed state estimation (DSE), condition-monitoring (CM), data-mining (DM). These applications are different with respect to criticality, execution pattern (periodic, aperiodic) and required compute resources. The proposed characteristics are summarized in table 2 were chosen as an example scenario. At the same time they are based on the input provided by the partners in the project.
DNO Application Scheduling characteristics
DSE mission-critical, periodic, short submission interval, parallel
CM High-priority, periodic, long submission interval, batch of serial tasks DM non-critical w.r.t timely results, aperiodic, long-execution time
Table 2. Model DNO application, execution requirements.
The DSE is a periodic job with a short submission interval (few minutes; for the purpose of this prototype shortened significantly). It is a parallel task requiring a number of compute nodes and it is a mission-critical task, thus requiring immediate execution at the moment of submission.
The CM is a periodic task with long submission intervals. The CM submission consists of a batch of small serial jobs submitted together at regular intervals. CM jobs are high priority and are executed on a limited number of compute nodes
The DM jobs are not periodic; they are serial tasks, submitted at arbitrary intervals, requiring long execution times. They are not critical jobs from the DNO point of view and can be executed when the system load due to higher priority jobs allows. Thus their execution can be postponed. The model DM applications includes checkpointing/warm restarting, thus allowing DM jobs to pre-empted and rescheduled (kill-and-requeue).
Running the example
In the previous chapter HPC hardware infrastructure and software components (Torque, MAUI, NFS, HDFS) were set up. Before the prototype can be run the DNO model applications and submission scripts must be deployed from the host, as well as basic data layout in the DSS.
Start the prototype by executing the following script on the host: >./startvms
When all virtual machines are running, deploy all scripts and DNO model applications from the host by running:
Open the terminal window and log into the hdfs1 node. Create data folders for DNO applications: >hadoop dfs –mkdir DSEIn
>hadoop dfs –mkdir DSEOut >hadoop dfs –mkdir CMIn >hadoop dfs –mkdir CMOut >hadoop dfs –mkdir DMIn >hadoop dfs –mkdir DMOut
Next, generate example input files for DNO applications by running: >./set.py DSE 10
>./set.py CM 2 >./set.py DM 500
These commands will create the input files needed by applications running in the prototype. The second parameter behind the name of the application (DSE, CM, DM) specifies the length of its input file and effectively the execution time of the application.
Run the Data Store server script: >./ds.py
The script is a part of the client-server interface, which sits on top of the Data System. Open the terminal window and log into the node hpc1. Run the following script: >./monitor.sh
Open the second terminal window and log into the node storage. Run the following script: >./filemon.sh
The two terminal windows will show scheduling and checkpoint information accordingly.
Open another terminal window and login into hpc1 node. Run the prototype demonstration with the following command:
>./demo.sh
The demo takes a couple of minutes to execute. In a terminal window with scheduling information, one can see which jobs are submitted but waiting for execution, which are executing and on which nodes. The second terminal window shows how checkpoint files are created. Both terminal windows are shown in figure 2, which presents a snapshot from the running prototype. We can see that five jobs are present in the HPC system: four DM jobs and one DSE job. Three DM jobs are not running because of the DSE job pre-emption. Only one DM job is in execution. In the right terminal window we can see that new checkpoint files are being generated for the DM job running.
Figure 2. HPC Prototype demo.
Other key commands (HDFS, Torque, MAUI)
In order to manage the prototype the following commands may prove to be useful. More complete list of commands to manage HDFS, Torque, MAUI can be found through the references [2-4].
HDFS
The basic operations on HDFs include formatting the HDFS file system, copying, listing, and removing the files. To format the HDFS , using the following command:
>sudo hadoop namenode –format
To display the list of files in the HDFS directory issue the following command: >hadoop dfs –ls URI
To remove a file from the HDFS, issue the following command: >hadoop dfs –rm URI
To display the content of the file onto a terminal, issue the following command: >hadoop dfs –cat URI
To copy a file from the local file system to HDFS, use the following command: >haddop dfs –copyFromLocal {localSrc} URI
To copy a file from HDFS to a local file system, use the following command: >hadoop dfs –copyToLocal URI <localdst>
For the complete list of HDFS commands please refer to [4]. Torque
By default, Torque starts its standard PBS scheduler when the head node boots up. The PBS scheduler is not part of the prototype and it should be stopped manually, before enabling MAUI (see the next section). To stop the PBS scheduler, type the following command on the hpc1 front node:
are running, use the following command on the hpc1 front node: >pbsnodes
In the output of this command check if for each node the state is set to free. It if is, it means the node is up and running and waiting for jobs to be submitted.
In order to check the list of submitted jobs use the following command on the head node HPC1: >qstat
The command will display the list of jobs submitted to the resource manager, together with their state. MAUI
The key configuration tool for the MAUI scheduler is its configuration file. It is not the purpose of this document to go in depth on the configuration parameters of the MAUI schedulers. For those interested, refer to [2]. The configuration file maui.cfg is provided together with this deliverable and it should be copied to appropriate location on the HPC1 head node before starting MAUI scheduler. The location for MAUI configuration file is: /var/spool/maui/maui.cfg.
Make sure that the default PBS scheduler is not running (see previous section). To start MAUI use the following command on the HPC1 head node:
>maui
Editing schedules
The prototype is shipped with an example scenario “demo.sh”. It is possible to submit CM, DM, DSE jobs without demo.sh script. In order to submit CM or DM type:
>./submit.sh DM >./submit.sh CM
To submit DSE, one can specify additionally how many nodes DSE job would require (max 3). >./submit.sh DSE 2
>./submit.sh DSE 3
To influence the runtime of the jobs modify the size of the input file stored in the Data Store (see section Data System setup in the previous chapter). For example to shorten by half the execution time of the DSE execute the following command on the hdfs1:
5. From Prototype to System Deployment
The HPC Prototype described in this document lays the ground for the work ahead in the WP1.2 and other work packages in the HiPerDNO project.
We are planning to deploy the current HPC prototype on real hardware. This deployment will be made available to partners in the HiPerDNO project. Based on initial discussions within the OeRC team we will make available an Intel-based cluster consisting of seven or eight dual-CPU nodes. It will be configured so as to allow both the development and tuning of real applications as well as study of the overall HPC System under realistic deployment conditions.
We plan to use Rocks (http://www.rocksclusters.org), to ease the deployment and maintenance of the system, and interconnects adequate to the task in hand (hopefully, 10 Gbit Ethernet). Work on the deployment will commence in August and we plan to make it available to all partners in September. We are also starting collaborations with various HiPerDNO partners (EDF, BU, Indra/UF) with the aim of porting to and integrating their algorithms on the HPC platform. More information can be obtained from the forthcoming HiPerDNO deliverable 3.1.1 ([7]).
6. References
1. S. Salvini, P. Lopatka, D. Wallom, HiPerDNO Deliverable D.1.2.1 “Report on the
architecture and performance criteria used for selection.”
2. Adaptive Computing Enterprises, “Maui Scheduler Administrator’s Guide version 3.2”,
http://www.adaptivecomputing.com/resources/docs/maui/mauiadmin.php
3. Cluster Resources, “TORQUE Admin Manual version 3.0”,
http://www.clusterresources.com/torquedocs21/
4. The Apache Software Foundation, “HDFS User Guide.”,
http://hadoop.apache.org/common/docs/r0.18.3/hdfs_user_guide.pdf
5. Oracle Corporation, “Oracle VM VirtualBox”,
http://www.virtualbox.org/manual/UserManual.html
6. Ubuntu documentation, “Setting up NFS – HowTo”,
https://help.ubuntu.com/community/SettingUpNFSHowTo
7.
HiPerDNO project deliverable, D3.1.1, “Detailed specification report on HPC architecture and platform standardisation to support the development of novel DMS functionality”