NeCTAR NCRIS 2013 Final Project Plan
1. DEVELOPMENT OF PROJECT PLAN
1.1 [Please outline how the project plan was developed, including consultation undertaken, and with what parties, to develop Draft Project Plan, issues raised during consultation, how issues were resolved, and any other relevant matter.]
This NeCTAR NCRIS Project Plan has been developed through consultation with key NeCTAR stakeholders, including the NeCTAR Project Board, the NeCTAR Platforms Steering Committee
(including the Nodes of the Research Cloud), key NeCTAR Virtual Laboratory projects and a number of the NCRIS/SS capabilities/projects.
The Research Cloud and NSP Operations Subproject has been drafted in close consultation with a working group of the NeCTAR Platforms Steering Committee, which includes representation of the organisations hosting Nodes of the Research Cloud infrastructure. Significant input has also been provided by the current operator of the NSP node at the University of Melbourne, the NeCTAR Platforms Steering Committee and the Integrated Marine Observing System (IMOS) NCRIS capability as a significant user of the NSP service.
The Virtual Laboratory Operations Support Subproject has been drafted following consultation with NeCTAR Virtual Laboratory projects in early operational phase, including: the Virtual Geophysics Laboratory (VGL), the Genomics Virtual Laboratory (GVL), the Marine Virtual Laboratory (MarVL), and the Characterisation Virtual Laboratory (CVL). NeCTAR has also consulted with NCRIS/SS capabilities, including representatives of the International Marine Observing Systems (IMOS), BioPlatforms Australia, AuScope, the National Computing Infrastructure (NCI), Australian National Data Service (ANDS) and the Research Data Storage Infrastructure (RDSI).
2. PROJECT OVERVIEW
2.1 Scope, Objectives and Benefits
The primary objective of the NeCTAR NCRIS Project is to continue to enhance research collaboration and research outcomes by
● Improved and continued operation of eResearch infrastructure that ○ creates new information-‐centric research capabilities;
○ significantly simplifies the combining of instruments, data, computing, and analysis applications; and
○ enables the development of research workflows based on access to multiple resources.
has access to a full suite of digitally enabled data, analytic and modelling resources, specifically relevant to their research.
The scope of the NeCTAR NCRIS Subproject includes activities under two Subproject areas: ● Research Cloud and NSP Operations; and
● Virtual Laboratory Operations Support.
Research Cloud and NSP Operations Subproject:
Benefits:
● Research Cloud:
○ Continue to operate the NeCTAR Research Cloud as a federated national infrastructure to mid 2015;
○ Meet research community expectations for robust provision of eResearch services which are fit for purpose; and
○ Reduce overall operational costs across the federation of research cloud nodes. ● National Servers Program:
○ NCRIS Renewal funding for the NSP program would enable the program to continue and to fully fund the operational costs of the service. As an operationally funded service, the program would continue to deliver efficiencies for other research infrastructure providers, including the NCRIS and Super Science capabilities. Under this NeCTAR NCRIS Project the operations of the Research Cloud and NSP programs will be merged under the same sub-‐project. The Research Cloud and NSP will be two complementary computational platforms which will be managed under, and offered to research users through, the NeCTAR NCRIS Research Cloud and NSP program. This will deliver improved efficiencies in operating the two platforms and remove barriers to improving alignment between the service offerings of both platforms. Managing the NSP platform within the larger Research Cloud program will also improve opportunities to address risks to future sustainability of the NSP platform.
Improved service levels for the Research Cloud and NSP platforms will be developed and
incorporated into the Service Level Agreements associated with the Research Cloud and the NSP.
The Software Infrastructure Virtual Laboratory and eResearch Tools programs:
● Support operations of key Virtual Laboratory infrastructure, especially where these are aligned with NCRIS Capability investments. It is anticipated that there would not be a general allocation of NCRIS renewal funding to new Virtual Laboratory and eResearch Tools
sub-‐projects.
2.2 Participating Organisations
[Please list the participating organisations, and their key roles and responsibilities, as below] ● The University of Melbourne
○ NeCTAR Lead Agent
○ Research Cloud Operations Subproject Lead
● ANU/NCI, QCIF, Intersect, eRSA, iVEC, University of Tasmania, Monash University, The University of Melbourne
○ Members of the NeCTAR Platforms Steering Committee ○ Participants in Research Cloud Operations Subproject ○ Hosts of nodes of the NeCTAR Research Cloud
● eResearch Service provider members of the Australian eResearch Organisation (AeRO) ○ Candidate operators of Virtual Laboratory Operations support services
● Operators of the NeCTAR Virtual Laboratory and eResearch Tools projects ○ including State-‐based eResearch Organisations, NCI and Universities
2.3 Governance
[Please outline the management and governance arrangements, and the names of occupants of key positions]
The NeCTAR NCRIS Project will operate under Governance and Management arrangements based on the existing arrangements for the NeCTAR Super Science Project which are described in the NeCTAR Final Project Plan. Those governance arrangements have been adapted to reflect this NCRIS project’s focus on the stable operation and delivery of high-‐value, fit for purpose services based on the infrastructure created under the NeCTAR Super Science Project.
Under this NCRIS Project Plan we:
● Strengthen the reporting and communication channels between the NeCTAR Project Board and the Platforms Steering Committee; and
● Introduce the NeCTAR Platforms Research Reference Group to represent the interests of the research users of the NeCTAR infrastructure.
Lead Agent
The University of Melbourne is Lead Agent for the delivery of the NeCTAR NCRIS Project and has overall responsibility for the management and implementation of the Project in accordance with the reporting and accountability requirements outlined in the NeCTAR NCRIS Funding Agreement executed between the University of Melbourne and the Commonwealth on 2 August 2013. The University of Melbourne’s Deputy Vice Chancellor (Research), Professor Jim McClusky is the Project Delegate under the funding agreement with Commonwealth and the NeCTAR Director is Associate Professor Glenn Moloney at the University of Melbourne.
The NeCTAR Project Board is advisory to the Lead Agent and the University of Melbourne will receive that advice through internal oversight processes at the University and through the NeCTAR
Directorate.
NeCTAR Project Board
The existing NeCTAR Project Board is the independent body formed under the NeCTAR Super Science Project to provide strategic guidance to The University of Melbourne and the NeCTAR Director. The NeCTAR Project Board will continue to perform this role under the NeCTAR NCRIS
Project.
The NeCTAR Project Board will extend an invitation to the members of the PSC to nominate a
member to attend Project Board meetings along with the NeCTAR Director to facilitate reporting and communications between the PSC and the Project Board. This invitation may be temporarily or permanently revoked at the discretion of the Project Board.
The NeCTAR Project Board will receive reports and advice from the Platforms Steering Committee and will have access to advice provided to the Platforms Steering Committee by the Research Reference Group. The Project Board may also request advice from the Research Reference Group as needed. The Project Board is not bound to respond to or follow the advice provided by either committee.
Advice provided to the NeCTAR Project Board by the Platforms Steering Committee and the Research Reference Group will also be made available to the Department representatives. The NeCTAR Project Board comprises the following members as of 12 December 2013:
Board Member Organisation
Doctor Graham Mitchell AO (Chair) Independent chair
Professor Andrew Cheetham (Deputy Chair) University of Western Sydney
Professor G. Q. Max Lu University of Queensland
Doctor Paul Arthur University of Western Sydney
Professor Robyn A Owens University of Western Australia
Doctor Roger Proctor Integrated Marine Observing System
Mister Paul Sherlock University of South Australia
Professor Liz Sonenberg University of Melbourne
Professor John A. Taylor CSIRO
NeCTAR Platforms Steering Committee (PSC)
As identified in the Super Science NeCTAR Final Project Plan, the NeCTAR Platforms Steering Committee (PSC) provides oversight and strategic guidance to the participants in the NeCTAR Research Cloud and National Servers Programs (NSP) to achieve the NeCTAR Project objectives. The PSC will provide oversight and guidance in the execution of the Research Cloud and NSP Operations Subproject under this NeCTAR NCRIS Project.
The Platforms Steering Committee membership includes:
● nominees of the operators of the Research Cloud and NSP infrastructure: the participating nodes of the Research Cloud and NSP programs;
● the Chair of the Platforms Research Reference Group (see below); ● and is Chaired by the NeCTAR Director.
The PSC comprises the following members as of 12 December 2013:
PSC Member Organisation
Glenn Moloney (Chair) NeCTAR Director
Nathan Bindoff University of Tasmania
Lindsay Botten Australian National University
Rob Cook Queensland Cyber Infrastructure Foundation (QCIF)
Ian Gibson Intersect
Mary Hobson eResearch SA
Steven Manos University of Melbourne
Steve Quenette Monash University
Neil Stringfellow iVEC
TBA Chair, Platforms Research Reference Group
The PSC will request and receive advice from the NeCTAR Platforms Research Reference Group as representatives of key research user stakeholders of the Research Cloud and NSP infrastructures. Advice from the Research Reference Group will be available to the PSC and to the NeCTAR Project Board. The PSC will actively consult with the Research Reference Group on:
● Strategic direction of the Research Cloud and NSP Programs to maximise the delivery of value to the Australian research community;
● Appropriate service levels and operational stability of the infrastructure and services offered.
The PSC will provide consolidated reports to the NeCTAR Project Board and Research Reference Group on the status of Research Cloud and NSP operations across the infrastructure operators, including:
● Delivery against agreed Service Levels; and
● Schedule for ongoing infrastructure and service deployment.
The NeCTAR Project Board will extend an invitation to the members of the PSC to nominate a member to attend Project Board meetings to facilitate reporting and communications between the PSC and the Project Board.
The PSC will also be responsible for ensuring metrics on Service Levels, Service Availability, Service Usage and other key performance metrics are transparently published to the research users of the
Research Cloud and NSP.
NeCTAR and the PSC will continue to support alignment of the PSC operations with the oversight and management arrangements of the RDSI Super Science and NCRIS Projects.
NeCTAR Platforms Research Reference Group (RRG)
NeCTAR will establish in early 2014 a NeCTAR Platforms Research Reference Group to provide advice to the Platforms Steering Committee and the NeCTAR Project Board on the needs and requirements of research users of the Research Cloud and NSP. Representation is expected to include:
● Representatives of the NCRIS/SS capabilities; ● Research institutions;
● Operators and developers of services deployed on the Research Cloud and NSP ● NeCTAR Virtual Laboratory projects; and
● Other research users of the Research Cloud and NSP infrastructure.
NeCTAR will consult with the NeCTAR Project Board and the members of the PSC to identify
candidates for participation on the Research Reference Group. Terms of Reference for the Research Reference Group will be developed by the NeCTAR Directorate and the PSC for endorsement by the NeCTAR Project Board. Membership will also be by endorsement of the NeCTAR Project Board. The Research Reference Group may also consult more broadly with the Australian research sector in seeking endorsement and feedback on the value of the services offered through the Research Cloud and NSP programs.
NeCTAR Directorate
The NeCTAR Directorate carries out the program management activities of the NeCTAR Super Science and NCRIS Projects, including oversight of the 39 existing NeCTAR Super Science subprojects. The roles of the Project Directorate staff are included in the NeCTAR Final Project Plan. The NeCTAR Directorate program management responsibilities will be fully absorbed into the existing Super Science NeCTAR Directorate budget to 31 March 2015. Under the existing funding arrangements for the NeCTAR Super Science Project the NeCTAR Directorate will operate at a reduced staffing level from 31 March 2015 to 30 June 2015. NeCTAR NCRIS Project funds will be used to extend the Finance Officer, Project Coordinator and Deputy Director roles within the NeCTAR Directorate from 1 April 2015 to 30 June 2015 in support of the obligations of the Directorate under the NeCTAR NCRIS Project.
Governance Overview
Sustaining Governance
The governance arrangements for the NeCTAR Super Science and NCRIS Projects have been
designed to ensure the interests of all key stakeholders are appropriately represented, including the research end-‐users of the infrastructure (through the RRG), the infrastructure operators and service providers (through the PSC), research institutions and organisations (through the Project Board) and the central funding agency (through the Lead Agent and attendance at meetings of the NeCTAR Project Board).
Throughout the period of this NCRIS Project the NeCTAR Directorate and the operators of the NeCTAR infrastructure will seek to secure additional sources of funding for enhancing and sustaining operations of the NeCTAR infrastructure. Accordingly, the NeCTAR Directorate will work with the NeCTAR Project Board, the Department, the PSC, the Research Reference Group and other stakeholders to undertake a review of the operations of the existing governance arrangements from Q3 2014, including preparation of recommendations for changes to future governance arrangements to improve sustainability of appropriate governance arrangements.
Obligations of Operators of the Research Cloud and NSP node infrastructure
Obligations for NeCTAR funded Research Cloud and NSP nodes with respect to the NeCTAR Governance arrangements are contained in the NeCTAR Request For Proposals and existing contractual arrangements, including the following excerpt from the NeCTAR Super Science Project RFP:
Research Cloud Nodes are charged with the responsibility to work collaboratively with all participating nodes of the Research Cloud to:
● work with the Lead Node in developing procedures and policies for the deployment and operation of the Research Cloud Infrastructure Framework;
● work with the Lead Node in developing architectural requirements for the underlying hardware deployments at the Nodes;
● procure, deploy, maintain and operate the underlying compute and storage infrastructure according to the architectural requirements published by the Lead Node and agreed by the NeCTAR Platforms Steering Committee;
the Node in accordance with the procedures and policies published by the Lead Node and agreed by the NeCTAR Platforms Steering Committee;
● monitor and report on service delivery at the Node to ensure the infrastructure operates within the appropriate service levels;
● respond to support requests generated from the Research Cloud central support service; and
● receive advice and strategic direction from the NeCTAR Project Board through its subcommittees, including the NeCTAR Platforms Steering Committee under the Research Cloud governance arrangements described in Section 4, and act in accordance.
The mutual obligations of the operators of the Research Cloud nodes, the Platforms Steering Committee and the operators of the node infrastructure are described more fully in the Research Cloud and NSP Operations Subproject plan (Appendix A.1).
3 PROJECT INFRASTRUCTURE
3.1 The Sub-‐projects
Subprojects to be undertaken are described in Attachments A.1 and A.2 of this project plan.
3.2 Assets
The NeCTAR NCRIS Subprojects described in Appendix A.1 and A.2 will expend funds in the operations of infrastructure assets created and deployed under the NeCTAR Super Science programs. Additional assets may be acquired or created in support of the operations of those Subprojects. Any Assets created under the NCRIS Subprojects will be recorded and maintained in a NeCTAR NCRIS Asset Register alongside the existing NeCTAR Asset Register for Assets created under the NeCTAR Super Science Project.
4 RISK MANAGEMENT
[Please provide a risk management strategy covering key risks]
The NeCTAR Directorate will maintain a risk register for the NeCTAR NCRIS Project. That risk register will be reviewed regularly in partnership with the PSC and reported to the NeCTAR Project Board. Initial program wide key risks and proposed management strategies are included below.
Furthermore, each of the infrastructure and services operators are required to actively manage internal risks through their obligations under the agreements with NeCTAR.
Risk: Delays in underlying infrastructure deployment Likelihood: Moderate. Impact: High
Impact: Deployment of Research Cloud infrastructure at the research cloud nodes has been significantly delayed from initial projections. Further delays will further negatively impact the
availability and delivered value of the services and infrastructure.
Mitigation: Efforts to accelerate and prioritise the deployment and production readiness of cloud infrastructure are underway. The Lead Node is actively supporting the nodes to fast-‐track
on-‐boarding and federation of the Research Cloud nodes. The PSC will maintain and report to the Project Board a consolidated schedule for infrastructure deployment to identify further delays. Risk: Delays due to external dependencies
Likelihood: Moderate. Impact: High
Impact: There are notable dependencies for operation of many of the proposed services on
infrastructure and services being delivered through external providers. In particular, the RDSI project is delivering research storage infrastructure at the research cloud node operators. Further delays in availability of data storage capacity and data access services will impact adversely on the value of Research Cloud services and Virtual Laboratory operations. Other notable external dependencies include the group management services being developed by the AAF.
Mitigation: Improved alignment and planning in provision of data storage services between RDSI and NeCTAR. The operators of the Research Cloud nodes are also operators of the RDSI Node storage infrastructure and services. Early engagement with other providers of key services and
infrastructure, eg. the AAF.
Risk: Delays in appointing staff to roles funded through this NeCTAR NCRIS Project Likelihood: Medium, Impact High
Impact: Delays in appointing key operational staff will impact on the timely delivery for improvements to service operations and the availability of higher-‐level services.
Mitigation: Existing operators of the NeCTAR infrastructure are well placed to access appropriate expert staff through their relationships with key eResearch and IT stakeholders through recruitment and secondment. Secondment of staff from partner organisations will provide an appropriate mechanisms for rapidly filling key early roles. NeCTAR has also encouraged that operators foster a mixed dev-‐ops culture in infrastructure operations. This enables the opportunity for existing staff currently employed predominantly in creation and development roles to transition to predominantly operational roles which complement their development roles.
Risk: Delays in establishing Research Reference Group Likelihood: Medium, Impact Moderate
Impact: Delays in identifying and recruiting appropriate representatives for participation on the Research Reference Group will delay the voice of the research users to inform the operations and direction of the Research Cloud and NSP platforms.
Mitigation: Agree an early strategy for nomination and selection of representatives on the Research Reference Group. Members of the PSC and the NeCTAR Project Board to assist the Directorate in supporting nominations from the identified stakeholders groups. If this risk is realised alternative methods for access to the voice of the research users could be provided through direct consultation
with key groups such as the NeCTAR Virtual Laboratory projects.
Risk: Failure to achieve stable operations for infrastructure and services Likelihood: Moderate, Impact: High
Impact: Failure to meet agreed expectations for robust and responsive operations to agreed service levels will negatively impact on the delivered value for Australian research, reduce uptake and utilisation of services and impact negatively on attempts to establish sustainable models for ongoing operations of the services.
Mitigation: Establish a robust framework for transparent reporting of service levels and service availability. Establish a comprehensive monitoring framework for early identification of service problems and the operational teams to respond rapidly.
Lack of uptake and utilisation of offered services and infrastructure Likelihood: Low, Impact: High
Impact: Reduction in delivered value to Australian research. Negative impact on attempts to establish sustainable models for ongoing operations of the services. Current experience suggests the likelihood is low. The existing NeCTAR infrastructure is seeing high demand, especially in the case of the Research Cloud where supply is failing to keep up with demand as of December 2013.
Mitigation: Clear plan for communications of the value of the NeCTAR services coordinated across the operators and stakeholders of the current infrastructure providers and service operators. Prioritise operation of services with end-‐user self-‐service capability and low barriers to initial access and use. Training and outreach activities engaged with research disciplines.
Failures in Governance to achieve the project objectives Likelihood: Low, Impact: High
Impact: Failures in the governance arrangements may lead to delayed delivery of infrastructure and services, fragmentation in service offerings, failures to deliver key infrastructure and impact
negatively on establishing a basis for sustaining operations beyond 2015.
Mitigation: Each of the governance bodies will operate in a manner which provides high levels of transparency to other governance bodies in order to identify at an early stage emerging issues and breakdowns in the existing governance arrangements. The governance arrangements will be reviewed in late 2014.
5 IMPLEMENTATION STRATEGY AND FINANCIAL INFORMATION
5.1 Financial Projections
A summary of the Financial Projections for the NeCTAR NCRIS Project is outlined in the tables below. Detailed statements for each of the Components / Subprojects are shown in the relevant
Attachments A.1 and A.2.
5.1.1 Summary of overall income and expenditure for the Project (GST Exclusive)
$ (GST Exclusive) Financial year: 2013-‐14 2014-‐15 Total Income
NCRIS 2013 -‐ Cash Contribution $4,136,000 $5,264,000 $9,400,000
Recipient and Other Cash contribution $0 $0 $0
Subtotal -‐ (NCRIS 2013 and Co-‐Investment) $4,136,000 $5,264,000 $9,400,000
In-‐Kind Contribution $0 $0 $0
Total Income (NCRIS 2013, Co-‐Investment and In-‐Kind) $4,136,000 $5,264,000 $9,400,000 Expenditure
Total Expenditure (Cash and In-‐Kind) $4,136,000 $5,264,000 $9,400,000 Participants in the existing NeCTAR Super Science components which will benefit from investment under this NCRIS proposal have each committed firm coinvestment in the operations of the
infrastructure identified in the project proposals. This includes the operations of the Research Cloud Nodes and each of the Virtual Laboratory and eResearch Tool projects. These commitments of coinvestment will be maintained. The proposed investment under this project plan are
complementary to the existing commitments of coinvestment. As the Project executes,
opportunities for additional commitments of coinvestment will be identified and articulated within the subprojects and programs of work.
Commitments of operational coinvestment by Research Cloud nodes under the NeCTAR Super Science Research Cloud and NSP node subprojects:
RC Node Operator Co-‐investment
RC001 -‐ University of Melbourne $2,340,000 RC000 -‐ University of Melbourne -‐ Lead Node $600,000 NSP001 -‐ University of Melbourne $637,143
RC003 -‐ ANU/NCI $2,911,087
RC005 -‐ QCIF $1,535,000
RC007 -‐ Monash University $2,489,000
RC201 -‐ Intersect $2,301,304
RC202 -‐ University of Tasmania $1,630,632
RC203 -‐ eResearch SA $1,768,434
RC204 -‐ iVEC/CSIRO $1,898,000
Declaration by authorised representative of the lead agent
I, Jim McCluskey, make the following declaration:
1. that I have authority to make this declaration on behalf of the lead agent, The University of Melbourne, ABN 84 002 705 224;
2. that all of the information that I have provided to the Commonwealth (including the contents of this declaration and attachments) is complete, true and correct; and
3. I have taken all reasonable steps to ensure its accuracy.
I understand that the Commonwealth is relying on the accuracy of the information provided by me (including the contents of this declaration) in determining the lead agent’s funding under the National Collaborative Research Infrastructure Scheme 2013.
Signature: ____________________________________________Date: _____________
Name: Professor Jim McCluskey
A1 Research Cloud and NSP Operations Subproject
This Component/Sub-project is of High/Medium/Low priority(Please provide brief reasoning for Priority level) NeCTAR Research Cloud
The NeCTAR Research Cloud has seen substantial uptake and utilisation by Australian researchers and research service providers, including the NeCTAR Virtual Laboratories and eResearch Tools.
Operational funding under NCRIS 2013 would ensure the capacity to: ● Continue to operate the NeCTAR Research Cloud as a federated
national infrastructure to mid 2015;;
● Meet research community expectations for research cloud service levels;; and
● Reduce overall operational costs across the federation of research cloud nodes.
The eight nodes of the NeCTAR Research Cloud have committed to support the costs of operating the infrastructure deployed at the nodes through co-investment identified in their agreed NeCTAR Project Plans. This subproject will support only those activities which fall under the scope of the operational costs associated with the current NeCTAR Lead Node Activity. As envisaged in the NeCTAR Project Plan, the Lead Node Activity enables reduction of operational costs at the nodes through consolidation of shared services and operations.
NeCTAR National Servers Program
The National Server Program (NSP) seeks to provide high levels of service for hosting core national eResearch services.
NCRIS Renewal funding for the NSP program would enable: ● The continued operation of the NSP as a national research infrastructure service to mid 2015;;
● Improved capacity to identify and deliver service levels consistent with the expectations of the NSP program and the needs of research service providers;; and
● Improved alignment and efficiency of operations between the NSP and the Research Cloud.
As an operationally funded service, the program would continue to deliver efficiencies for other research infrastructure providers, including the domain-oriented NCRIS and Super Science capabilities.
High Priority
Is this Component/Sub-project receiving support under the Collaborative Research Infrastructure Scheme. If yes,
● Is support to extend the time period of operations? ● Is support to increase the capacity under CRIS?
No
What is the total funding sought from NCRIS 2013 to support operations from 1/7/13 to 30/6/15?
Please estimate the following eligible costs:
● management and governance costs at both the node and national governance levels
● salaries and on costs for technical staff
$6,000,000
$767,000 $5,033,000
● infrastructure maintenance ● utilities
● rent
● consumables
● international collaborations
● Other (skills training/development, consultancy services)
$0 $135,000 $0 $0 $50,000 $15,000
A1.1 SubProject Description
The NeCTAR Research Cloud and NSP Operations Subproject:
The Research Cloud Lead Node Activity has been established under the NeCTAR Super Science project to create and deploy the common cloud software infrastructure platform. This Lead Node Activity will be reconstituted as the NeCTAR Research Cloud and NSP Operations subproject to reflect its role as encompassing the deployment and operation of the Research Cloud fabric across the nodes of the Research Cloud and to operate the NSP nodes during the funding period.
The reconstituted Research Cloud and NSP Operations Subproject will be structured as a consortium of the nodes of the Research Cloud and the NSP:
1. Subproject Lead: University of Melbourne
2. Framework to distribute funding to nodes, as foreseen in the NeCTAR Project Plan: a. According to contributions to Research Cloud Operations Work Plan. 3. Governance:
a. NeCTAR Platforms Steering Committee (PSC) as oversight and coordination b. PSC oversight of Research Cloud Operations Work Plan and Planning processes
Scope of the Research Cloud Operations Activities:
1. Operate and maintain NeCTAR (OpenStack) Cloud Fabric Services
a. Operate central cloud services, eg: OpenStack Dashboard, API Server, Scheduler, ... b. Plan and coordinate major OpenStack software maintenance upgrades across all
nodes every 6 months
c. Support the day to day operation and maintenance of the National Openstack software infrastructure service
d. Plan and coordinate weekly cloud-‐wide security patching, bug fixes and upgrades e. Operate the national development and test environment, which all node operators
have access to for the development and testing of new features 2. Operate and maintain cloud monitoring and reporting services
a. Manage a central monitoring and management service that is capable of monitoring all cloud infrastructure across nodes, sending automated alerts as needed;
b. Generate and provide access to usage data, service level data and help desk statistics for reporting and service improvement purposes
c. Operate management capabilities for virtual machine images and instances. 3. Security monitoring and incident response
a. Coordinate responses to security incidents across the research cloud nodes. b. Proactive and responsive monitoring of cloud-‐wide virtual instance traffic. 4. Quality Assurance of NeCTAR Virtual Machine Images
a. Support, patching, testing, documentation and maintenance of NeCTAR reference Virtual Machine images
5. Continuous improvement
a. Provide and maintain the central configuration and code repository, from which all cloud nodes are built.
b. Manage a change control and review service for all changes, updates and development activities associated with the Research Cloud fabric;
c. Manage a continual improvement program to enhance existing services according to priorities advised by the NeCTAR PSC and its advisory committees; ensuring their effective and efficient use by the research community;
d. Managing and incorporating research user community feedback on the status and priority of services.
6. User community support, Documentation and Helpdesk
a. Coordinate national cloud break/fix and incident responses
b. Provide Level 1 support across the nodes, and co-‐ordinate Level 2 and Level 3 support functions between nodes.
c. Develop and maintain community support knowledge base.
d. Coordinate with and build on the outputs of existing user support services operated by the nodes and the AeRO User Support Working Group ICT Support Framework Project.
7. Operation of Cloud Ecosystem services
a. Support and operations of Cloud Platform as a Service (PaaS) and Software as a Service (SaaS) offerings, including, and not limited to:
■ Orchestration and management of software services on the Research Cloud: (OpenStack Heat project);
■ Object (cloud) storage services such as OwnCloud; ■ Database as a Service (DaaS) (OpenStack Trove project);
■ Domain-‐Name as a Service (DNSaaS) (OpenStack Designate project); ■ Cloud monitoring and reporting tools: (OpenStack Ceilometer project); ■ Virtual Clusters in the Cloud service; (eg. StarCluster)
■ Hadoop “big-‐data” parallel processing services: (OpenStack Savanna project);
8. Cloud Resource Allocations
a. Co-‐ordinate the operations, maintenance and support of the allocations processes b. Co-‐ordination and alignment with RDSI resource allocation schemes;
c. Participation in National Computational Merit Allocation Scheme (NCMAS) 9. Governance and Processes
a. Support NeCTAR Governance arrangements and processes, including ■ Platforms Steering Committee
■ Technical Advisory Group
■ Platforms Research Reference Group (including participation by NCRIS and Super Science capabilities)
10. International Collaboration
a. Provide representation to the International Openstack community on behalf of the needs of the Australian research community (including attendance at OpenStack summits)
b. Coordination and alignment with emerging international Research Cloud infrastructures and research programs
11. National Server Program Operations
a. Operate NSP computing hardware infrastructure
i. Maintenance of NSP server hardware (extension of maintenance agreement) ii. Data Centre hosting, including power and cooling
iii. Server equipment refresh/refurbishment. iv. System administration
b. Improve alignment of NSP Operations with Research Cloud Operations
i. Common security management, incident response and help desk operations c. Operate base NSP platforms services (aligned with and leveraging capabilites in the
Research Cloud Operations)
i. Virtualised Data Centre management infrastructure
ii. Maintain mature development and test environments as per the NeCTAR Service deployment design (in support of staged service deployment) iii. Self service resource allocation management
iv. Monitoring and reporting of usage data, service levels and help desk statistics for reporting and service improvement.
v. Plan and coordinate major software system upgrades across data centres vi. Plan and coordinate software and operating system patching, bug fixes and
upgrades
d. Operation of services in support of high service levels i. Robust failover and Disaster Recover (DR) capability ii. Expanded support hours coverage
iii. Robust clustering of hardware across data centres iv. Billing operations
NCRIS Renewal funding for the NSP program would enable the program to continue and to fully fund the operational costs of the service. As an operationally funded service, the program would
continue to deliver efficiencies for other research infrastructure providers, including the NCRIS and Super Science capabilities. Operating the NSP Platform under the arrangements for operation of the Research Cloud Program will:
● Reduce the costs of sustaining the NSP service:
○ Through consolidation of personnel, expertise and common services
● Ease the process for research users to manage software infrastructure deployed across both platforms
Sustaining the NSP service and mitigating risk
Over the NeCTAR NCRIS Project period, NeCTAR will seek to improve the ongoing sustainability of the NSP service by:
● Reducing the cost of operating the service through consolidation of operational costs with the Research Cloud program; and
● Addressing the risk to the NSP service of reliance on the single existing node at the University of Melbourne by:
remaining $500k in NeCTAR Super Science NSP program funding to establish one or more NSP nodes, to be operated by an existing research cloud node or nodes.
Alignment of Research Cloud and NSP Platforms
The NSP and the Research Cloud offer complementary services. The Research Cloud offers access to elastically scalable, self-‐service virtual machine infrastructure consistent with commercial cloud infrastructure providers. The NSP aims to provide more robust individual service levels associated with a managed virtual data centre infrastructure.
The Research Cloud and NSP may be viewed as differential service-‐level components of a national computing platform. As the Research Cloud and NSP services mature their associated services will be consolidated where possible to efficiently support the operational needs of the NSP and the Research Cloud. Such areas for consolidation include User Support, Monitoring, Resource Allocation and Incident response.
While the NSP will incur a requirement to contribute to these consolidated operations of the Research Cloud, the NSP will leverage the capabilities provided through the Research Cloud Operations, including:
1. User support, documentation and helpdesk operations a. Provide Level 1, 2 and 3 helpdesk support functions b. Define and refine service levels and agreements c. Maintain community support knowledge base 2. Security monitoring and incident response
a. Monitor and coordinate responses to security incidents. 3. Resource Allocation
a. Operations, maintenance and support of the central allocations service.
Review of NSP operational arrangements
In Q1 2014 a review of the existing operational arrangements for the NSP will be initiated by NeCTAR with support from the NSP node operator and Platforms Steering Committee. The findings and recommendations of the review will be provided to the Project Board for endorsement. The review will evaluate and provide recommendations on:
● The appropriate underlying platform for the NSP:
○ Continue with the existing Citrix CloudStack solution; or ○ Migrate to an OpenStack-‐based platform; or
○ Migrate to another platform, such as a VMWare platform
● Addressing the risks arising from reliance on a single operator for the existing NSP node, including a multi-‐node solution for the NSP, eg:
○ Identify and fund the establishment of one or more additional nodes of the NSP from the $500k available under the Super Science NSP program.
NeCTAR notes that if an OpenStack based platform were deemed appropriate for delivering the service levels required for the NSP, there would be substantial benefits arising from efficiencies of operations with the Research Cloud, which would improve the outlook for sustaining NSP operations
within the Research Cloud program. It may be expected that operators of a future Research Cloud service, with a range of flexible funding mechanisms would provide a continuum of service levels to the Australian research community, with the current NSP representing the highest service level offerings within that continuum.
Since initial launch in 2011, the NSP Service has operated continuously and grown to host services for 25 service providers. The project first delivered a fully functioning hosting service in 2011 to a handful of users, rising to 16 services supported in June 2012 (Basic Access Phase, NSP1), and 25 in June 2013 (Full Service Access, NSP2).
NSP Service Levels
The availability of operational funding for the NSP through the NCRIS 2013 program will permit support of improved service levels consistent with the expectations of operators of significant national research infrastructure services.
Experience with existing and prospective users of the NSP indicates that the existing service levels are broadly sufficient for hosting researcher owned services. However, operators of mature, robust and large-‐scale infrastructure services require improved service levels from the NSP as a hosting platform. The Platforms Research Reference Group will assist in providing advice to the NSP operators on appropriate service levels.
Initial focus on Creation and Development of Research Cloud Infrastructure
In the initial phase of the Research Cloud, the Lead Node Activity has been focussed on creation and development of the cloud middleware infrastructure (based on OpenStack) and the central services which comprise the Research Cloud. Currently, operational costs are borne through co-‐investment by the University of Melbourne. The current agreement with the University for the Lead Node Activity expires at end 2013. Continuing to sustain the operations of the Lead Node would place an undue burden on the University of Melbourne. The NCRIS 2013 funding will enable continued and improved operations of the infrastructure and services established by the Lead Node throughout the period of operations for all the Research Cloud nodes.
Increasing focus on operational maturity to meet service level targets
As the Research Cloud fabric stabilises and additional nodes of the Research Cloud come online, the Lead Node Activity becomes increasingly focused on operational activities to support the service levels required by the Australian research community. While the Research Cloud nodes are fully committed to support the operations of the infrastructure at the nodes, there is a requirement for a collaborative Research Cloud Operations project to maintain and support:
● The NeCTAR Research Cloud Framework software base (based on OpenStack)
● Centrally deployed infrastructure cloud services, including the User Dashboard, API Servers and Resource Schedulers.
● The ecosystem of high-‐level cloud services required by users of the Research Cloud, including services in support of:
○ Platform as a Service (PaaS) and Software as a Service (SaaS) offerings. NCRIS renewal funding in support of the operations of the Research Cloud will:
● Ensure the capacity to continue to operate the NeCTAR Research Cloud as a federated national infrastructure
● Reduce operational costs across the federation;
● Improve ability to deliver service levels required by research users of the Research Cloud; and
● Provide time for NeCTAR and the service operators to identify additional sources and implement additional mechanisms for co-‐funding the future capital and operational costs of the Research Cloud and NSP infrastructure.
Future operations and sustainability
As the federated services and node operations mature throughout the NCRIS funding period, the NeCTAR Research Cloud and NSP Ops project will seek to improve efficiencies in order to minimise the cost of sustaining the infrastructure while continuing to deliver value to the Australian research community. As articulated in the NeCTAR Super Science Project Plan, this will include identifying opportunities to engage with commercial partners in support of the ongoing operations of the Research Cloud, especially as the market for OpenStack cloud services and operators matures. Furthermore, NeCTAR will work with the funded operators through the Platforms Steering
Committee to develop a Research Cloud and NSP Operating Model for the Research Cloud beyond the funding period. This will include and articulate:
● The operating costs of the federated and node services required to sustain the service beyond the funding period;
● The suite of services offered through federated operations and by the node operators; ● The dependencies and relationships between the federated service operations of the
Research Cloud and the NSP and the node operations;
● The relationship between the Research Cloud and NSP operations; and ● Recommendations on implementation of the operating model.
The Research Cloud and NSP Operating Model will be provided to the NeCTAR Project Board and the Department.
NeCTAR will also work through the Platforms Steering Committee to develop proposals for identifying additional sources of funding to sustain the federated operations and the node operations in support of transition to a mixed funding model beyond the funding period. NeCTAR seeks to provide a high degree of autonomy for research cloud nodes in developing funding models for sustaining the node operations according to the needs of their partners and
stakeholders. The operating model will retain that autonomy for node funding models. Nodes are encouraged to collaborate in the development of their proposed funding models and to share their approaches to development of those plans through the Platforms Steering Committee.
A1.2 SubProject Governance and Management Arrangements
This Subproject will operate under the overall governance arrangements for the NeCTAR NCRIS Project described in Section 3.2 above. We articulate here the basis of the relationships and responsibilities between the Platforms Steering Committee, the NeCTAR Cloud Ops project (formerly the Lead Node Activity) and the operators of the Research Cloud and NSP nodes. The Research Cloud and NSP Ops Subproject will support the development, deployment and operations of the NeCTAR Research Cloud middleware (OpenStack) and “central” cloud services foruse by all the nodes of the Research Cloud as well as the operations of the NSP nodes. The activities to be funded under this component will have oversight and guidance from the Research Cloud node operators through their representation on the NeCTAR Platforms Steering Committee. This component also transitions the Lead Node Activity to a distributed activity (Research Cloud Ops) supported by contributions from node operators to be funded under this component.
The activities undertaken through these funded activities will consolidate operational costs which would otherwise be borne by the node operators individually, leading to reduced operating costs at the nodes.
The Platforms Steering Committee will endorse and recommend operational leads for activities which are not designated to be led by the University of Melbourne (the current Lead Node). Each activity may also be implemented as a distributed activity, subject to agreement by the participants, with node operators funded to support personnel to participate in the activity. Funding will be provided to nodes to participate in these activities based on their demonstrated capability to support the activities.
Some of the core activities will be led by the University of Melbourne in support of the continuation of those activities undertaken under the Super Science Lead Node funding. Some key roles are highlighted to be established at the University of Melbourne in support of the core operations. Wherever possible, all of the activities funded under this component will seek to align their
operations with those established under the RDSI Super Science and NCRIS Projects, including, and especially, in the areas of resource allocations, user support, resource status and usage monitoring and security management. Through the Research Cloud node operators, who are also node
operators for the RDSI infrastructure, NeCTAR will support the alignment and consolidation of NeCTAR and RDSI operating processes and services at the nodes.
Throughout 2013 NeCTAR and RDSI have aligned their governance and management processes, including through:
● Co-‐scheduling of NeCTAR PSC and RDSI Node Steering Committee meetings; ● NeCTAR Director participation in RDSI Node Steering Committee meetings.; ● RDSI Director participation in NeCTAR PSC meetings; and
● Cross project participation in Working Groups.
NeCTAR will continue to align governance processes with the RDSI project over the funding period. Roles and Responsibilities of the Research Cloud and NSP Ops Project (Formerly the Lead Node Activity)
The role of the Lead Node Activity (now Research Cloud and NSP Ops) of the Research Cloud is described in the NeCTAR (Super Science) Final Project Plan:
4.3.1.2 Research Cloud Lead Node
will create, deploy and operate the Research Cloud Infrastructure Framework, and monitor service delivery across all other nodes.
The Lead Node of the Research Cloud is charged with the responsibility to work collaboratively with all participating nodes of the NeCTAR Research Cloud to:
● Create, deploy and operate the defined Research Cloud Infrastructure Framework;
● Maintain and update the Research Cloud framework in consultation with all Research Cloud node hosts;
● Develop and publish procedures and policies for the deployment and operation of the Research Cloud Infrastructure Framework as agreed with the NeCTAR Platforms Steering Committee;
● Develop and publish architectural requirements for the underlying hardware deployments at the Nodes as agreed with the NeCTAR Platforms Steering Committee;
● Monitor and report on service delivery across the other distributed nodes to ensure that the Research Cloud infrastructure runs within the appropriate service levels;
● Act as a central point to triage Research Cloud allocation requests, fault reports, and so on; ● Provide and manage user and administration interfaces to the NeCTAR Research Cloud
services;
● Receive advice and strategic direction from the NeCTAR Project Board through its subcommittees, including the NeCTAR Platforms Steering Committee under the Research Cloud governance arrangements described above, and act in accordance.
The Research Cloud Infrastructure Framework referenced above has subsequently been identified as the OpenStack open source cloud computing platform.
Roles and Responsibilities of the Research Cloud and NSP nodes
The role and responsibilities of the operators of the RC and NSP node infrastructure are also described in the NeCTAR FInal Project Plan and Part B of the NeCTAR Request For Proposals:
Research Cloud Nodes are charged with the responsibility to work collaboratively with all participating nodes of the Research Cloud to:
● work with the Lead Node in developing procedures and policies for the deployment and operation of the Research Cloud Infrastructure Framework;
● work with the Lead Node in developing architectural requirements for the underlying hardware deployments at the Nodes;
● procure, deploy, maintain and operate the underlying compute and storage infrastructure according to the architectural requirements published by the Lead Node and agreed by the NeCTAR Platforms Steering Committee;
● deploy and operate the designated Research Cloud Infrastructure Framework at the Node in accordance with the procedures and policies published by the Lead Node and agreed by the NeCTAR Platforms Steering Committee;
● monitor and report on service delivery at the Node to ensure the infrastructure operates within the appropriate service levels;
● respond to support requests generated from the Research Cloud central support service; and
● receive advice and strategic direction from the NeCTAR Project Board through its subcommittees, including the NeCTAR Platforms Steering Committee under the Research Cloud governance arrangements described in Section 4, and act in accordance.
Some key roles have been identified below to be deployed at the current Research Cloud Operations Lead Node (University of Melbourne), or to be associated with the NeCTAR Directorate or NSP node.
For other roles and activities the Platforms Steering Committee will recommend to the NeCTAR Directorate a distribution of funds to designated node operators for personnel and activities according to capability and interest of the node operators.
A1.3 Access and Pricing arrangements
Under the NeCTAR Super Science project access to the NeCTAR Research Cloud infrastructure has been free of charge to Australian researchers subject to merit-‐based allocation processes and in accordance with the principles articulated in the 2011 Strategic Roadmap for Australian Research Infrastructure.
During the term of operation of this Subproject, NeCTAR will work with the operators of the Research Cloud and NSP nodes to develop operational funding models for sustaining the ongoing operations of the Research Cloud and the nodes, including identification of additional funding sources and mechanisms to support future operations and capacity growth of the NeCTAR
infrastructure. Access to services on the basis of the Commonwealth investment will remain on the basis of the principles articulated in the roadmap. No cost recovery will be performed against expenses incurred by the operators funded through the NeCTAR Super Science or NCRIS funds.
A1.4 Outputs
[Please identify and provide a quantitative estimate of relevant outputs to be achieved by this Component/Sub-‐project using NCRIS support]
Output Estimate
National Research Cloud Operations fully established
Eight Research Cloud nodes fully established and participating in the national research cloud federation. Improved Service Levels
delivered at full scale deployment
Updated Service Level Agreement established with improved service levels and demonstrated delivery against those service levels.
Updated SLA to be proposed by PSC and endorsed by Research Reference Group and Project Board.
Monitoring, Reporting and Billing Service operating
Detailed Service availability and Usage monitored and Reported on Research Cloud website across all participating nodes of the Research Cloud.
Billing services operating in support of node business models.
Security processes across cloud federation documented and enacted.
Security incident management and response compliant with processes established in NeCTAR RFP
User Support and Help Desk operations established across all Research Cloud nodes
User Support and Help Desk response times compliant with service levels to be agreed, supporting the national cloud user base (up to 8000 users).
High-value cloud ecosystem services deployed and accessible on the Research Cloud
Significant research user uptake of ecosystem services across the research cloud nodes. Services in use by over 100 research cloud software deployment.
Continued operations of NSP Hosting service available for NCRIS/SS capabilities and eResearch service providers
Improved NSP service levels Improved Service Levels offered in updated SLA and demonstrated delivery against those service levels. NSP Disaster Recovery
Capability Fail-‐over capability demonstrated across multiple datacentres.
A1.5 Key Performance Indicators
Please identify and provide estimated figures for key performance indicators (see Guidance) [Consider using KPIs / performance measures in NCRIS 2006/Super science/CRIS agreements]
Key Performance Indicator Estimate
Number of Research Cloud Nodes operating at full capacity within Research Cloud federation
8
Scale of computing resource available to research cloud users
30,000 CPU cores
Breadth of uptake of Research Cloud and NSP services
Resource Allocations across 80% of 2-‐digit Field of Research (FOR) codes.
Number of registered users of the Research Cloud and NSP services
6000 Users across Australia
Number of high value cloud ecosystems commissioned and operating
7
Number of cloud services using high-‐level cloud ecosystem services.
50
Number of Virtual Laboratory and eResearch Tool sub-‐project operating production infrastructure on the Research Cloud and NSP
20
Number of NCRIS and Super Science capabilities operating infrastructure on the Research Cloud and NSP
>5
A1.6 Financial
The following table includes the proposed budget allocation to the activities agreed by the members of the Platforms Steering Committee and proposed in the Draft Project Plan (listed above.)
Summary of NCRIS Budget Allocations by Component Activity: Activity and Roles
Funding Allocation
1. Operate and maintain NeCTAR (OpenStack) Cloud Fabric Services $1,191,000 2. Operate and maintain cloud monitoring and reporting services $228,000
