On-demand (Pay-per-Use) HPC Service Portal

(1)

1

On-demand (Pay-per-Use) HPC Service Portal

Wang Junhong

High Performance Computing, Computer Centre

INTRODUCTION

The Service Portal is a key component of the On-demand (pay-per-use) HPC service delivery. The Portal, together with other Cloud computing tools, enables various operations such as online resource reservation, reservation management, automation of HPC cluster provisioning and management of reserved resources. As the Portal undergoes beta testing, I will share with you some details on how it works using a series of screen captures. There will also be some tips on how to make use of the reserved resources from On-demand (pay-per-use) HPC Subscription.

The process involves three key components:

1) HPC resources online subscription at the On-demand Service Portal 2) Management of reserved HPC resources at the On-demand Service Portal 3) Access and use of the reserved HPC resources

Step - 1

On-Demand HPC service portal login

Valid users can login to the On-demand HPC service portal using their NUSNET accounts (pre-registered at HPC portal) securely.

Figure 1 On-demand HPC Service Portal Login

Login to Portal Check Balance Make Reservation Verify Reservation Check Out Delegate Admin Set Permission User Guide Use Reservation O Onn--DDeemmaanndd H HPPCC SSeerrvviiccee

(2)

2

Step - 2

Check Project Balance

Before you can subscribe resources for your project, you must have positive “Funds” or “Credit” in the appropriate project account. The “Funds” or “Credit” is predefined between the user and service provider. It can be used by both the service provider and the user to ensure that there are enough funds for subscription. You can check the balance and expense via Bank  Your Balance, and the sample output is shown in Figure 2.

Figure 2 Home Page of On-demand HPC Service Portal

(3)

3

Step - 3

Shop HPC Resources by Making a Reservation

To shop on-demand pay-per-use HPC resources, you can make a reservation request, Reservation  Make Request as shown:

Figure 3 Reservation Menu

On the reservation request interface, you can define the Durations with Start Time and End Time; you can choose the Profile Type and server type via Procs Per Node, as in Figure 4.

After those are defined, click on Add To Request and continue shopping. Figure 4 Reservation Request Menu

Note: the price showed here is for illustration purpose.

(4)

4

Step - 4

Verify the Requested Reservation

The verification of the requested reservation will be displayed at the right column of the portal after you click on Add To Request.

The default number of nodes to reserve is 1 , you can input the exact number of nodes based on your needs. For example, if two nodes are required, enter 2 at the Qty box and click on Verify Request button to check if the resources are available and the cost for this reservation.

Figure 5 Reservation Verification

If the resources requested are Available, click on Add to Cart to check out and confirm the shopping.

(5)

5

Step - 5

Check Out the Shopping Cart

In your Shopping Cart, click on the Check Out button to confirm the reservation request; or click on Remove button to discard the request. Once you check out, the amount of this reservation will be deducted from your project fund (verify it via Bank  Your Balance.)

With a successful check out, the reservation will be accepted and provisioned based on the requested period, server type and number of nodes.

Figure 6 Check Out Shopping Cart

(6)

6

Step - 6

Manage a Reservation – Administration Delegation

You can delegate the administration work of your reservation to another valid NUS user, who may be your colleague, research fellow or student.

Click on Reservation  Management, with the specific Reservation highlighted, click on the Manage Users button to open the Manage VPC Users panel. Input the

username and enable the Admin access box, then click on OK button to confirm. Figure 7 Administration Delegation

(7)

7

Step - 7

Manage a Reservation – Permission to Use

If you make a reservation for your research team or a group of users, you (or the delegated administrator) can set the permission for other team members to access and use this reservation to run their computational jobs.

Click on Reservation  Authorization, click on the Modify button at the right of the Reservation to open authorization interface. Highlight the user(s) from the Users’ list at the left column and click on >> button to transfer the account(s) into the Existing Users at right side. Finally, click on Save button to confirm the permission setting.

Figure 8 Administration Delegation

(8)

8

Step - 8

Online User Guide at the Portal

You may check out the online user guide on the portal to learn how to • Check Balance

• Make Reservation • Use Reserved Resource • Job Submission

Step-by-step guide lists reference and help.

Figure 9 Online User Guide

At any time, click on Contact Us at the top-right corner of the portal to send us your queries or questions regarding the On-Demand HPC service portal.

Figure 10 Online Feedback

(9)

9

Step - 9

Login and Use Reserved HPC Resource

Users with permission to use the requested HPC reservation can submit and run jobs onto the reserved nodes exclusively via a special Linux-based login server. An example of how to make use the reserved HPC reservation to run a job is given as below, with the following Linux commands to be used.

Command Function

myres To list the authorized reservation that you can use

gcc, icc Available C/C++ compiler

gfortran, ifort Available Fortran compiler

mpicc, mpif77 Available MPI C/Fortran compiler

qsub Batch job submission command

bjobs, qstat To list batch jobs and their status

 List my authorized reservation #> myres

rsvid state owner totalnodes users stime etime --- --- --- --- --- --- ---

DRSV.3109 active ccev708 2 ccev708,g0600279 Tue Jul 12 14:57:43 2011 Tue Jul 19 14:57:43 2011 DRSV.3157 pending ccev708 2 ccev708,t0904009 Tue Jul 19 14:39:35 2011 Tue Jul 26 14:39:35 2011

 Compile a MPI program

#> mpicc helloworld.c –o hello.exe

 Submit MPI batch job to the reserved compute nodes

#> qsub -l advres=DRSV.3109 run_hello

1000288.stratus.nus.edu.sg

 List and check batch job status

#> bjobs

JOBID USER STAT QUEUE FROM_HOST EXEC_HOST JOB_NAME SUBMIT_TIME 1000288 ccev708 PEND NORMAL hostname.s Jul 13 17:32

#> qstat

Job id Name User Time Use S Queue --- --- --- --- - ---

1000270.stratus hostname.sh ccev708 0 Q NORMAL 1000288.stratus run_hello ccev708 00:00:00 C NORMAL

Note: the status of “C” listed by qstat indicates that the job is completed.  Check and view the result data

#> ls -ltr

total 2172

-rw-r--r-- 1 ccev708 svuusers 706 Jul 12 16:29 helloworld.c -rwxr-xr-x 1 ccev708 svuusers 1077254 Jul 18 14:50 hello.exe

(10)

10

-rw-r--r-- 1 ccev708 svuusers 222 Jul 18 14:51 run_hello

-rw--- 1 ccev708 svuusers 0 Jul 18 14:52 run_hello.e1000288 -rw--- 1 ccev708 svuusers 250 Jul 18 14:52 run_hello.o1000288 #>

#> cat run_hello.o1000288

Hello MPI from the server process! Hello MPI!

mesg from 1 of 6 on cougar-c05 Hello MPI!

mesg from 6 of 6 on cougar-c06

SUMMARY AND CONCLUSION

With the On-demand HPC Service Portal, researchers can shop for on-demand pay-per-use HPC resources when there is a need. The on-demand pay-per-pay-per-use resources can be requested and reserved dynamically based on the demand and available research fund. The reserved HPC resources are dedicated for authorised users. The owners have the full management right to authorise other users to use the reservation, request to add additional compute nodes and to extend the reservation period.