In practice, executing an algorithm that runs over a large number of iterations de- lays the output. In other words, computational methods in science require lots of processing time. One way to overcome this obstacle is to aggregate computing power in a way that delivers much higher performance than a single desktop computer or workstation. These are very exotic computers by virtue of the elements inside them, and the scale at which they are built. The University of Cape Town via the Informa-
tion and Communication Technology Services (ICTS) offers such facilities with the
aim of supporting the scientific community.
This section serves as a mini-manual to access the UCT ICTS HPC cluster II for scientists using Windows as operating system. For further information the interested readers should access the service via the following link:
http://srvslnhpc001.uct.ac.za/eresearch/. A list of available softwares that
are on the clusters by default can be found at
http://srvslnhpc001.uct.ac.za/eresearch/?page id=73
2.7.1 Connecting to the UCT ICTS HPC cluster
The following softwares must be downloaded in order to facilitate the access to the UCT ICTS HPC cluster as well as file transfers, scripts editing, job submissions:
• PuTTY which is a free implementation of SSH for Windows platform. The down- load page for PuTTY is
http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html.
• WinSCP is an open source free SFTP Client, FTP Client, WebDAV client and SCP client for Windows. Its main function is file transfer between a local computer and a remote computer, the HPC cluster to be more precise. The download page for WinSCP is http://sourceforge.net/projects/winscp/.
Connecting with WinSCP
WinSCP allows the user to navigate through the folders in the cluster as well as to copy files or folders from a local computer to the cluster and vice versa. Figure 2.17 shows the window where the following details would have to be typed in order to login:
• Host name: hex.uct.ac.za
• User name: campus ID number
• Password: supplied by the HPC cluster administrator.
After the abovementioned details have been provided, WinSCP prompts the user to a new window as in figure 2.18:
Figure 2.17: Login Window
Connecting with PuTTY
Through PuTTY, users access the cluster using an SSH protocol. SSH (which stands for ’secure shell’) ensures a highly protected connection against eavesdropping, hijacking and other attacks. Connecting to the UCT HPC cluster using PuTTY only requires the user to enter the Host name hex.uct.ac.za as it is shown on Figure 2.19. Once the personal profile details have been entered, PuTTY prompts the user to a new window as in Figure 2.20 where the user should type their campus id number and password. Then once the abovementioned steps are executed, PuTTY prompts the
Figure 2.19: Configuartion Window
user to figure 2.21.
Figure 2.21: Inside the cluster
2.7.2 Interacting with the Cluster
At this point, it is possible to perform various operations using command lines. A typical set of operations that can be done on the cluster is the following:
• ls list information about files in current directory • cd change directory
• cp copy and paste • cd ∼ home directory
• mkdir create a new directory • mv move file
• rm delete files • vim text editor
• qstat request the status of jobs, queues • qsub job submission to the cluster.
Shell Scripts
A shell script is a plain text file with Bash commands that is interpreted by a shell process. Below is an example of a shell script where the user can specify the number of nodes (computers) and the number of cores per node:
# P B S - N f i l e n a m e # P B S - q U C T l o n g
# P B S - l n o d e s = 1 : p p n = 1 : s e r i e s 6 0 0
cd / h o m e /e s s r e n 0 0 1/d i s s e r t a t i o n/ R-c o d e s
m p i r u n - h o s t f i l e $PBS_N O D E F I L E /opt/ exp _s o f t/ R- 3 . 0 . 2/bin/ R - - s l a v e CMD B A T C H f i l e n a m e .R
Before submitting a job to the cluster, the user must ensure that both the shell script and the R-file are in the same folder. The R-script that is processed looks like this
. l i b P a t h s (c(. l i b P a t h s () , "/h o m e/e s s r e n 0 0 1/d i s s e r t a t i o n/R - p a c k a g e s ") ) o p t i o n s( s c i p e n = 9 9 9 ) s u p p r e s s P a c k a g e S t a r t u p M e s s a g e s (l i b r a r y( fda ) ) s u p p r e s s P a c k a g e S t a r t u p M e s s a g e s (l i b r a r y( fda . usc ) ) s u p p r e s s P a c k a g e S t a r t u p M e s s a g e s (l i b r a r y( m a t r i x c a l c ) ) s e t w d("/h o m e/e s s r e n 0 0 1/d i s s e r t a t i o n/R - c o d e s ")
d a t a( aemet ,p a c k a g e = " fda . usc ")
dtt < - a e m e t$t e m p$a r g v a l s
t e m p < - as.d a t a.f r a m e( a e m e t$t e m p$ data,row.n a m e s= F )
c e n t . t e m p < - d a t a.f r a m e(a p p l y( X = temp , M A R G I N = 2 , FUN = scale, s c a l e= F A L S E ) )
r a n g e. dtt < - a e m e t$t e m p$r a n g e v a l # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # s o u r c e(" F u n c t i o n s 4 C h a p t e r 5 . R ") # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # S p l i t t h e d a t a s e t in t r a i n i n g s e t ( 7 0 % ) a n d t e s t s e t ( 3 0 % ) df. t e m p < - s p l i t d f ( d a t a f r a m e = c e n t . temp , s e e d = 808 ,s p l i t = 70) t r a i n . t e m p < - df. t e m p$t r a i n s e t # # # # # # # # # # B a s i s f u n c t i o n : G a u s s i a n b a s i s # # # # T e m p e r a t u r e K1 = seq(5 ,363) # n u m b e r of b a s i s f u n c t i o n s nK1 = l e n g t h( K1 ) y < - as.m a t r i x( t r a i n . t e m p [1 ,]) # s t a t i o n 1 l o g l a m < - seq( -10 ,10 ,0.01) # l a m b d a v a l u e s n l a m < - l e n g t h( l o g l a m ) GIC_ mat < - m a t r i x(0 , nK1 , n l a m ) for( i in 1: nK1 ) { B = G a u s s i a n_b s p l i n e s ( tt = dtt , m = K1 [ i ]) # b a s i s f u n c t i o n s n = K1 [ i ]
for( k in 1: n l a m ) {
ob < - Pen_Max_L i k e l i h o o d ( B = B , n = n , l a m b d a = l o g l a m [ k ] , y = y )
GIC_ mat[ i , k ] < - gic_fun ( y = y , ob = ob , n = n )
cat(" b a s i s f u n c t i o n : ", K1 [ i ] , i ," l a m b d a ", l o g l a m [ k ] ," \ n ")
} }
s a v e.i m a g e(" R_O u t p u t . R D a t a ")