Software Development Toolkit Installation and User's Guide Version 1.1

(1)

Cell

Broadband

Engine

Software

Development

Toolkit

Installation

and

User's

Guide

Version

1.1

(2)

(3)

Cell

Broadband

Engine

Software

Development

Toolkit

Installation

and

User's

Guide

Version

1.1

(4)

Note: Beforeusingthisinformationandtheproductitsupports,readthegeneralinformationinAppendixB,“Notices,”onpage41.

FirstEdition(September2006)

ThiseditionappliestotheBladeCenterQS20andtoallsubsequentpreleasesandmodificationsuntilotherwise indicatedinneweditions.

©CopyrightInternationalBusinessMachinesCorporation2006.Allrightsreserved.

USGovernmentUsersRestrictedRights–Use,duplicationordisclosurerestrictedbyGSAADPScheduleContract withIBMCorp.

(5)

Chapter

1.

Introduction

TheSoftwareDevelopmentToolkit (SDK)for theCellBroadbandEngine(CellBE) isa completepackageoftoolstoallowdevelopers firsthandprogramming

experience withtheCellBEProcessor.TheSDKiscomposedof developmenttool chains, softwarelibraries andsamplesourcefiles,asystemsimulator,anda Linux kernelthatfullysupportthecapabilitiesof theCellBE.

ThecompleteSDKisavailableasa .isoimage,orasindividualcomponents. Alternativelyyoucandownloadindividual SDKcomponents.Thepackages are availablefromtheIBMEserveralphaWorksWebsitehttp://

www.alphaworks.ibm.com/tech/cellswandtheBarcelonaSupercomputingCenter (BSC)Websitehttp://www.bsc.es/projects/deepcomputing/linuxoncell/.Allofthe filesintheSDKaredistributedasRPMpackagesexceptfortheSDKinstallscript (cellsdk).

ThealphaWorksWebsite hoststheIBMsoftwareincludingtheCellBEFullSystem Simulator,XLC/C++compiler,librariesandsamples,SynergisticProcessor Unit (SPU)timingtool,andthecellsdkscript.

TheBSCWebsite hoststheopen source-derivedpackages suchasthe

GCC-basedtool chainfor theCellBE,theLinuxkernelandsupportlibraries,and system supportfortheCellBEFullSystemSimulator.

About

this

book

Each sectioncoversadifferenttopic:

v Chapter2, “InstallingLinuxFedoraCore5,” onpage 5explainshowto installthe

FedoraCore 5Linuxoperatingsystem onthesupportedhardware platforms

v

Chapter3, “InstallingtheSDK,”onpage17explainshowto installthe

BladeCenterQS20SDK

v Chapter4, “ContentsoftheSDK,”onpage 23describeseach ofthecomponents

oftheSDKinmoredetailandwhatisneededforeachof thesupported platformsandconfigurations

v Chapter5, “UsingtheSDK,”onpage 31describessomeadditionalfeaturesand

bestpracticesfordeveloping applicationusingtheSDK

In addition,there isa programmingtutorialprovidedontheSDKCDwhichgivesan introduction towritingapplicationsfor theCellBEplatform.

New

in

this

release

SDK1.1containsanumberofsignificantenhancementsovertheCellBESDK1.0 and1.0.1 andcompletelyreplacestheseSDKsversions.Theseenhancements include:

v Linuxkernel(2.6.16)andlibrarysupportfor BladeCenterQS20sthatcontaintwo

CellBEProcessorsgivinga totalof 16SynergisticProcessor Elements(SPEs)

v Supportfor PowerPC64-bithardware suchasApple PowerMacG5andIBM

PowerPCasdevelopmentplatforms

v AddedC++supportto theXLCcompilerforPowerPCProcessingUnit(PPU)

applications

v AddedsupportforGnu Debugger(GDB) serverrunninginbothPowerPC

(8)

v UpgradedGNUCompilerCollection(GCC) compilerfor PPUandSPUprograms

toversion 4.0.2

v Upgradedbinutilsto version2.16.1

v Additionsandupdates tothelibrariesandsamplesincludinganew samplethat

raytracesthequaternionJuliaSet

v Addedsupportfor Non-UniformMemoryAccess(NUMA)toimprovethe

performanceofmemoryaccesses betweenSPEs

v

Improvedinstallationbyusinganewprocess andRPMs

SDK1.1hasbeenrefreshed tocoincidewiththegeneralavailabilityof the

BladeCenter QS20.ThisrefreshcontainsBladeCenterQS20 documentationonthe ISOimageandanupdatedIBMFull-SystemSimulatorthatrunsfaster on64-bit platformsandcansimulateaSMPconfigurationwithdual CellBEprocessors as areused intheBladeCenterQS20.

Supported

platforms

CellBEapplicationscanbedevelopedonthefollowing platforms:

v x86

v

x86-64

v 64-bitPowerPC (PPC64)

v BladeCenterQS20

How

to

use

the

SDK

TheSDKincludesbothPPUandSPUcompilersfor allthesupportedplatforms.A CellBEapplicationcanberuneithernativelyonaBladeCenterQS20 orintheIBM Full-SystemSimulator (simulator),which issupportedonallofthehostplatforms. ThesimulatorontheBladeCenter QS20isusefulfor debuggingorverifyinga problem. Forexample,itispossibletobuildanapplicationonanx86 system,test under thesimulatoronthatsystem,andthenlaterrunthesamebinarynativelyon a BladeCenterQS20.

Prerequisites

TheCellBESDKrunsinFedora Core5, whichmustbeinstalledbeforeyouinstall theSDK,seeChapter2,“InstallingLinuxFedoraCore 5,”onpage5for details.

Note: ThisSDKmayworkonotherLinuxdistributions,butit hasonlybeentested

andcertifiedto runonFedoraCore 5.AkeyrequirementforanyLinux installationisglibcVersion2.4ornewer.

Table1 showstherecommendedminimumconfigurationoneachplatform.

Table1.Configurations

System Recommendedminimumconfiguration

x86 2GHzPentium4processor

PowerPC 64-bitPPCwithaclockspeedof1.42GHz. 32-bitPPCplatformsarenotsupported. BladeCenterQS20 Mustbeatrevision31orgreaterandhavea

minimumfirmwarelevelofFW6.14.7orlater (see“Checkingthefirmwareversion”on page13)

(9)

Allsystemsmusthave:

v Harddiskspace:5GB(minimum)toinstallthesourcepackageandthe

accompanyingdevelopmenttools

v 1GBRAM(minimum)onthehostsystem

Note: ThesimulatorrequiresthattheminimumamountofRAM mustbetwicethe

amountofsimulatedmemory.For example,tosimulateasystemwith512 MBof RAM,thehostsystem musthave atleast1GBofRAM.

Licenses

Thesourcecode andbinaries thatarepartof thetotal SDKpackagearedistributed withdifferentlicenses.ThepackagesontheBSCWebsite aregenerallyopen sourceandusetheGeneralPubliclicense(GPL)(http://www.gnu.org/copyleft/ gpl.html) orLesserGeneralPublic(LGP)(http://www.gnu.org/licenses/

licenses.html#LGPL)license.If youarenotalreadyfamiliarwitheither,visittheFree SoftwareFoundation(FSF) formoreinformation.

TheCommonPublicLicense(CPL),isused forprojectslike Eclipseandisthe licensefortheSDKlibrariesandsamplespackage.ItisthesuccessortotheIBM PublicLicense(IPL)andyoucanfinda FAQ(http://www-128.ibm.com/

developerworks/library/os-cplfaq.html)about theCPLlicenseontheIBM

developerWorks.TheCellFullSystemSimulatorandSPUTiming toolarelicensed under theIBMInternationalLicenseAgreementfor EarlyReleaseofPrograms (ILAR). TheXLC/C++compilerforCellProcessorislicensedundera modified ILAR.

Getting

support

TheBladeCenterQS20 SDKissupportedthroughtheCellBEarchitectureforumon thedeveloperWorksWebsiteathttp://www.ibm.com/developerworks/power/cell/. There isalso supportfortheIBMCellFull-SystemSimulatorandXLC/C++ CompilerthroughtheirindividualalphaWorksforums.Ifindoubt,startwiththeCell architecture forum.

As mentionedearlierthis versionoftheCellBESDK1.1supersedes allprevious versionsof theSDK.

Note: TheCellBESDKisprovidedonan“as-is”basis.Whereverpossible,

workaroundstoproblemswillbeprovidedintherespectiveforums.

There isa setoftutorialandreferencedocumentationfor theCellBEstoredinthe IBMonlinetechnicallibraryhttp://www.ibm.com/chips/techlib/techlib.nsf/products/ Cell_Broadband_Engine. Thefollowing documentationisavailable:

Cell

BE

processor

v CellBroadbandEngineArchitecture

v CellBroadbandEngineProgrammingHandbook

v CellBroadbandEngineRegisters

v SPUC/C++LanguageExtensions

(10)

v SPUApplicationBinary InterfaceSpecification

v AssemblyLanguageSpecification

v CellBroadbandEngineLinuxReferenceImplementationApplicationBinary

InterfaceSpecification

Cell

BE

programming

using

the

SDK

Afteryouhave installedtheSDK,youcanfind thefollowing PDFsinthe

/opt/IBM/cell-sdk-1.1/docsdirectory:

v CellBroadbandEngineProgrammingTutorialdocumentation

v SPERuntimeManagementlibrarydocumentation

v SDKSampleLibrarydocumentation

v IDLcompilerdocumentation

IBM

Full-System

Simulator

ThefollowingPDFs canbefoundinthe/opt/IBM/systemsim-cell/docdirectory afterinstalling theSDK:

v IBMFull-SystemSimulator UsersGuide

v IBMFull-SystemSimulator CommandReference

v

PerformanceAnalysiswiththeIBMFull-SystemSimulator

v IBMFull-SystemSimulator BogusNetHowTo

PowerPC

base

Thefollowingdocuments canbefound onthedeveloperWorksWebsiteat http://www.ibm.com/developerworks/eserver/library:

v PowerPCArchitectureBook, Version2.02

– BookI:PowerPCUserInstructionSetArchitecture

– BookII:PowerPCVirtual EnvironmentArchitecture

– BookIII:PowerPC OperatingEnvironmentArchitecture

v PowerPCMicroprocessorFamily: Vector/SIMDMultimediaExtensionTechnology

ProgrammingEnvironmentsManualVersion2.06c

(11)

Chapter

2.

Installing

Linux

Fedora

Core

5

Before youinstalltheCellBESDK,youmust installFedoraCore 5. This sectiondescribeshow to:

v DownloadFedora Core5

v InstallFedoraCore 5onanx86, aPowerPCora BladeCenterQS20

v Installprerequisitepackages

Downloading

the

Fedora

Core

5

installation

packages

You downloadRedHatFedora Core5 installationpackagesfromthedownloadsite, http://fedora.redhat.com.Downloadsareavailableas.isoimageswhich canbe used tocreateCDsorDVDs,or asanunpackedinstallationtree.TheFedora Core site alsohastheinstallationinstructionsavailablefor download.

Note: Youshouldallowapproximately threehoursforinstallationandconfiguration.

Useyourbrowseror FTPtodownloadthe.isoimages. Ifyouwishtodownloadto anexistingLinuxserverorworkstation,youcanuse wgetto downloadthe

unpackedinstallationtree.Todothis, issuethefollowingcommand: wget -r http://download.fedora.redhat.com/pub/fedora/linux/core/5/ppc/os/

Note: Dependingonyourenvironmentthesimplestwayisto downloadthe*.iso

imagesandthenunpackthem onyourHTTP/FTPserverratherthan downloadingtheunpackedinstallationtree.Youdonotneedtherescue image.

Copying

the

installation

files

If youarealready runningLinux,youcan savethe.isoimagesto adirectory on your serveroranotherserveronthenetwork.You canthenmounttheimageand copythefilesto adirectory whichthenbecomestheinstallationsourcedirectory. Alternatively,youcanmakeCDsorDVDsandmounttheminyourCDor DVD drive.

1. Createtheinstallationdirectoryfromisoimages.

Theinstallationfilescan becopiedfromthe.isoimagesto anydirectory accessibleoverthenetwork.Thefollowingexampleshowshow tomountthe .iso imagesandcopythefilestoa directorynamed/srv/repos/p/FC5. a. Createdirectory foreachCD:

mkdir /mnt/cd1 mkdir /mnt/cd2 b. MounttheCDs:

cd /srv/repos/p

mount -o loop FC5/FC5-ppc-disc1.iso /mnt/cd1 mount -o loop FC5/FC5-ppc-disc2.iso /mnt/cd2 c. Copyeach CD: cp -rv /mnt/cd1/* /srv/repos/p/FC5 cp -rv /mnt/cd2/* /srv/repos/p/FC5 d. Cleanup: umount /mnt/cd1 umount /mnt/cd2

(12)

2. Downloadall.isoimagesandburnthem onCDsor DVDs(thiscanbedoneon

a differentsystem)

a. Makeadirectory wheretheCDorDVDimageswillbecopied

b. Copytheimagesto thatdirectoryfromtheCDor DVD.

Installing

Fedora

Core

5

on

an

x86

or

PPC64

server

If youareusinganx86orPPC64 hostserver,youcaninstallFedora Core5from theCDsor DVDandgoto“FinishingtheFedoraCore 5installation”onpage 14.

Installing

Fedora

Core

5

on

a

BladeCenter

QS20

This sectiondescribeshow toinstallFedora Core5for PPC64onaBladeCenter QS20.

Notethefollowing:

v FedoraCore 5asksyouattheendof theinstallationprocessto rebootthe

BladeCenterQS20.Atthispoint DONOTREBOOTtheblade,DONOTCLICK therebootbuttonotherwiseyouwillhaveanon-workingLinux®onyour

BladeCenterQS20andyouwillhave torestarttheinstallation.

v Thekernelshipped withFedora Core5 doesnotsupportthefull CellBE-based

bladefunctionality.Theeasiestwayto gettheupdatedkernelistoinstalltheCell BESDKasdescribedinChapter3, “InstallingtheSDK,”onpage17.

v Youmust downloadanupdatedppc64bcmfix.img fromhttp://www.bsc.es/projects/

deepcomputing/linuxoncell/to yourinstallserver.Theoriginalppc64.imgdoes not supportthenetworkhardwareof thishardwarerevisionandisthereforenot usablefora networkinstallation.

Fedora

Core

5

installation

overview

Theoutlinefor theinstallationisasfollows: 1. Set upa netbootenvironment.

2. Set upa netinstallenvironment.

3. Perform amanualinstallation.

TheFedora Core5installationprocessstartsbybootingakernelwiththeinstall

initrdfromthenetworkdevice (thisistheonlysupportedinstallationmethodona BladeCenter QS20).Theinitprocess/sbin/initstarts/sbin/loaderpromptsyou for theinstallationlanguageandinstallationmethod.Foranetwork installation,the loaderalsoconfigures thenetwork andqueriestheparameter fortheinstallserver, before itdownloadsthesecondarystageimageFedora/base/stage2.imgfromthe install server.Aftermountingthediskimage,loaderpassescontrolto thePython scriptanaconda.

TheAnaconda installeristhemaininstallationprogramforFedora Core5and performstheremainingstepsoftheinstallation,eithermanuallythrough

configurationscreens orautomaticallyusingthekickstartconfigurationfile.This includesdownloadingallRPMs,whichareselectedforinstallationfromtheinstall server.

(13)

The

network

installation

environment

Notes:

1. TheIPaddressesusedintheexamplesbelowareforillustrativepurposesonly.

UseIPaddressesallocatedtoyournetwork.

2. Duringinstallation,youarepromptedfor thedirectory containingtheFedora

Core 5installationfiles.Theillustrationsbelowshow/fedoratreeasthesource containing theinstallationfiles.Changethisto thepathcontainingyour

installationfiles.

For theremainderofthis document,itisassumedthatyouhavethefollowing environment:

v ABladeCenterQS20(10.32.5.11).Thisistheinstallationtarget.

v ADHCP/BOOTPserver(10.32.0.1).

v Aninstallserver(10.32.0.1)runningaTFTPserver,withtheinstallationsource.

Thisservermustalso beableto runFedoraCore5 iftheinstallationmaterial requiresmodification.

v AnHTTPorFTPserver(10.64.0.31)withtheinstallationsource.

TheHTTP/FTPservercanresideonthesameserverastheDHCP/BOOTP/TFTP server.

Figure1shows atypicalnetwork installationenvironment.

blade 10.32.5.11 dchp/bootp 10.32.0.1 http 10.64.0.31

(14)

Setting

up

a

netboot

environment

You mustdownloadanupdatedppc64bcmfix.imgfromhttp://www.bsc.es/projects/ deepcomputing/linuxoncell/toyour installserver.Theoriginalppc64.img doesnot supportthenetwork hardwareofthis hardwarerevisionandisthereforenotusable for anetwork installation.

Copythenew ppc64bmcfix.imgfiletothe/tftpboot directoryoftheTFTP/BOOTP serverandmakesurethatitmatchestherespective entryin/etc/dhcpd.conf.

Note: Rebuildingtheppc64bmcfix.imgispossiblebutoutsidethescopeof this

document.

Setting

up

a

network

installation

environment

Fedora Core5 isinstalledoverthenetwork usingTFTP.Generally,aTFTPserveris used astheinstall server.

Installing

Fedora

Core

5

This sectiondescribeshow toinstallFedora Core5.

Starting

the

installation

ToinstallFedora Core5, dothefollowing:

1. Eitherinserta newharddiskintotheBladeCenterQS20,oroverwritean

existingharddiskwithanexistingLinux.

2. ConnecttheBladeCenterQS20 toaserialconsole(115200,N,1,8,no

handshake)andbootitupto thefirmware prompt. 3. Tostarttheinstallation,enterthefollowing:

> netboot vnc console=hvc0

4. Selectthelanguageyouwouldliketo usefortheinstallation.

Welcome to Fedora Core

+---+ Choose a Language +---+

| |

| What language would you like to use |

| during the installation process? |

(15)

5. Selectthemediatypethatcontainstheinstallationpackages.

+---+ Installation Method +---+

| |

| What type of media contains the |

+--- Networking Device ---+

| |

| You have multiple network devices on |

| this system. Which would you like to |

| install through? |

| |

| > eth0 - Unknown device 102f:01b3 |

| eth1 - Unknown device 102f:01b3 |

| | | +----+ +---+ | | | OK | | Back | | | +----+ +---+ | | | | | +---+

7. EntertheIPconfigurationforyourserver.SelectUsedynamicIP

configuration(BOOTP/DHCP).Fedora Core5determinesthehostname and

domainfromthedhcp/bootp server.

+---+ Configure TCP/IP +---+

| |

| Please enter the IP configuration for this machine. Each |

| item should be entered as an IP address in dotted-decimal |

| notation (for example, 1.2.3.4). |

| |

| [*] Use dynamic IP configuration (BOOTP/DHCP) |

| |

| IP address: _______ |

| Netmask: _______ |

| Default gateway (IP): _______ |

| Primary nameserver: _______ | | | | +----+ +---+ | | | OK | | Back | | | +----+ +---+ | | | | | +---+

(16)

8. Enterthename theFTPsite nameandthedirectorythatcontainstheFedora

Core:

+---+ HTTP Setup +---+

| |

| Please enter the following information: |

| |

| o the name or IP number of your FTP server |

| o the directory on that server containing |

| Fedora Core for your architecture |

| |

| FTP site name: 10.32.0.1_______________ |

| |

| Fedora core directory: /fedoratree____________|

| | | [*] Use non-anonymous ftp | | | | +----+ +---+ | | | OK | | Back | | | +----+ +---+ | | | | | +---+ 9. EnteranFTPaccount nameandpassword.

+--- Further FTP Setup ---+

| |

| If you are using non anonymous ftp, enter the |

| account name and password you wish to use below. |

| |

| Account name: userid__________________ |

| Password: password_________________ | | | | +----+ +---+ | | | OK | | Back | | | +----+ +---+ | | | | | +---+ 10. Thefollowingisdisplayed:

+---+ | Running anaconda, the Fedora Core system installer - please wait... | | Framebuffer ioctl failed. Exiting. |

| Probing for video card: Unable to probe |

| Probing for monitor type: Unknown monitor |

| Probing for mouse type: No – mouse |

| No video hardware found, assuming headless |

| Starting VNC... |

| |

| WARNING!!! VNC server running with NO PASSWORD! |

| You can use the vncpassword=<password> boot option |

| if you would like to secure the server. |

| |

| The VNC server is now running. |

| Please connect to 10.32.5.11:1 to begin the install... |

| |

| Press <enter> for a shell |

| Starting graphical installation... |

+---+ 11. Starta VNCsessiononanothercomputer inthenetwork.Atthecommand

promptof that computer entervncviewer <target IP>, where<target IP> is theaddress oftheBladeCenterQS20beinginstalled,forexample,

10.32.5.11:1.Continuetheinstallprocessfromthecomputerrunningthe vncviewersession,nottheBladeCenterQS20wheretheinstallisactually takingplace.

(17)

Note: DoNOTclosetheserialconsoleonthebladeserver.Thissessionis

stillneededattheendoftheinstallationprocess.

12. Continuetheinstallation,completing eachsectionasrequired.Takenoteofthe

following:

Whenyoupartitiontheharddisk,selectCreateCustomLayout.Figure2is anexampleofwhatacustomlayoutshouldlook like.

v Fedora Core5isnotableto bootlargerpartitionscrossingcertainsector

boundaries,this meansthatyou needasmallbootpartitionof approximately 100MB

v Thepartitioncontainingtheyaboot.confmust bereadable atboottime (and

thereforecannotbeanLVMpartitionbutmustbeext3).

v IfFedora Core5 asksyouat theendoftheinstallationtoreboottheblade,

DO NOTREBOOTtheBladeCenterQS20andDONOTCLICKthereboot buttonotherwiseyouwillhaveanon-functionalLinuxonyourBladeCenter QS20. Dependingonyourfirmware,orhardware revisions,orboth, you needto performadditionalsteps.

Copying

a

network-enabled

kernel

version

TheunpatchedkernelfromFedoraCore 5doesnotsupportthenetworkhardware andthereforeleavesa bootablebladewithnonetworksupport.

(18)

Tocorrectthis,you MUSTeither:

v Copythenetworkmodulesfromtheinstallimageto theserverharddisk

or

v CopyandinstallthenewkerneltothebladeBEFOREyoureboot the

BladeCenterQS20.

cp *.ko /mnt/sysimage/lib/modules/2.6.15-1.2054_FC5/kernel/drivers/net/ or usetheserialconsoletodownloadthenew kernelRPM totheharddisk:

1. DownloadtheBladeCenterQS20RPMsfromtheBSCWebsiteif youhavenot

alreadydone so(forexample,viayourHTTP/FTPserver). 2. CopytheBladeCenterQS20 RPMsto theblade(for example:scp

server:/...kernel-2.6.15.4-bsc3.0.ppc64.rpm /...)

3. Perform thefollowing commandto installtheRPMs:Forexample:

ftp 10.32.0.1 cd <directory> get <kernel_name>

mv <kernel_name> /mnt/sysimage/root

where<directory>isthenameof thedirectorycontainingtheRPMs.

Additional

installation

steps

for

installed

InfiniBand

cards

If youhaveInfiniBandcardsinstalled,you mustissuethefollowingcommand: echo blacklist ib_mthca >> /mnt/sysimage/etc/modprobe.conf

Afteryouhave installedtheFedora Core5kernelortheSDK1.1youmayneedto upgradethekernel.AkernelupdateisavailablethatfixesanissuewithInfiniBand andthesecondGigabit Ethernetinterface andisintendedonlyif youwishtouse both. Seehttp://www.bsc.es/projects/deepcomputing/linuxoncell/stable/

linuxkernel.htmlfor moreinformation.

Rebooting

the

BladeCenter

QS20

Reboot thesystem fromtheinstallationscreen.

Checking

and

adapting

/etc/yaboot.conf

If theinstallationhascompletedsuccessfully,yaboot.confshows anentry forthe Fedora Core5 kernelandanentryfor thepatched kernel.Checkthatthepatched entry iscorrectandeditthelabelvariableto somethingmeaningful.

Thefollowingisa sampleyaboot.conffile: # yaboot.conf generated by anaconda

boot=/dev/hde1

init-message=Welcome to Fedora Core!\nHit <TAB> for boot options

partition=2 timeout=20 install=/usr/lib/yaboot/yaboot delay=5 enablecdboot enableofboot enablenetboot nonvram fstype=raw image=/vmlinuz-2.6.16-1.17.driver.be0613 label=cell

<< change default name to user friendly name

read-only

initrd=/initrd-2.6.16-1.17.driver.be0613.img

append="console=hvc0 rhgb quiet root=LABEL=/"

(19)

image=/vmlinuz-2.6.15-1.1826.2.10_FC5

label=linux

read-only

initrd=/initrd-2.6.15-1.1826.2.10_FC5.img

append="console=hvc0 rhgb quiet root=LABEL=/"

ppc64bcmfix

Checking

and

setting

up

a

swap

area

(if

required)

If required,youcansetupaswapareaasfollows:

free << check if swap space is available

swapon -a << enable devices and files for swapping

mkswap /dev/VolGroup00/LogVol01 << set up a Linux swap area

Configuring

yum

(if

required)

If required,configurethe/etc/yum.conffilesothatit pointstotheHTTPserver. You mustchangethebaseurlentry:

[main] cachedir=/var/cache/yum debuglevel=2 logfile=/var/log/yum.log pkgpolicy=newest distroverpkg=redhat-release tolerant=1 exactarch=1 retries=20 obsoletes=1 gpgcheck=0

# PUT YOUR REPOS HERE OR IN separate files named file.repo # in /etc/yum.repos.d

/etc/yum.repos.d/fedora-core.repo

[base]

name=Fedora Core $releasever - $basearch - Base

#baseurl=http://download.fedora.redhat.com/pub/fedora/linux/core/$releasever/$basearch/os/ baseurl=http://10.64.0.31/ <<<< modify baseurl here

mirrorlist=http://fedora.redhat.com/download/mirrors/fedora-core-$releasever enabled=1

gpgcheck=0

gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora

Managing

a

BladeCenter

QS20

This sectiondescribeshow tocheckthefirmwareversion, boot,shutdown,and restart aBladeCenterQS20.

Checking

the

firmware

version

Tocheckthefirmwareversion, dothefollowing: 1. Access theBladeCenter ManagementModule.

2. SelectMonitors→ FirmwareVPD.TheBladeServerFirmwareVPDwindow

containsthebuildidentifier,release, andrevision.

Booting

a

BladeCenter

QS20

Toboota BladeCenterQS20,dothefollowing: 1. OpentheBladeCenterManagementModule.

2. Set theappropriatebootdevice (network,harddisk)fortheBladeCenterQS20

byselectingBladeTasks→ Configuration→BootSequence.

3. Power ontheBladeCenterQS20 byselectingBladeTasks →Power/Restart→

(20)

Note:

v Thebootprocess oftheBladeCenterQS20can onlybemonitoredwitha

serialconsole(115200,N,1,8,nohandshake)connected tothe BladeCenter QS20'sserialportonthefrontbezel

v You mayneeda null-modemcable

v Toforce theBlade intotheOpenFirmwarePrompt presss ontheserial

consoleduringtheearly stagesofthebootprocess

Shutting

down

and

restarting

the

BladeCenter

QS20

Always shutdownandrestarta bladethathasbeenbootedto theLinuxpromptwith oneof thefollowingcommandsfromaLinuxshellontheBladeCenterQS20: shutdown -g0 -i0 -y

halt reboot

shutdown -r now

Do notusetheBlade CenterManagement Moduletopowerdownorrestartthe Blade (usingBladeTasks →Power/Restart→checkmarktheblade→PowerOff Blade/RestartBlade)asyou cancorruptthefilesystem:theBlade Centerwill poweroff theBladeCenter QS20withoutfirstnotifyingtheoperatingsystem.

Finishing

the

Fedora

Core

5

installation

At thispointyoushouldhave FedoraCore 5installedandrebootedonyour machine,whichiseither anx86, aPowerPCora BladeCenterQS20.It is

recommendedthatyoudoayumupdateregularlyto keepuptodate withthelatest Fedora Core5 changes.In addition,thereareuptosevenotherdependenciesthat need tobeinstalledbeforeyouinstalltheCellBESDK.

Note: AdefaultFedoraCore 5installdoes nothave alltherequiredpackages.

gcc

TheregularFedoraCore 5gccGNUcompilerisusedto buildtwo samplesthatare runonthehost machineratherthanonaCellBEProcessor.Toverifyif itis

installed, runrpm –qgcc.The resultshouldbesimilar to: gcc-4.1.1-1.fc5

To getthegcc packageand itsdependencies, youmust doa yuminstall gcc.

make

Makeisusedto buildthelibrariesandsample programs.Toverifyif itisinstalled, run rpm -q make.Theresultshould besimilar to:

make-3.80-10.2

To getthemakepackage,you must doa yum installmake.

perl

Perlisrequiredbyseveralparts oftheSDK.Toverifyif itisinstalled,runrpm –q perl.Theresultshouldbesimilar to:

perl-5.8.8-5

To gettheperl package,you mustrun yum installperl.

(21)

freeglut

Two ofthesampleprograms dependonOpenGLandrequireglut.hto compile successfully.Toverify ifit isinstalled, runrpm –qfreeglut-devel.Theresultshould besimilarto:

freeglut-devel-2.4.0-4

To getthefreeglut-develpackage, youmust runyum install freeglut-devel.

libX11

Toexecute theapplicationsthatdependonOpenGL,variousX11libraries are needed.Togettherequiredpackages, run:

yum install libX11 yum install libXau yum install libXdmcp

TK

The simulatorrequireslibtk8.4.so. Toverifyif itis installed,runrpm –qtk.The resultshouldbesimilarto:

tk-8.4.13-1.1

If thetkpackageisnotinstalledyouseethefollowingmessagewhenyourunthe full systemsimulator:

systemsim-cell/run/cell/../../bin/systemsim-cell: error while loading

shared libraries: libtk8.4.so: cannot open shared object file: No such file or directory. To getthetk(and tcl)package,run yum installtk.

netpbm

TheJuliasetsampleneedsthenetpbmandnetpbm-develpackagesontheCell BE-based bladeserver.To verifyif installed doa rpm –qa | grep netpbm.The result should besimilarto:

netpbm-devel-10.33-1.fc5 netpbm-10.33-1.fc5

(22)

(23)

Chapter

3.

Installing

the

SDK

This sectionassumesthatyouhaveinstalledFedora Core5onyourhostsystem andarenowready toinstall theCellBESDK.Thefollowing Websitesarerelevant:

v

http://www.alphaworks.ibm.com/tech/cellsw

TheIBMalphaWorkssitecontainsemergingtechnologies.In particularit contains theIBMSource LibrariesandSamples,theFull-SystemSimulatorandtheCell BEXLC/C++compiler.

v http://www.bsc.es/projects/deepcomputing/linuxoncell/cbexdev.html

TheLinuxonCellBE-basedsystemsweb siteat theBarcelona Supercomputing Center(BSC)providesinformationabouthow toenableLinuxonCellBE platforms.Ithostsalloftheopensourcepatchesandbinaries neededtoinstall SDK1.1.

v http://www.ibm.com/developerworks/power/cell/

TheIBMdeveloperWorkssitecontainstechnicalarticles andsupportforumsfor CellBE.

SDK

components

TheCellBESDKconsistsofasetof RPMfilesandascripttoinstallthe

appropriateRPMsforthetargetplatform.Thecomponentsareexplainedinmore detailinChapter4, “ContentsoftheSDK,”onpage 23.Thetablebelowshows combinations ofplatformsandcomponents:

Table2.Supportedcombinationsofplatformsandcomponents

Component x86/x86-64 PPC64 BladeCenterQS20

GNUToolchain Mandatory Mandatory Mandatory XLC/C++Compiler Optional Notapplicable Notapplicable LinuxKernel Built-intosystemroot

image

Built-intosystemroot image

Mandatory

Linuxsupport libraries

Built-intosystemroot image

Mandatory Full-SystemSimulator Mandatory Mandatory Optional SystemRootImage

forSimulator

Mandatory Mandatory Optional

Librariesand Samples

Mandatory Mandatory Mandatory

Some componentsconsistofoneormoreRPM filesandthedetailedlist ofRPMs for eachcomponentandplatformisdescribedin“RPMsinSDK”onpage 21.

Simple

install

of

the

SDK

ThesimplestwaytoinstalltheSDKistodownloadtheCellBESDKISOimage fromtheIBMalphaWorksWebsitehttp://www.alphaworks.ibm.com/tech/cellsw.The ISOimagecontainsalloftheIBMlicensedsoftwareandalsoallof CellBEand SDKtechnicaldocumentationinAdobePDFformat.Youcanalsocreateaphysical CDfromthisISOimageandreaditonnon-Linuxsystems.

(24)

Thecellsdk shellscripthandles theSDKinstallation.TheSDK’stoolsareprimarily installedin/opt/IBM,althoughsomeofthetoolchaincommandsareinstalledinthe regularpath.for example,thespu-gcc commandisinstalledin/usr/binonaPPC machineandin/opt/sceonanx86machine.Thesimulatorisinstalledin

/opt/IBM/systemsim-cell,andtheSDKin/opt/IBM/cell-sdk-1.1/.

Thescriptbeginsbyaskingyouwhetheryou havereadtheassociatedlicense agreements.You musttypeyesto continue;y isnotacceptedasconfirmation.The scriptcheckstoseewhatfeaturesmightbeneededandwhatfeaturesare

available. Ifsomeoftheotherinstallationfiles(fromtheBSCWebsite)areneeded, they areautomatically downloadedto thedirectory /tmp/cellsdk-1.1.

Do thefollowing toinstalltheSDK,andtoupgradetheSDK,see“UpgradingSDK 1.1to SDK1.1.1”:

1. As rootdownloadtheSDKISOdiskimage,CellSDK11.isofromtheCellBE

SDKalphaWorksWebsite http://www.alphaworks.ibm.com/tech/cellsw. 2. Createa mountdirectory andmakesurenothingelse ismounted onthis

directory:

mkdir –p /mnt/cellsdk

3. Mountthediskimageonthemountdirectory:

mount –o loop CellSDK11.iso /mnt/cellsdk 4. Change directoryto /mnt/cellsdk/software:

cd /mnt/cellsdk/software

5. Install theSDKbyusingthefollowing commandandanswerany prompts:

./cellsdk install

6. Change directoryto anydirectorywhichisnotthemountdirectory orbelowit:

cd /

7. Unmountthediskimage:

umount /mnt/cellsdk

If youhavepreviously installedSDK1.1,downloadthelatestSDKfromalphaWorks anduse thefollowing commandto upgradeyoursystemto theSDK1.1.1.

./cellsdk upgrade

During theinstall processyoumightbeaskedquestionsabout whichcomponents to installorreinstall.RefertoTable2onpage17tounderstandwhatthemandatory andoptional componentsare.As partof theinstallproceduretheSDKlibrariesand samplesarecompiledandlinkedbydefaultusingtheGNUtools.See“Otherways to installtheSDK”onpage 19for othercellsdktasksandoptions thatallowyouto compilethelibrariesandsampleswiththeXLCcompileror installtheSDKwithout prompts.

You canrunthecellsdkscriptmultipletimestoinstall orreinstallcomponents.The install processalso buildsallofthelibrariesandsampleswithoutstoppingifthereis anerror.Usethefollowing commandcanbeusedto keepa logofwhatwasdone duringtheinstallprocessincasethereisaproblem:

./cellsdk install 2>&1 | tee /tmp/cellsdk-install.log

Upgrading

SDK

1.1

to

SDK

1.1.1

If youhavepreviously installedSDK1.1,downloadthelatestSDKfromalphaWorks anduse thefollowing commandto upgradeyoursystemto theSDK1.1.1.

./cellsdk upgrade

(25)

Other

ways

to

install

the

SDK

Theinstallapproach describedin“Simpleinstallof theSDK”onpage17maynot workinyourenvironment.Thissection describesseveralothermethodsto getthe componentsoftheSDKandinstallit.

One problemisthatthedownloadfromtheBSCWebsite maynotworkif you accesstheinternetthrougha proxy.One solutionto thisproblemistousethe

http_proxyenvironment variabletotell wgetabout theproxy.For example,if you use aproxywiththeIPaddress192.168.10.11onport8080,settheenvironment variable asfollows:

export http_proxy="http://username:[email protected]:8080"

If thisdoesnotworkforyoubecauseoffirewallsor othernetworkissue,download theRPMfilesmanually.Table3onpage21givesa completelist ofalltheRPMs neededfor agivenhostingenvironment.You canalsousethecellsdkscriptitself to determineif allofthedependentRPMshavebeenacquired.Issuethefollowing commandto seethelist ofmissingfiles:

./cellsdk depend

Here issomeexampleoutputfromthis command:

Do you plan to install the Cell Full System Simulator? (y/n):y

WARNING! The following required file(s) for SDK install were not found Missing file(s): ppu-gcc-c++-3.2-4.ppc.rpm ppu-gcc-3.2-4.ppc.rpm ppu-binutils-3.2-4.ppc.rpm spu-newlib-3.2-6.ppc.rpm spu-gcc-c++-3.2-6.ppc.rpm spu-gcc-3.2-6.ppc.rpm spu-binutils-3.2-6.ppc.rpm spu-gdb-3.2-6.ppc.rpm Paths searched: /home/user23 /tmp/cellsdk-1.1

These files may be downloaded from:

http://www.bsc.es/projects/deepcomputing/linuxoncell/cellsimulator/sdk1.1/

Would you like to attempt automatically downloading these files to /tmp/cellsdk-1.1? (y/n):n

ERROR: Missing SDK dependencies

Uninstalling

the

SDK

If youneedto uninstalltheSDKforwhateverreasonthiscanbedoneasrootby executingthescriptasfollows:

./cellsdk uninstall

Note: Toensurethatthesystem isleftinausablestate,thescriptdoesnot

uninstallthekernelonBladeCenterQS20s.Youmustchange theyaboot configurationfile(/etc/yaboot.conf)to loadadifferentkernelatthenext rebootandusethefollowingcommandto uninstallthekernel:

(26)

rpm –e kernel-2.6.16-bsc4.2.

Other

script

tasks

and

options

TheSDKscriptcellsdkimplementsanumberoftasksandoptionsfor thosetasks. If yourunthescriptwithoutany parameters,it displaystheusagestatementas follows:

Task ’install’ and ’depend’ options:

-f, --force assume ’yes’ to all questions.

-x, --xlc use XLC to compile the libraries and samples (x86 only) -g, --gcc use GCC to compile the libraries and samples (default)

--nosim do not install the IBM Cell BE Full System Simulator

--nokernel do not install the Cell BE kernel (CBE hardware only)

Task ’uninstall’ options:

-f, --force

assume ’yes’ to all questions.

Task ’build’ options:

-x, --xlc

use XLC to compile the libraries and samples (x86 only)

-g, --gcc

use GCC to compile the libraries and samples (default)

Thescriptunderstandsthefollowingtasks:

Task Description

install InstalltheSDKasexplainedin“SimpleinstalloftheSDK”on page17

uninstall UninstalltheSDKasexplained“UninstallingtheSDK”onpage19 depend DetermineifalloftherequiredRPMsareavailableforaninstall

asexplained“OtherwaystoinstalltheSDK”onpage19

build Buildorrebuildthelibrariesandsamples.

Note: Amakecleanisalsodonebeforethebuild.

synch Synchronizesbetweenthe/opt/IBM/cell-sdk-1.1/sysroot

directoryandthesystemrootimagefortheFullSystemSimulator. See“Systemrootimageforthesimulator”onpage25formore information.

upgrade OnlyavailablefortheSDK1.1RefreshtoupgradeSDK1.1tothe SDK1.1Refresh.

verify Determinesifthecorrectcomponentversionshavebeeninstalled.

version PrintstheSDKversionstring:

IBM Cell Broadband Engine SDK 1.1 (build 20060705) ORfortheSDK1.1refresh:

IBM Cell Broadband Engine SDK 1.1.1 (build 20060918)

Thescriptalsosupports thefollowingfiveflagswhichareignoredif notspecifiedfor thetask:

Flag Description

-for--force Forcesaninstalloruninstallwithminimalprompting.

-gor--gcc SelectstheGNU/GCCtoolchainonanx86machineforcompiling thelibrariesandsamples.

(27)

Flag Description

--nokernel DonotinstallthekernelontheBladeCenterQS20.Bydefault installingthekernelputsitfirstintheyabootconfigurationfileand itisusedonthenextrebootoftheserver.Inmostcasesyou shouldusethiskernelasithasfullsupportfortheBladeCenter QS20.

--nosim Donotinstallthesimulator.Thisoptiononlyappliestoa

BladeCenterQS20whereasimulatorisnotrequired.Otherwiseit isrecommendedthatthesimulatorisinstalledsothatitcanbe usedfordebuggingorproblemrecreation.

-xor--xlc SelectstheXLC/C++compileronanx86machineforcompiling thelibrariesandsamples.TheGNU/GCClinkerisusedtobuild thecompleteexecutable.

RPMs

in

SDK

Thetable belowshowsacompletelistof RPMsthatcanbeinstalledfor each supportedplatform.TherowsinboldareRPMsthatyou candownloadfromIBM alphaWorksWebsitehttp://www.alphaworks.ibm.com/tech/cellsw.Youcandownload therowsthatarenotinboldfromtheBSCWebsitehttp://www.bsc.es/projects/ deepcomputing/linuxoncell/cbexdev.html.

Table3.RPMsthatcanbeinstalledforeachsupportedplatform

Componentarea x86/x86_64 PPC64 BladeCenterQS20

GNUToolChainforPPU ppu-binutils-3.2-4.i686.rpm ppu-gcc-3.2-4.i686.rpm ppu-gcc-c++-3.2-4.i686.rpm ppu-sysroot-3.2-4.noarch.rpm ppu-sysroot64-3.2-4.noarch.rpm ppu-binutils-3.2-4.ppc.rpm ppu-gcc-3.2-4.ppc.rpm ppu-gcc-c++-3.2-4.ppc.rpm

GNUToolChainforSPU spu-binutils-3.2-6.i686.rpm spu-gcc-3.2-6.i686.rpm spu-gcc-c++-3.2-65.i686.rpm spu-newlib-3.2-6.i686.rpm spu-gdb-3.2-6.i686.rpm spu-binutils-3.2-6.ppc.rpm spu-gcc-3.2-6.ppc.rpm spu-gcc-c++-3.2-6.ppc.rpm spu-newlib-3.2-6.ppc.rpm spu-gdb-3.2-6.ppc.rpm

IBMXLC/C++Compiler xlc-cell-lib-1.1-9.i386.rpm xlc-cell-cmp-1.1-9.i386.rpm xlcpp-cell-lib-1.1-9.i386.rpm xlcpp-cell-cmp-1.1-9.i386.rpm

NotSupportedforPPC64

LinuxKernel reintegratedwithinsystemrootimageforsimulator kernel-2.6.16-bsc4.2.ppc64.rpm kernel-cbesim-2.6.16-bsc4.2.noarch.rpm

LibrarySPE libspe-1.1.0-1.ppc.rpm libspe-devel-1.1.0-1.ppc.rpm elfspe-1.1.0-1.ppc.rpm libspe-1.1.0-1.ppc64.rpm libspe-devel-1.1.0-1.ppc64.rpm

NUMAControl Pre-integratedwithinsystemrootimageforsimulator numactl-0.9.8-1.cbe.ppc.rpm

OProfile Pre-integratedwithinsystemrootimageforsimulator oprofile-0.9.1-8.cbe.ppc.rpm oprofile-devel-0.9.1-8.cbe.ppc.rpm

IBMFullSystemSimulator (SDK1.1)

systemsim-cell-1.1-6.i386.rpm systemsim-cell-1.1-6.ppc.rpm

IBMFullSystemSimulator (SDK1.1Refresh)

systemsim-cell-1.1-10.i386.rpm systemsim-cell-1.1-10.x86_64.rpm

systemsim-cell-1.1-10.ppc64.rpm

Systemrootimagefor simulator

sysroot_image-1.1-011.noarch.rpm

IBMSamplesandLibraries cell-sdk-lib-samples-1.1-10.noarch.rpm

(28)

(29)

Chapter

4.

Contents

of

the

SDK

This sectiondescribesthecontentsof theSDK,whereit isinstalledonthesystem, andhow thevariouscomponentsworktogether.

GNU

tool

chain

TheGNUtool chaincontainstheGCCcompilerforthePPUandSPUimplemented bySonyComputerEntertainment.For thePPUit isacross-compileronx86anda replacement forthenativeGCCcompileronPPCplatforms.TheGCC compilerfor thePPUispreferredandthemakefilesareconfiguredtouseit whenbuildingthe libraries andsamples.

TheGCCcompileralsocontainsaseparate SPEcross-compilerthatsupportsthe standardsdefinedintheSPUC/C++ LanguageExtensionsV2.1,SPUApplication Binary InterfaceSpecificationV1.4,andSynergisticProcessorUnit(SPU)

InstructionSetArchitectureV1.0documents.

TheassociatedassemblerandlinkeradditionallysupporttheSPUAssembly LanguageSpecificationV1.3.Theassemblerandlinkerarecommontoboth the GCC andXLC/C++ compilers.GDBsupportisprovidedfor bothPPUandSPU debugging,andthedebuggerclientcanbeinthesameprocessor aremote process.

Ona non-PPCmachine,theGNUToolChainisinstalledintothe

/opt/sce/toolchain-3.2/ppuand/opt/sce/toolchain-3.2/spudirectories, containing theseparate toolchainsforthePPUandSPU,respectively.Ona PPC64 orBladeCenterQS20, bothtoolchainsareinstalledinto/usr.

Thepatches tothestandard4.0.2GNU/GCCcompilerareavailableontheBSC WebsiteandaredistributedundertheGPLlicense.

IBM

XL

C/C++

compiler

TheIBMXLC/C++compilerfor CellBEProcessor isacross-compilerfor x86 platformsonly.ThisC/C++ cross-compilergeneratescode forthePowerPC

Processor Unit(CandC++)or SynergisticProcessorUnit(Conly)andistunedfor theCellBEArchitectureThecompilerrequirestheGCCToolchainforCellBE, which providestoolsforcross-assembling andcross-linkingapplicationsforboththe PPEandSPE.

ThecompilersupportsthelatestISOC1999(International StandardISO/IEC 9899:1999)standard, alsoknownasC99.Italsosupports theC89 Standardand K&Rstyle.Tohelpportthecode thatwasoriginallywritten forGCC,a subsetof featuresrelatedto GNUCissupportedbyXLC/C++.OtherGNU compilerfeatures arerecognized(acceptedandignored).Thishelpsminimizethenumberof changes thatyoumust maketo yourGCCsourcecode toensurethatitcompiles

successfullywithXLC.

TheXLC/C++compilerprovidesthreeinvocation commands:

v ppuxlc

v ppuxlc++

v

(30)

Thecommandsppuxlcandppuxlc++areusedto generatecodefor thePPU,and

spuxlcisused togeneratecode fortheSPU.

Thecompilerinvocationcommandsfor thePPUperformallnecessarystepsfor compiling C/C++sourcefilesbyppuxlcor(C++ sourceusingppuxlc++)into .ofiles andlinking theobjectfilesandlibrariesbyppu-ldintoanexecutableprogram. Similarly,thecompilerinvocation commandfor theSPUperformsallnecessary stepsfor compilingCsourcefilesbyspuxlcinto.sfiles,assembling.sfilesinto.o

filesbyspu-as,andlinkingtheobjectfilesandlibraries intoanexecutableprogram byspu-ld.Theppu-embedsputoolthatispartof theGNU toolchainisused tolink a PPUexecutable andaSPUexecutableintoasingleCellBEexecutable.

TheXLC/C++compilerincludesfivebaseoptimizationlevels:

v -O0:almostnooptimization

v -O2:strong,low-leveloptimizationthatbenefitsmost programs

v -O3:intense,low-level optimizationanalysiswithbasicloopoptimization

v -O4:allof-O3anddetailed loopanalysisandgoodwhole-programanalysisat

linktime

v

-O5:allof-O4anddetailed whole-programanalysis atlinktime.

Auto-SIMDization isenabledatO3-qhot orO4andO5bydefaultforthePPE,and at O3-qhotor O4andO5for theSPE.SIMDisanabbreviationfor Single

InstructionMultipleData;thisisoneofthekeyperformanceboosters intheCellBE architecture.

TheXLC/C++compilerisinstalledinto the /opt/ibmcmp/xlc/1.0directoryandis distributedunder theIBMILARlicense.

Note: Thesourcecodeisnotavailable.

IBM

Full-System

Simulator

TheIBMFullSystemSimulator(simulator)isa softwareapplicationthatemulates thebehavior ofafull systemthatcontainsaCellBEProcessor.Youcanboota Linuxoperating systemonthesimulatorandrunapplicationsonthesimulated operating system.Thesimulatoralsosupportstheloadingandrunningof

statically-linked executablesandstandalonetestswithoutanunderlyingoperating system.

Thesimulatorinfrastructureisdesignedformodelingprocessor-andsystem-level architecture atlevelsof abstraction,whichvary fromfunctionaltoperformance simulation modelswithanumberofhybridfidelitypointsinbetween:

v Functional-onlysimulation:Modelstheprogram-visibleeffectsof instructions

withoutmodelingthetimeit takesto executetheseinstructions.Functional-only simulationassumesthateach instructioncanberuninaconstantnumberof cycles.Memoryaccessesaresynchronousandarealsoperformed ina constant numberofcycles.Thissimulationmodelisusefulforsoftwaredevelopmentand debuggingwhenaprecisemeasureof executiontimeisnotsignificant.

Functionalsimulationproceedsmuchmorerapidly thanperformancesimulation, andsoisalso usefulforfast-forwardingto aspecificpointof interest.

v Performancesimulation:For systemandapplicationperformanceanalysis,the

simulatorprovidesperformancesimulation(also referredto astimingsimulation.) Aperformancesimulationmodelrepresentsinternalpoliciesandmechanismsfor systemcomponents,suchasarbiters,queues, andpipelines.Operationlatencies

(31)

aremodeled dynamicallytoaccount forbothprocessing timeandresource constraints.Performancesimulationmodels havebeencorrelatedagainst hardwareor otherreferencesto acceptablelevelsof tolerance.TheFullSystem Simulatorfor CellBEProcessor providesacycle-accurate SPUcoremodelthat canbeusedforperformanceanalysis ofcomputationally-intenseapplications. However,this modelcannotbeusedfor measuringortrackingmemoryaccess latencies.RefertotheIBMdeveloperWorksSPUPipelineExamination article http://www.ibm.com/developerworks/power/library/pa-cellspu/foradditional information.

v Thesimulatorcanalso beconfiguredto fast-forwardthesimulation,usinga

functionalmodel,toa specificpoint ofinterestintheapplicationandto then switchto atiming-accuratemode toconduct performancestudies.Thenvarious typesofoperationaldetailscanbegatheredto gaininsightintothebehaviorof real-worldhardwareandsoftwaresystems.Thisoptionaffords ahighlevelof configurability:userscandynamicallytradedetailandaccuracyoftimingfor fastersimulation speed.

Theversionof thesimulatorshipped withtheCellBESDKiscompiledto runon Fedora Core5. Seethe/opt/IBM/systemsim/docssubdirectoryforcomplete documentation includingasimulatoruser’sguide.Thepre-releasenameof the simulatoris“Mambo”andthisnamemay appearinsomeof thedocumentation. ThesimulatorfortheCellBEProcessorisalsoavailableasanindependent technology athttp://www.alphaworks.ibm.com/tech/cellsystemsim.

Thesimulatorisinstalledintothe/opt/IBM/systemsim-celldirectory andis distributedunder theIBMILARlicense.

Note: Thesourcecodefor thesimulatorisnotavailable.

System

root

image

for

the

simulator

Thesystem rootimagefor thesystem simulatorisa filethatcontainsa diskimage of FedoraCore 5files,librariesandbinariesthatcan beused withinthesystem simulator.Thisdiskimagefileispreloadedwithafullrangeof FedoraCore 5 utilitiesandalso includesallof theCellBELinuxsupportlibrariesdescribedin “Linuxsupportlibraries”onpage26.ThisRPMfileisbyfarthelargestof theRPM filesandwheninstalledtakesup1.6GBonthehostserver’sharddisk.

Thesystem rootimagefor thesimulatormustexistinthesamedirectory asthe Linuxkernelwhichiseitherthecurrentdirectorywhenstartingthesimulatoror the default/opt/IBM/systemsim-cell/images/celldirectory.Thecellsdkscripttakes care ofautomaticallyputtingthesystemrootimageinthedefaultdirectory.

This systemrootimagecanbemounted toseewhatitcontains.Assumingamount point of/mnt/cell-sdk-1.1-sysroot, whichisthemountpointused bythecellsdk script,thecommandtomountthesystemrootimageis:

mount -o loop /opt/IBM/systemsim-cell/images/cell/sysroot_disk /mnt/cell-sdk-1.1-sysroot/ Thecommandtounmount theimageis:

umount /mnt/cell-sdk-1.1-sysroot/

Do notattemptto mounttheimageonthehostsystem whilethesimulatoris running.Thesystemrootimageshouldalwaysbeunmountedbefore youstartthe

(32)

simulator.Youshould notmountthesystemrootimagetothesamepoint asthe rootonthehostserverasthesystemcan becomecorruptedandfailto boot. There arethreewaysto changefilesonthesystemrootimagedisk:

v Mountit asdescribedabove.Thenchangedirectory (cd)tothemountpoint

directoryor belowandmodifythefile usinghost systemtoolssuchasviorcp.

v

Usethe./cellsdksynch commandto synchronizethesystemrootimagewith

the/opt/IBM/cell-sdk-1.1/sysrootdirectoryfor librariesandsamples(see “Systemrootdirectory”onpage34)thathave beencross-compiledandlinkedon ahostsystem andneedto becopiedto thetargetsystem.

v Usethecallthrumechanism(see“Thecallthru utility”onpage 32)to sourceor

sinkthehost systemfileduringexecutionofthesimulator.Thisisonlymethod thatcanbeusedwhilethesimulatorisrunning.

Thesourcefor thesystem rootimageisavailableontheBSCWebsiteandis distributedunder theGPLlicense.

Linux

kernel

Anumberof patcheshave beenmadeto theLinux2.6.16kerneltoprovideservices neededto supportthehardwarefacilitiesof theCellBEProcessor.

For theBladeCenterQS20thekernelisinstalledintothe/boot directory,

yaboot.conf ismodifiedandareboot isrequiredto activatethis kernel.The

cellsdk installtask (see“Simpleinstall oftheSDK”onpage17)providesanoption (--nokernel)notto installthiskernel.

Note: Thecellsdk uninstallcommanddoes notautomatically uninstallthekernel

toavoidleaving thesystem inanunusablestate.

Thepatches forthe2.6.16kernelareavailableontheBSCWebsitedirectoryand aredistributedundertheGPLlicense.

Linux

support

libraries

Thefollowingsupportlibraries areprovided bytheCellBESDKtoaiddevelopment andperformancetest ofCellBEapplications.

SPE

runtime

management

library

TheSPEruntimemanagementlibrary(LIBSPE)containsanSPEthread programming modelfor CellBEapplications.TheELFSPEenablesdirectSPE executable executionfromaLinuxshellwithouttheneedof aPPEapplication creating anSPEthread.

For BladeCenterQS20stheLIBSPEheaders, librariesandbinariesareinstalledby theSDKintothe /usrdirectory andthestandalone SPEexecutive,ELFSPE,is registeredduringsystemrootbycommandsaddedto /etc/rc.d/init.d.

For thesimulatortheLIBSPEandELFSPEbinariesandlibraries arepreinstalledinthe samedirectoriesinthesystem rootimageandnofurtheractionisrequiredatinstall time.

Thesourcefor theSPEruntimemanagement libraryisavailableontheBSCWeb site andisdistributed undertheGPLlicense.

(33)

numactl

numactl isusedtocontrolNUMApolicyforprocesses orsharedmemory.An applicationcantakeadvantageof thisfacilitybybinding aregionof virtualstorage to aspecificbank ofmemoryaswellasbinding aprocessto aspecificNUMA node.

Note: ThestandardFedoraCore 5versionof numactldoes notworkcorrectlywith

theCellBEprocessor. Insteadthelatestversionof thenumactl-0.9.8for FedoraCore 6isinstalledbytheSDKonBladeCenterQS20s.TheFedora Core6 versionofnumactl isavailableontheBSCWebsite anddistributed undertheGPLlicense.

Libraries

and

samples

Thelibraries andsamplesRPMpackageprovidesarichsetofoptimizedstandard SPEClibraryroutinesthatgreatlyreducethedevelopmentcost andenhancethe performanceof SPUprograms.Avarietyof application-orientedlibraries,including Fast FourierTransform(FFT),image,audioresample,math,game math,intrinsics, matrixoperation,multi-precisionmath, noisegeneration,oscillator, surface,

synchronization, andvectorarealsoincludedinordertodemonstratetheversatility of CellBEarchitecture.Additionalsamplesandworkloads demonstratehowa programmer canexploittheon-chipcomputationalcapacity.IncludedisalargeFFT workloadthatshowcasesperformancethatismorethananorderofmagnitude higherthanatraditionalprocessor.

Thelibraries andsamplesourcesareinstalledinthe/opt/IBM/cell-sdk-1.1

directory undertheIBMCPLlicense.

Libraries

and

samples

subdirectories

Thelibraries andsamplesRPMhas beenpartitionedintothefollowing subdirectories.

Table4.SubdirectoriesforthelibrariesandsamplesRPM

Subdirectory Description

bin ExecutablesdirectorycontainingtheSPUTimingtoolandtheILARlicenseforthistool. docs ContainsdocumentationaboutlibrariesandtoolssuchasIDLcompiler.

host Hostsystemexecutables,headersandlibrariesfortheIDLtool. license ContainsthetextfortheCPLlicense.

src/include Systemheaderfiles.Thesefilesareexportedtothe$SDKINC_<target>(wheretargetiseither ppuorspu)directoryforgeneralusethroughouttheSDK.

scr/lib Seriesoflibrariesandreusableheaderfiles.Theseareexportedto$SDKLIB_<target>or $SDKINC_<target>directories(respectively).Completedocumentationforalllibraryfunctionsand availablein/opt/IBM/cell-sdk-1.1/docs/lib/libraries.pdf.

(34)

Table4.SubdirectoriesforthelibrariesandsamplesRPM (continued)

Subdirectory Description

src/samples ThesamplesdirectorycontainsexamplesofCellBEprogrammingtechniques.Eachprogram showsaparticulartechnique,orsetofrelatedtechniques,indetail.Youmayreviewthese programswhenyouwanttoperformaspecifictask,suchasdouble-bufferedDMAtransfersto andfromaprogram,performinglocaloperationsonanSPU,orprovideaccesstomainmemory objectstoSPUprograms.

Somesubdirectoriescontainmultipleprograms.Thesyncsubdirectoryhasexamplesofvarious synchronizationtechniques,includingmutexoperationsandatomicoperations.

TheJuliasubdirectory,whichisnewforSDK1.1,containssamplecodethatraytracesthe quaternionJuliaSetintoabitmapimage–seeAppendixA,“QuaternionJuliaSetRay-tracing sample,”onpage37formoredetails.

Thespuletsubdirectoryofsamplesincludescustomizedstartupcodetoletsimpleprogramsbe runontheSPUdirectlyfromaLinuxcommandprompt.Forinstance,thehelloexampleprintsa traditionalgreeting.Thespuletmodelisintendedtoencouragetestingandrefinementof programsthatneedtobeportedtotheSPUs;italsoprovidesaneasywaytobuildfiltersthat takeadvantageofthehugecomputationalcapacityoftheSPUs,whilereadingandwriting standardinputandoutput.

src/tests Thetestsdirectorydefinessomeregressiontestsforthesystem.Theseprogramsexercisekey systemcomponents.Forinstance,theasm/verifycoveragetestcontainsanon-executable sampleprogramfortheSPUwhichexistsonlytobeassembled;itisnotactuallymeantto execute,butitvalidatesthefunctionalityofanSPUassembler.Theeventssubdirectoryprovides asampleprogramshowingthehandlingofuser-definedevents;thisprogramhelpsverifythe functionofthesimulator.

src/tools ThetoolsdirectorycontainstoolsthatareusefulforsoftwaredevelopmentsuchastheInterface DefinitionLanguage(IDL)compilerandthecallthruprogram.TheIDLcompilerreadshigh-level interfacespecificationsandgeneratesPPUandSPUstubfunctionstoallowPPUcodeto″call″a functionthatisimplementedandrunsontheSPU.Thisismostusefulforcaseswhereyouhave acomputationallyintensivetaskthatiswellsuitedtosimplybeinghandedofftoanSPUfor processing.ThecodethisgeneratesisinformativeaboutPPU/SPUcommunications,butis primarilyintendedasaprototypingtoolratherthanalearningtool.

TheIDLtoolhasitsownsampleswhichshowhowtooffloadsomeprocessingtotheSPU.The IDL’sPPUstubcodesupportsdynamicallocationofmultipleSPUstohandlesimultaneous offloadedfunctions,andmultiplefunctionscanbeloadedonasingleSPU,iftheyaresmall enough.Somefeaturesarestillunderdevelopmentsuchasdoublebufferingsupport.

src/workloads Theworkloadsdirectoryprovidesahandfulofexamplesthatcanbeusedtobetterunderstand theperformancecharacteristicsoftheCellBEProcessor.Therearefoursampleprograms, whichcontaininsightsintohowreal-worldcodeshouldrun.

Note: RunningtheseexamplesusingthesimulatortakesmuchlongerthanonthenativeCell BE-basedhardware.Theperformancecharacteristicsinwall-clocktimeusingthesimulatorare extremelyinaccurate,especiallywhenrunningonmultipleSPUs.InsteadtheemulatorCPUcycle countsneedtobeexamined.

Forexample,thematrix_mulprogramletsyouperformmatrixmultiplicationsononeormore SPUs.MatrixmultiplicationisagoodexampleofafunctionwhichtheSPUscanaccelerate dramatically.

Unlikesomeoftheothersampleprograms,theseexampleshavebeenhand-tunedtogetthe bestperformance.Thismakesthemhardertoreadandunderstand,butitgivesanideaforthe kindofperformancecodethatcanbewrittenfortheCellBEprocessor.

sysroot Containssomeoftheheadersandlibrariesusedduringcross-compilingandcontainsthe compiledresultsofthelibrariesandsamples.Thiscanbesynchedupwiththesystemroot imagebyusingthecommand:/opt/IBM/cell-sdk-1.1/cellsdk synch

(35)

SPU

timing

tool

TheSPUstatictimingtool,spu_timing,annotatesanSPUassemblyfilewith scheduling,timing,andinstructionissueestimatesassumingstraight, linear executionof theprogram.Thetoolgeneratesatextualoutputoftheexecution pipelineoftheSPEinstructionstreamfromthisinput assemblyfile.Execute

spu_timing –helpto see itsusagesyntax.

TheSPUtimingtoolisdistributedasanRPMunder theIBMILARlicenseandis located inthe/opt/IBM/cell-sdk-1.1/bindirectory.

(36)

(37)

Chapter

5.

Using

the

SDK

This sectionisashortintroduction intousingtheSDK.Refertotheprogramming tutorial,theFull-SystemSimulatoruser’s guide,andotherdocumentation formore details.

Running

the

Full-System

Simulator

ToverifythattheFull-SystemSimulator(simulator)isoperating correctlyandthen runit,issuethefollowing commands:

# mkdir sandbox # cd sandbox

# cp /opt/IBM/systemsim-cell/run/cell/linux/.systemsim.tcl . # export PATH=/opt/IBM/systemsim-cell/bin:$PATH

# systemsim -g

The.systemsim.tclfileisastartupscriptforthesimulatorthatconfiguresthe system andpreparesitto bootandruntheLinuxoperatingsystem.

Thesystemsim scriptfound inthesimulator’sbin directorylaunchesthesimulator andthe–gparameterstartsthegraphicaluserinterface.

(38)

Notes:

1. You mustbeona graphicalconsole,orat leasthave theDISPLAYenvironment

variable pointedtoa friendlyXserverto runthesimulator'sgraphicaluser interface.

2. If anerrormessageaboutlibtk8.4.soisdisplayed, youmustloadtheTK

packageasdescribedin“TK”onpage15

When theGUIisdisplayed,clickGotostartthesimulator.

Note: Tomakethesimulatorruninfastmode,youcanclickFastModeandthen

Go.Thisforcesthesimulatorto bypassitsstandardanalysisandstatistics collectionfeatures. Fastmodeisusefulifyouwanttoadvancethesimulator throughsetupor initializationfunctionsthatarenotthefocusofanalysis, suchas,theLinuxbootprocessing.You shoulddisablefastmode whenyou reachthepoint atwhichyouwishto dodetailed analysisordebugthe application.

You canusethesimulator'sGUIto getabetterunderstandingoftheCellBE architecture. Forexample,thesimulatorshows twosets ofPPEregisters.Thisis becausethePPEprocessorcore isdual-threaded andeachthreadhas itsown registers andcontext.Youcanalso lookat the128-bitregisterarraysprovidedon theSPEs.

Thesyntaxof thesystemsimcommandis: systemsim [-f file] [-g] [-n]

where:

Parameter Description

-f<filename> specifiesaninitialrunscript(TCLfile)

-g specifiesGUImode,otherwisethesimulatorstartsincommand-line mode

-n specifiesthatthesimulatorshouldnotopenaseparateconsole window

You canfinddocumentationabout thesimulatorincludingauser’s guideinthe

/opt/IBM/systemsim-cell/doc directory.

The

callthru

utility

Thecallthruutilityallowsyoutocopyfilesto orfromthesimulatedsystemwhileit isrunning.Theutilityislocatedinthesimulatorsystemrootimageinthe/usr/bin

directory.

If youcalltheutilityas:

v

callthrusink <filename>, it writesits standard inputinto <filename>onthe

hostsystem

v callthrusource <filename>, it writesthe contentsof <filename>onthehost

systemto standardoutput.

Redirectingappropriatelyletsyoucopyfilesto andfromthehost. Forexample, whenrunningonthehost,youcouldcopya CellBEapplicationinto/tmp: cp matrix_mul /tmp

Then, intheconsolewindowofthesimulatedsystem,you couldaccessitlike this:

(39)

callthru source /tmp/matrix_mul > matrix_mul chmod +x matrix_mul

./matrix_mul

Note: Theuseofthe>redirect.Ifyouomit >(callthrusource)and<(callthrusink)

thiscancauseproblemswhenyouaredebugging. The/tmpdirectoryisshown asanexampleonly.

Thesourcefilesforthecallthruutilityarein/opt/IBM/systemsim-cell/sample/ callthruandtheMakefiletobuildtheutilityisin/opt/IBM/cell-sdk-1.1/src/ tools/callthru.

Read

and

write

access

to

the

simulator

sysroot

image

By defaultthesimulatordoes notwritechangesbackto thesimulatorsystem root (sysroot) image,thismeansthatthesimulatoralwaysbeginsinthesameinitial state ofthesysrootimage.As youbecomemoreexperienced,youcanoverwrite this defaultbehaviorsothatanyfilechangesmadebythesimulatedsystemtothe sysroot imageapplyto subsequentsimulatorsessions.

Tospecifythatyouwantto usechangesyouhavemade tothesysrootimage, change the newcowparameter onthe mysim bogus disk initcommand in

.systemsim.tclto rw(specifyingread/writeaccess)andremovethelasttwo parameters.Here isthechanged linefrom.systemsim.tcl:

mysim bogus disk init 0 $sysrootfile rw

Enabling

Symmetric

Multiprocessing

Support

By defaultthesimulatorprovidesanenvironment thatsimulatesoneCellBE processor.TosimulateabladeenvironmentwheretwoCellBEprocessorsexist, you mustenableSymmetricMultiprocessing(SMP) support.Todothis, editthe startupscriptthatyoursimulatorisusing,which bydefaultis.systemsim.tcl.Add thesethreelinestothestartof thefile:

proc config_hook { conf } {

config_dual_be $conf

}

When thesimulatorisstarted,ithasaccessto16SPEsacrosstwo CellBE processors.

SDK

programming

samples

Each ofthesampleshas anassociatedREADME.txtfile.Thereisalso atop-level

readmeinthe/srcdirectory,whichintroducesthestructureof thesamplecode sourcetree.ThereareanumberofPDFdocumentsinthe/opt/IBM/cell-sdk-1.1/ docsdirectory includingaprogrammingtutorial.

Almostallof thesamplesrunbothwithinthesimulatorandontheBladeCenter QS20. SomesamplesincludeSPU-onlyprogramsthatcanberunonthesystem simulatorinstandalonemode.

Code,which isspecificto agivenCellBEprocessorunittype, isinthe correspondingsubdirectorywithinagivensample’sdirectory:

v ppufor codecompiledto runonthePPE

(40)

v spu_simfor codecompiledtorunonanSPEunder thesystem simulatorin

standaloneenvironmentnativelyonaBladeCenterQS20

Changing

the

default

compiler

In /opt/IBM/cell-sdk-1.1therearesometoplevelmakefilesthatcontrolthebuild environment forallof thesamples.Most ofthedirectoriesinthelibraries and samplescontaina makefilefor thatdirectoryandeverythingbelowit.Allofthe sampleshave theirownmakefilebutthecommon definitionsareinthetoplevel makefiles.

Thebuildenvironmentmakefilearedocumented in/opt/IBM/cell-sdk-1.1/ README_build_env.txt.

Environmentvariablesinthe/opt/IBM/cell-sdk-1.1/make.envmakefileareusedto determinewhichcompilerisusedtobuildthesamples.

Thecellsdkscriptcontainsa taskwhichautomatically switchesthecompiler,doesa makecleanandthenamakewhichrebuildsallof thesamplesandlibraries.The syntax ofthiscommandis:

./cellsdk build [-x | -g ]

wherethe –xflag selectstheXLC/C++compiler(x86 only)andthe–gflagselects theGCCcompiler.

Afteryouhave selecteda particularcompiler,thatsamecompilerisusedfor all future builds.BydefaulttheGCCcompilerisselectedfor compilingthesamples. For x86hosts,youcanoverwritethisdefaultatinstall timebyusingthe–xoption with thecellsdk install command.

Building

and

running

a

specific

program

You donotneedto buildallthesamplecode atonce;youcanbuildeachprogram separately.Tostart fromscratch, issueamakeclean usingthemakefilein

the /opt/IBM/cell-sdk-1.1/srcdirectoryoranywhereinthepathtoa specific libraryorsample.

If youhave performeda makecleanat thetoplevel,youneedto rebuildtheinclude filesandlibrariesfirstbefore youcompileanythingelse.Todothisrunamakeinthe

src/include andsrc/lib directories.

For example,buildthehellospulet.Thespuletexamplesrequirea specialversion of crt0,builtfromsrc/samples/spulet/crt0.Aftertheinclude filesandlibraries are built,changedirectory tosrc/samples/spulet/hello,andrunmake.

System

root

Support

for

huge

TLB

file

systems

TheSDKsupportsthehugetranslationlookaside buffer(TLB)filesystem,which allowsyouto reservelarge pagesof pinned,contiguousmemory.Thisfeatureis particularly usefulforsomeCellBEapplicationsthatoperateonlargedatasets suchastheFFT16MworkloadsampleandCellBETerrainRenderingEngine. ToconfiguretheCellBE-basedbladeserverfor 20pages(320MB),runthe following commands:

mkdir -p /huge

echo 20 > /proc/sys/vm/nr_hugepages mount -t hugetlbfs nodev /huge

If youhavedifficulties configuringadequatehuge pages,it couldbethatthe memoryisfragmentedandyouneedto reboot.Youcanaddthecommand sequenceshown aboveto astartupinitializationscript,suchas

/etc/rc.d/rc.sysinit,sothatthelargeTLBfilesystem isconfiguredduringthe system boot.

Toverifythelarge memoryallocation, runthecommandcat/proc/meminfo. The outputissimilarto:

MemTotal: 1010168 kB MemFree: 155276 kB . . . HugePages_Total: 20 HugePages_Free: 20 Hugepagesize: 16384 kB

SDK

development

best

practices

This sectio