SETUP AND MANAGEMENT OF MODELS IN CUBE CLOUD. Colby M. Brown, AICP PTP

1

SETUP AND MANAGEMENT

OF MODELS IN CUBE CLOUD

• Develop the Model with Cube in the Desktop Environment • Publish the Model from Cube to the Cube Cloud

• Create, run and analyze scenarios from anywhere

INTRODUCING CUBE CLOUD

 The exporter converts the catalog into a “minimal” directory structure

UNDERSTANDING CUBE CLOUD

 Web

▪ Working directory/scenario directory

 Apps

▪ Model script file and catalog information (.xml)

 Inputs

▪ All inputs (scenario specific, fixed inputs)

 Outputs

▪ Empty and used to store fixed outputs when model is run

UNDERSTANDING CUBE CLOUD

Cube Cloud Export

 Compile model scripts and model catalog

 Create one model script and a .xml file in ‘web\apps’

 Copy input files to ‘web\inputs’

 Compress ‘web’ folder for upload

Cube Cloud Model Run

 Each scenario run gets its own copy of the model folder

 All outputs are saved in the ‘Outputs’ folder or the ‘Model’ folder (scenario/working directory)

UNDERSTANDING CUBE CLOUD

DIRECTORY STRUCTURE

7

PREPARING FOR CLOUD

 Use Cube Catalog and Application Manager

 Use file keys for input files

 Use unique file names for scenario specific input files

 Reference all model run outputs to the {SCENARIO_DIR}

 Directly link output files in application manager instead of reading the file internally

 Always use double quotes for file paths and names

 Execute user programs from scenario dir

Scenario Specific Input files

 FILEI input file statements with ‘{SCENARIO_DIR}’ in path name

▪ e.g., FILEI ZONEI[1]=“{SCENARIO_DIR}\Zones\ZoneData.DBF”

 Cube Cloud exporter does not recognize scenario specific inputs

▪ Warning: Scenario Specific Inputs. The Application scripts are

valid, but there is currently no interface to this data in Cube Cloud.

 Use file keys

▪ e.g., FILEI ZONEI[1]=“{zonedata}”

Fixed Input Files

 FILEI statements with fixed file path

▪ FILEI RECI = “C:\Model\Parameters\MC_PAR.DBF“

 Cube Cloud exporter will copy these files to the ‘Inputs’ folder

 The Cube Cloud model script will be updated as

▪ FILEI RECI = “{SCENARIO_DIR}\Inputs\MC_PAR.DBF”

 Cube Cloud user will not be able to update a new version or edit this file

 Use a file key to input this file

Internal Input Files

 ‘FILE=’ statements for READ, RENUMBER, LOOKUP statements.

▪ e.g., READ FILE ="{SCENARIO_DIR}\INPUT\ACCESS.PRN“

 Cube Cloud Exporter does not accept any file path keys in ‘FILE=’ statements. Exporter does not copy this file to the ‘inputs’ directory.

▪ Warning. The following Internal "FILE=" entries are present in the model. They have been associated with the VOYAGER command shown below, and copied to the appropriate folder. Note that for Cube Cloud, the best and safest approach is to avoid use of

FILE=,and use an alternative VOYAGER construct if available; e.g. for PRINT commands, use PRINTO=.

 Use a user defined key for these types of files or reference these files using {CATALOG_DIR} key.

▪ e.g., READ FILE ="{ACCESS}"

 ‘FILEO’, ‘PRINT FILE=’ and ‘COPY…ENDCOPY’ statements

▪ e.g., FILEO MATO[1] =

"{SCENARIO_DIR}\OUTPUT\MATRIX\OUTPUT.MAT“

 Cube Cloud model run will output these files to the ‘outputs’ directory unless the file is output directly to the scenario directory

 Subsequent steps reading these files internally will fail in Cube Cloud.

 Output all scenario specific files directly to “{SCENARIO_DIR}”

▪ e.g., FILEO MATO[1] = "{SCENARIO_DIR}\OUTPUT.MAT“

 FILEO/COPY/PRINTO to READ, RENUMBER,LOOKUP (FILE=) statements

▪ Program 1-Matrix:

FILEO PRINTO[1] = "{SCENARIO_DIR}\READ_OUT.PRN"

Program2-Matrix:

READ FILE = "{SCENARIO_DIR}\READ_OUT.PRN“

 Cube Cloud Exporter will show a error message

▪ Warning: The following Internal files (defined via FILE=) could not be found, so the Applications will not run correctly

 It is a good practice to directly link output files to inputs in subsequent programs rather than reading internally

FILEI statements with user defined file path keys

▪ FILEI MATI[1] = "{LOCATION}\AllWalk.MAT"



Cube Cloud exporter does not accept any keys (other than

{CATALOG_DIR}) in file

PATH

of FILEI statements.

▪ Error: The following files have Keys in their Path. This construct is incompatible with Cube Cloud.



Use the {CATALOG_DIR} key to refer to this file as shown

below or use file keys as discussed above

▪ FILEI MATI[1] = "{CATALOG_DIR}\Base\Input\AllWalk.MAT"

 File references using tokens

▪ e.g., READ FILE = "@CPI_FILE@"

 Cube Cloud exporter does not give any warnings or errors

 Cube Cloud model script is updated to read

▪ READ FILE = “{CATALOG}\OUTPUTS\@CPI_FILE@”

 Use valid file paths instead of tokens

 Cube Cloud exporter appends a suffix (_cw#) input files with similar names.

 It also updates the catalog key values for these files with updated file names

▪ e.g., {File1} = “C:\Model\Base\ZoneData.dbf”

{File2} = “C:\Model\Base\Parameters\ZoneData.dbf”

Exporter will copy 2 files to the ‘inputs’ folder with the names

“ZoneData.dbf” & “ZoneData_cw1.dbf” and update the file key values as follows

{File1}=“inputs\ZoneData.dbf”

{File2}=“inputs\ZoneData_cw1.dbf”

 Built-in User Programs

▪ e.g., TRCOPY, TRDEL

 Cube Cloud exporter will show a warning

▪ Warning: User Programs currently require some manual intervention to implement in Cube Cloud.

 Built-in user programs will work without any problems in Cube Cloud

 User supplied user programs

▪ e.g., any .exe files to perform special functions

 All input files and output files to/from the user program should be referenced to the {SCENARIO_DIR}

 Update any control files/input files with file paths

 Execute the user programs from {SCENARIO_DIR}

 User programs requiring special set-up

▪ Certain user programs (e.g., Java Parallel Processing Framework) might require additional folder set-up and special configuration for running

▪ Additional set-up might include background processes to support the model run

 Citilabs will create a new Cube Cloud image to handle these special cases (e.g., CT-RAMP) according to each client’s needs

 Any MS-DOS system commands

▪ e.g., COPY, DEL etc.

 Cube Cloud Exporter will show a warning message

▪ Only copy system commands with no wildcards are currently supported by Cube Cloud for file name conversion.



It is recommended to copy the individual files using one copy

command for each file as shown below.

▪ E.g., *COPY “{SCENARIO_DIR}\INPUTS\*.csv” “{SCENARIO_DIR}” - Will not work

*Copy ““{SCENARIO_DIR}\INPUTS\File1.csv” “{SCENARIO_DIR}”

 Cube Cloud provides scalability of available cluster nodes for each model run

 ProcessNum/ProcessList for ‘DistributeMultistep’ and

‘DistributeIntrastep’ statements should be calculated within the script based on the number of cores available for that run

 The number of cores available for the model run is input using a scenario key.

 No need for cluster commands to start and close cluster nodes.

▪ e.g., *Cluster Test 1-10 start exit

PILOT:

If ({CORES}>1)

ClusterNodes={CORES}-1 Distribute Multistep=T Intrastep=T Else

ClusterNodes=1

Distribute Multistep=F Intrastep=F EndIf

Intranodes = INT((ClusterNodes-2)/3) Start1=3

End1= Start1 + Intranodes Start2= End1 + 1

End2= Start2 + Intranodes Start3= End2 + 1 End3= Start3 + Intranodes

DistributeMultistep ProcessId=‘Training’ ProcessNum=1 RUN PGM=MATRIX DistributeIntrastep ProcessId=‘Training’ ProcessList=@Start1@-@End1@ ENDRUN DistributeMultistep ProcessId=‘Training’ ProcessNum=2 RUN PGM=HIGHWAY DistributeIntrastep ProcessId=‘Training’ ProcessList=@Start2@-@End2@ ENDRUN RUN PGM=MATRIX DistributeIntrastep ProcessId=‘Training’ ProcessList=@Start3@-@End3@ ENDRUN

CUBE CLOUD CLUSTER SETUP

 From Application 1

FILEO MATO[3]="{SCENARIO_DIR}\OUTPUT.MAT” In Application 2

FILEI MATI[1]="{SCENARIO_DIR}\OUTPUT.MAT”

 Multiple applications are not currently supported in Cube Cloud

 Cube Cloud Exporter will show a warning message

▪ Warning: The following files require a previous Application to be run first in order to create them

MULTIPLE APPLICATIONS

OUTPUT/INPUT

 Cube Cloud does not currently allow uploading models with reports or more than one applier group

 Uploading a model with reports will cause an error in the uploading process

 Cube Cloud has in-built analysis tools to do charts and maps as well as built-in user access control and administration

 Delete reports and applier groups before uploading

MANAGING ACCESS THROUGH ADMIN

CONTROL PANEL

27 ANY QUESTIONS?