The Cherry L. Emerson Center for Scientific Computation
about us
center subscription
visiting fellows
lectureship award
user handbooks

The Queueing System

The Queueing System | FAQ | Hardware Specification | Software Description | How to get an account

How to Use the Emerson Center Facilities

1. LoadLeveler Queueing System

The EC uses LoadLeveler (LL) to provide efficient usage of the hardware equipment and software to our subscribers. LL runs scripts (jobs) on your behalf on the appropriate machine in a cluster of machines. LL issues standard UNIX commands in batch mode, submits your jobs into existing queues, and assigns a number and name to your jobs. In brief, you are required to prepare your input file, include the name of that input file into a LL shell script (usually ksh) called xxx.cmd (fixed formatted, xxx could be any name) and submit it (xxx.cmd) to our computers by executing the “llsubmit xxx.cmd” command.

EC has three independent groups of queues, namely SP3+SGI, SP2, and RS/6000. While the number of jobs submitted to the RS/6000 queue is unlimited, there exist limits for the maximum number of jobs per subscriber group on the SP2 and SP3+SGI systems corresponding to the shares (or units) of subscribers.

The groups of queues can be distinguished from each other by LL parameters such as "class" and "Features", which are LL keywords that need to be included in the LL xxx.cmd script file.

A. IBM RS/6000 and P+ (AIX 5.3) standalone service machines: file & mail server software & LL server gateway to SUN/Linux gateway to SGI; 4e "mirror" former web server

B. SUN Fire v40z/v20z Linux (SuSe 9.1) cluster Linux FS; PBS server; graphics
f01-f26 dual CPU Opteron compute nodes

C. IBM SP4 Power (AIX 5.3) cluster (*) 4-way Power-4 nodes

* X=1,2,...,6

D. IBM SP3 Power (AIX 5.3) cluster SP3 file server; storage facility (*) 4-way Power-3 nodes

* X=1,2,...,15

E. SGI Origin (IRIX 6.5) workstations software server; graphics graphics; compute node compute node

2. How to submit a job to LoadLeveler

Use this .cmd file as a template. Note that to submit to fire queues, substitute /home by /fire in the "initialdir" line below.

# @ initialdir = /home/<yourgroup>/<yourname>/...
# @ requirements = (Arch == "R6000") && (OpSys == "AIX53")
# @ class = fireq
# @ notify_user = <you@e-mail address>
# @ group = <yourgroup>
# @ error = error
# @ queue
. /libs/scripts/m2006.1 # molpro
#. /libs/scripts/g03d1 # gaussian

Please feel free to browse /libs/scripts for available codes.

Please include the direction for LoadLeveler in your $PATH environment variable. This can be done with placing the following additional lines at the end of your .profile (for ksh users):
export PATH
or .cshrc (for csh users):
set path=(/usr/lpp/LoadL/nfs/bin $path)

In order to submit a job, "ssh -l YOURNAME". Job submission is only allowed from this particular machine. The LL command for submitting a job is llsubmit:
euch4e# llsubmit xxx.cmd
Typical LL scriptfiles are located in the /usr/local/Samples directory of euch4e, and we highly recommend to start creating own script files by modifying these files appropriately.

3. Program specific LoadLeveler include scripts

Most installed software packages can be used by including ksh scripts provided by the center. These files can be found in the /libs/scripts directory on euch4e and are usually incorporated ino a LL script by the following three lines:
. /libs/scripts/<script_filename>
(please note the leading dot in the third line). This syntax species the location of input, output files and the type of program to run. Most important scripts are:/libs/scripts/g98a9 (to run GAUSSIAN98 Revision A.9 on SP3 nodes)
/libs/scripts/g98run (to run GAUSSIAN98 Revision A.7 on RS/6000 and SP2 nodes)
/libs/scripts/g98sp3 (to run GAUSSIAN98 Revision A.7 on SP3 nodes)
/libs/scripts/m98/m2000 (to run MOLPRO 2000.1 on RS/6000 and SP2 nodes)
/libs/scripts/m98/m2001 (to run MOLPRO 2000.1 on SP3 nodes)
/libs/scripts/sybyl (to run Sybyl Version 6.7 on SGI)
/libs/scripts/acesrun (to run ACESII 3.0 on RS/6000 and SP2 nodes)
/libs/scripts/acesp3 (to run ACESII 3.0 on SP3 nodes)
/libs/scripts/amber (to run AMBER Version 6.0 on SGI)
/libs/scripts/bmin (to run MacroModel_7.1 Batchmin on SGI)
/libs/scripts/cadpac (to run CADPAC Version 5.2 on IBM’s)
/libs/scripts/dalrun (to run DALTON Version 2.0 on IBM’s)
/libs/scripts/gmsrun (to run GAMESS Version Sept. 2000 on RS/6000 and SP2 nodes)
/libs/scripts/gmsrunsp3 (to run GAMESS Version Sept. 2001 on SP3 nodes)
Please note that there are different LL include scripts for the same software, depending on where the job will run. It is important not to mix platforms and include files up.
If a user has his own shell script, it is important to know that /work is the default location of temporary files. However, when using the SP3 nodes, the following part needs to be included in the script in order to determine the location of the work directory (either /work, /work1, /work2, or /work3):
LOADL_ID=`echo "${LOADL_JOB_NAME}" | sed 's/'`
w1=`cat $WDIR/work`
w2=`cat $WDIR/work1`
w3=`cat $WDIR/work2`
w4=`cat $WDIR/work3`
if [ "$w1" = "$LOADL_ID" ]
export scr=/work
elif [ "$w2" = "$LOADL_ID" ]
export scr=/work1
elif [ "$w3" = "$LOADL_ID" ]
export scr=/work2
elif [ "$w4" = "$LOADL_ID" ]
export scr=/work3
exit 111

short courses emory university shield
points of interest

About Us | Center Subscription | Visiting Fellows | Lectureship Award | Research | User Handbooks | News | Center Homepage

Short Courses | FAQs | Points of Interest | Search | Emory College | Emory University


Emory University