DV-Zeuthen

| Computer Center

Job submission

Computer Center

Job submission

All WGS nodes in Zeuthen that are capable to submit GridEngine jobs can be used as HTCondor submit node, as well. If you prefer “local” job submission, log in to htc-submit:

[wgs] ~ ssh htc-submit

1. Simple job submit file

Job scripts are no longer actual shell scripts! Use a submit file containing the relevant information instead and let Executable point to your old job script. GridEngine-specific options in the script like #$ -l ... are no longer interpreted.

Executable  = /path/to/your/jobscript
Log         = /path/to/log/log_$(Cluster)_$(Process).txt
Output      = /path/to/output/out_$(Cluster)_$(Process).txt
Error       = /path/to/output/err_$(Cluster)_$(Process).txt

# request 2GB RAM
request_memory = 2048

Queue 1

Variables like $(Cluster) & $(Process) are replaced as documented here.

Then submit the job with:

[wgs] ~ condor_submit <submit_file>

2. Convert GridEngine job scripts to HTCondor

Short overview how GridEngine-specific job requirements are replaced in HTCondor submit files:

Description
GridEngine
HTCondor
Notes
STDOUT
-o /path/to/file
output = /path/to/file
 
STDERR
-e /path/to/file
error = /path/to/file
 
Memory requirement
-l h_rss=2G
request_memory = 2G
Default: 1G
Disk requirement in $TMPDIR
-l tmpdir_size=2G
request_disk = 2G
Default: 1G
Job runtime
-l h_rt=10:00:00
+RequestRuntime = 10 * $(HOUR)
Optional at the
moment!
Multicore jobs
-pe multicore 4
request_cpus = 4
 
GPU jobs
-l gpu=1
request_gpus = 1
 
Send current shell environment to job
-V
getenv = True
Not recommened
Send mail on job events
-m <b|e|a>
notification = <Always|Complete|Error>
Not recommended
for mass jobs
Array jobs
-t 1-100
queue 100
Also consider usage of
max_materialize !

3. Interactive job submissions

Interactive job submissions to a farm node are supported:

[wgs] ~ condor_submit -i