Writing a Submission Script

A submission script is a file listing the options and execution steps you “submit” to cluster to schedule and run. The Asha cluster uses the Slurm Job Scheduler. Here is a sample script to compile and run a basic MPI program. 

#!/bin/bash

#SBATCH --partition=all
#SBATCH --job-name=test-mpi

mpicc ./mpi_test.c -o mpi_test
mpiexec ./mpi_test
  • The first line needs to start with the line  “#! /bin/bash” line. This is called a shebang
  • Option lines start with “#SBATCH”. 
    • In the example we choose the “all” partition and name our job “test-mpi”
  • The steps that need to be executed are written just like you would in a Unix terminal
    • In the example case the two lines starting with “mpicc and mpiexec”. 
After writing your submission script, you can submit it to the scheduler via the command “sbatch”. For example, “sbatch sample_submission.sh”

Common options for your submission script.

Option Short Description Example Line in Script
--partition Choose which partition (similar to a queue) of the cluster to submit your job to. More info on partitions in on the Asha cluster. #SBATCH --partition=all
--job-name Name your job to identify it during the run. #SBATCH --job-name=test-mpi
--ntasks Use this option when doing multiprocessing (MPI, Fluent, Matlab) to reserve a number of cores for your job. Can be combined with cpus-per-task option, but usually just use one or the other depending on the job. #SBATCH --ntasks=100
--cpus-per-task Use this option when doing multithreading (OpenMP, Tensorflow) to reserve a number of cpu cores. #SBATCH --nodes=4
--nodelist/--exclude Specify a list of nodes to run your job on, or a list of nodes to exclude from running on. Please note that --nodelist may include other nodes if your job requires more resources than are available from the list provided. If you need to restrict your job to certain nodes, use --exlcude instead. #SBATCH --nodelist=coe-node[1-3],coe-gpu1
--mail-user
--mail-type
Using these two options will have slurm email you at the end of your job. #SBATCH --mail-user=myemail@colostate.edu
#SBATCH --mail-type=END
--output Name your output file. All standard out from a node gets put into this file. #SBATCH --output=results.txt
--time Restrict your job to a certain time limit. Default is set on a per-partion basis. Time formats include "minutes", "minutes:seconds", "hours:minutes:seconds", "days-hours", "days-hours:minutes" and "days-hours:minutes:seconds". #SBATCH --time=1-3:45"

Important Note for Windows Users

Line endings differ between Windows and Linux. If you write your submission script without accounting for this, it may result in odd errors.  There are multiple options to mitigate this: 

  • Use a terminal editor on the cluster.
    • micro
    • nano
    • vim (harder to learn)
    • emacs (harder to learn)
  • Use a graphical editor on a Linux or OSX machine. Such as the Linux Compute Servers.
  • Configure your editor in Windows to use Linux line endings. Notepad++ (not Notepad or Wordpad), and others can do this for you. 
  • Use the command “dos2unix <filename>” on the cluster for each file you move to the cluster.