(Update: this, along with other examples of HPC submission scripts, can be found here: https://github.com/jdherman/hpc-submission-scripts).
Instead of using Bash to automate cluster submission scripts, you may be interested in trying Python instead. The end result is the same, but the process is a bit cleaner. I’ll paste an example below that iterates through a loop to submit a bunch of jobs. It follows the same outline as our usual Bash scripts (i.e., build a job submission string and then pipe it to qsub).
#!/usr/bin/python # Example PBS cluster job submission in Python from popen2 import popen2 import time # If you want to be emailed by the system, include these in job_string: #PBS -M your_email@address #PBS -m abe # (a = abort, b = begin, e = end) # Loop over your jobs for i in range(1, 10): # Open a pipe to the qsub command. output, input = popen2('qsub') # Customize your options here job_name = "my_job_%d" % i walltime = "1:00:00" processors = "nodes=1:ppn=1" command = "./my_program -n %d" % i job_string = """#!/bin/bash #PBS -N %s #PBS -l walltime=%s #PBS -l %s #PBS -o ./output/%s.out #PBS -e ./error/%s.err cd $PBS_O_WORKDIR %s""" % (job_name, walltime, processors, job_name, job_name, command) # Send job_string to qsub input.write(job_string) input.close() # Print your job and the system response to the screen as it's submitted print job_string print output.read() time.sleep(0.1)
Make sure you give executable permissions (chmod +x thisFile.py) and then just run ./thisFile.py from the command line.