|Help|| man page|
GNU Parallel is powerful, lightweight utility for running single-threaded (i.e. serial) programs multiple times with different inputs.
It parallelizes software which which was not designed for parallel execution, and works with Slurm's
GNU Parallel achieves this by creating "slots" (set by the
--jobs option), and as jobs in slots are completed, it runs remaining jobs.
Some of the more powerful features are:
- Tracking individual jobs in the
- Pausing and resuming
- Retrying failed jobs
- Reading parameters from files or stdin
- Parsing directory
- Memory control with
- Shebang line for scripts
To understand how to use some of these features, it's well worth spending an hour going through the tutorial.
Using the Parallel module with the scheduler
Clone our Git repository with the setup script and follow the instructions in the README to use GNU Parallel in your submission file: