Difference between revisions of "Singularity Guide"

From Storrs HPC Wiki
Jump to: navigation, search
(Initial page info box and description)
 
(Add vagrant instructions)
Line 25: Line 25:
  
 
This guide will help you create your own singularity images.
 
This guide will help you create your own singularity images.
 +
 +
Requirements:
 +
* Your computer with macOS, Windows, or a GNU/Linux operating system.
 +
* [https://www.virtualbox.org/ VirtualBox] 1.9 or later to create a virtual machine in which we will install singularity.
 +
* [https://www.vagrantup.com/ Vagrant] to greatly simplify administration of our virtual machine.
 +
* At least 40 GB of disk space.
 +
 +
= Getting started =
 +
 +
To be able to create and edit a singularity image,
 +
you root administrator access on a machine.
 +
Therefore you cannot manage singularity containers directly on the cluster;
 +
you must install your own copy of singularity on your own computer.
 +
Singularity makes use of the Linux kernel to run the container,
 +
therefore you need a copy of GNU/Linux installed for which
 +
we will use a virtual machine.
 +
For best results,
 +
it also helps to use a similar Linux distribution to the environment we will finally be our singularity image.
 +
As our cluster uses RHEL 6.7,
 +
we will create our singularity image using CentOS
 +
which for our purposes is functionally identical to RHEL.
 +
 +
If you are using macOS,
 +
you can install singularity via homebrew as explained on the singularity page [http://singularity.lbl.gov/install-mac] but which we will repeat here for completeness:
 +
 +
<syntaxhighlight lang="bash">
 +
# Only run these commands if you are using macOS!
 +
 +
# Install Brew if you do not have it installed already
 +
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
 +
 +
# The next commands will install Vagrant and the necessary bits
 +
brew cask install virtualbox
 +
brew cask install vagrant
 +
brew cask install vagrant-manager
 +
</syntaxhighlight>
 +
 +
If you are using GNU/Linux,
 +
you can install virtualbox and vagrant through your package manager.
 +
You may need to log out and log in to your system again
 +
for your user account to be recognized as being in the <code>vboxusers</code> group.
 +
 +
Windows users can install [https://www.virtualbox.org/  virtualbox] and [https://www.vagrantup.com/ vagrant] directly from the main websites.
 +
 +
Now that we have the necessary tools,
 +
we can create our CentOS virtual machine in which we will install singularity:
 +
 +
<syntaxhighlight lang="bash">
 +
# Create a working directory for the Vagrant configuration and
 +
# generate a template Vagrantfile for "centos/7"
 +
mkdir vm-singularity
 +
cd vm-singularity
 +
vagrant init centos/7
 +
vagrant box add centos/7
 +
 +
# Build and start the Vagrant hosted VM
 +
vagrant up --provider virtualbox
 +
 +
# Run the necessary commands within the VM to install Singularity
 +
vagrant ssh -c /bin/sh <<EOF
 +
    sudo apt-get update
 +
    sudo apt-get -y install build-essential curl git man autoconf libtool python vim emacs
 +
    git clone https://github.com/singularityware/singularity.git
 +
    cd singularity
 +
    ./autogen.sh
 +
    ./configure --prefix=/usr/local
 +
    make
 +
    sudo make install
 +
EOF
 +
 +
# Singularity is installed in your Vagrant CentOS VM! Now you can
 +
# use Singularity as you would normally by logging into the VM
 +
# directly
 +
vagrant ssh
 +
</syntaxhighlight>
 +
 +
[[Category:Software]]

Revision as of 12:10, 12 May 2017

Singularity
Author Gregory M. Kurtzer and others
Website singularity.lbl.gov
Source GitHub
Category Container, Commandline utility
Help documentation
mailing list
PLoS paper


Singularity allows complex software to be run on the cluster, that would otherwise be difficult or impossible to install. Using singularity, you can generate a single "image" file in which you install all your software, and then you can copy that single image file and run it on the cluster. Additionally, singularity allows more directly comparisons of running software between different clusters, because you can use the exact same versions of software that may not be installed on the different clusters.

This guide will help you create your own singularity images.

Requirements:

  • Your computer with macOS, Windows, or a GNU/Linux operating system.
  • VirtualBox 1.9 or later to create a virtual machine in which we will install singularity.
  • Vagrant to greatly simplify administration of our virtual machine.
  • At least 40 GB of disk space.

Getting started

To be able to create and edit a singularity image, you root administrator access on a machine. Therefore you cannot manage singularity containers directly on the cluster; you must install your own copy of singularity on your own computer. Singularity makes use of the Linux kernel to run the container, therefore you need a copy of GNU/Linux installed for which we will use a virtual machine. For best results, it also helps to use a similar Linux distribution to the environment we will finally be our singularity image. As our cluster uses RHEL 6.7, we will create our singularity image using CentOS which for our purposes is functionally identical to RHEL.

If you are using macOS, you can install singularity via homebrew as explained on the singularity page [1] but which we will repeat here for completeness:

# Only run these commands if you are using macOS!

# Install Brew if you do not have it installed already
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

# The next commands will install Vagrant and the necessary bits
brew cask install virtualbox
brew cask install vagrant
brew cask install vagrant-manager

If you are using GNU/Linux, you can install virtualbox and vagrant through your package manager. You may need to log out and log in to your system again for your user account to be recognized as being in the vboxusers group.

Windows users can install virtualbox and vagrant directly from the main websites.

Now that we have the necessary tools, we can create our CentOS virtual machine in which we will install singularity:

# Create a working directory for the Vagrant configuration and
# generate a template Vagrantfile for "centos/7"
mkdir vm-singularity
cd vm-singularity
vagrant init centos/7
vagrant box add centos/7

# Build and start the Vagrant hosted VM
vagrant up --provider virtualbox

# Run the necessary commands within the VM to install Singularity
vagrant ssh -c /bin/sh <<EOF
    sudo apt-get update
    sudo apt-get -y install build-essential curl git man autoconf libtool python vim emacs
    git clone https://github.com/singularityware/singularity.git
    cd singularity
    ./autogen.sh
    ./configure --prefix=/usr/local
    make
    sudo make install
EOF

# Singularity is installed in your Vagrant CentOS VM! Now you can
# use Singularity as you would normally by logging into the VM
# directly
vagrant ssh