Singularity Summary and Tips
How to create singularity recipe file
Three important sections should be defined in recipe file.
1: BootStrap: in this section you can figure out use which method (library, docker, debootstrap/yum) to get which OS (ubuntu, centos, redhat) with which version (trusty, xenial) from where (singularity hub, docker hub, ubuntu/centos official website) and install which libraries (bash vim less) by default.
Example 1 BootStrap: debootstrap OSVersion: xenial MirrorURL: http://us.archive.ubuntu.com/ubuntu/ Include: bash vim less man-db apt-utils tzdata --------- Example 2 BootStrap: docker From: ubuntu:xenial
2: %environment: in this section, you can figure out the language and location of OS. inaddition, you an export some paths into environment variables inside singularity container.
%environment LC_ALL=C LD_LIBRARY_PATH=/usr/local/lib64:/usr/local/lib:/usr/lib64:/usr/lib:/lib64:/lib:/usr/local/cuda/extras/CUPTI/lib64 TZ='America/New_York' export LC_ALL LD_LIBRARY_PATH TZ
3: %post: in this section, you can figure out all customized script that be used to install the specific libraries, software and packages.
%post #Get 3DSlicer wget -P /home/software/3DSlicer https://download.slicer.org/bitstream/1023242 tar -xvf /home/software/3DSlicer/1023242 -C /home/software/3DSlicer #For 3DSlicer apt-get install -y libfontconfig1 \ libxrender1 \ libgl1-mesa-dev \ libglu1-mesa \ libglu1-mesa-dev \ libxtst6 \ libegl1-mesa-dev
(1)The singularity recipe file for 3DSlicer: https://github.com/lxwgcool/singularity/blob/master/singularity_3DSlicer.recipe (2)The singularity recipe file for tensorflow with ubuntu xenial, python 3.6.7 and cuda 9.0: https://github.com/lxwgcool/singularity/blob/master/Singularity.tensorflow-gpu-1.12.0
How to create singularity container
1: There are two types of container, including "image" and "sandbox".
2: Any singularity container should be created by using "sudo".
Now, we use the recipe file of 3D Slicer as an example: https://github.com/lxwgcool/singularity/blob/master/singularity_3DSlicer.recipe
sudo singularity build ./3DSlicer.ssb ../singularity_3DSlicer.recipe Advantage 1: Singularity image container is a binary file. 2: The size of image container is small. Disadvantage 1: Hard to be expanded (size is fixed). 2: You are not allowed to install new packages and libraries once the image been created.
sudo singularity build --sandbox ./3DSlicer.ssb ../singularity_3DSlicer.recipe Advantage 1: It is very easy to be expanded (no size limitation). 2: You can install any additional packages and libraries after the container been created. 3: Sandbox is very easy to be converted to image by using the command line below: sudo singularity build 3DSlicer.sif 3DSlicer.ssb/ Disadvantage 1: Singularity sandbox container is a folder. 2: The size of sandbox container is around 3 times larger than image container.
Recommendation: since the storage is not a big issue for modern computer, sandbox is good choice to used when build singularity container.
How to run singularity container in HPC Cluster
Usually, the target