Difference between revisions of "LS-Dyna Guide"

Revision as of 09:36, 14 March 2016

LS-DYNA

LS-DYNA is an advanced general-purpose multiphysics simulation software package developed by the Livermore Software Technology Corporation (LSTC). While the package continues to contain more and more possibilities for the calculation of many complex, real world problems, its origins and core-competency lie in highly nonlinear transient dynamic finite element analysis (FEA) using explicit time integration. LS-DYNA is being used by the automobile, aerospace, construction, military, manufacturing, and bioengineering industries. LS-DYNA smd version is used on single node multicore and LS-DYNA mpp version is used on multiple nodes.

Group Permission

LS-DYNA on the cluster is owned by Arash Esmaili Zaghi and the license restricts the usage of LS-DYNA to a specific user group. If you want to be granted permission, please contact us through email (hpc@uconn.edu) for more information. We will liaison with the software technical owner, Arash Esmaili Zaghi. If granted, we will add you to user group.

Loading LS-DYNA Modules

There are two versions of LS-DYNA currently available on the cluster. To load the newest issue the command:

module load ls-dyna/R7.1.1

or

module load ls-dyna/R7.1.1_mpp

To make it load automatically upon login issue:

module initadd ls-dyna/<version>

Running LS-DYNA

Running an interactive serial job

To run the LS-DYNA in the interactive mode, please read interactive first.

$ fisbatch -c 12 -p Westmere --exclusive
FISBATCH -- the maximum time for the interactive screen is limited to 6 hours. You can add QoS to overwrite it.
FISBATCH -- waiting for JOBID 52319 to start on cluster=cluster and partition=Westmere
!
FISBATCH -- Connecting to head node (cn04)


[hpc-pei@cn04 ~]$ls-dyna_smp_d_r7_1_1_x64_redhat59_ifort131 
License option : check network license only
     Date: 12/14/2015      Time: 10:32:23  

    ___________________________________________________
    |                                                 |
    |  Livermore  Software  Technology  Corporation   |
    |                                                 |
    |  7374 Las Positas Road                          |
    |  Livermore, CA 94551                            |
    |  Tel: (925) 449-2500  Fax: (925) 449-2507       |
    |  www.lstc.com                                   |
    |_________________________________________________|
    |                                                 |
    |  LS-DYNA, A Program for Nonlinear Dynamic       |
    |  Analysis of Structures in Three Dimensions     |
    |  Version : smp d R7.1.1    Date: 04/04/2014     |
    |  Revision: 88541           Time: 09:35:54       |
    |                                                 |
    |  Features enabled in this version:              |
    |    Shared Memory Parallel                       |
    |    CESE CHEMISTRY EM ICFD STOCHASTIC_PARTICLES  |
    |    FFTW (multi-dimensional FFTW Library)        |
    |    Interactive Graphics                         |
    |    ANSYS Database format                        |
    |    ANSYS License (ANSYS150)                     |
    |                                                 |
    |  Licensed to: University of Connecticut - 1953  |
    |  Issued by  : trent_12112015                    |
    |                                                 |
    |  Platform   : Xeon64 System                     |
    |  OS Level   : Linux 2.6.18 uo                   |
    |  Compiler   : Intel Fortran Compiler 13.1 SSE2  |
    |  Hostname   : cn65                              |
    |  Precision  : Double precision (I8R8)           |
    |  SVN Version: 88833                             |
    |                                                 |
    |  Unauthorized use infringes LSTC copyrights     |
    |_________________________________________________|


 please define input file names or change defaults :
>i=full_input_file_name

[hpc-pei@cn04 ~]$ exit
Connection to cn04 closed.
FISBATCH -- exiting job

Running a parallel job

Firstly, you need a submission script, ls-dyna.sh:

#!/bin/bash
#SBATCH -c 12 # number of the cpus assigns to the job
#SBATCH -p Westmere # specify a partition

ls-dyna_smp_d_r7_1_1_x64_redhat59_ifort131 i=inputfile

Then you can run the parallel program

 sbatch ls-dyna.sh

Running LS-DYNA with LSOPT

Running LS-DYNA on single node

To run LS-DYNA in the interactive mode, please read interactive first.

$ ssh -X <netid>@login3.storrs.hpc.uconn.edu
$ fisbatch -c 12 -p Westmere --exclusive
$ module load ls-dyna/R7.1.1 lsopt
$ cd <directory of *.lsopt>
$ lsoptui *.lsopt

Running LS-DYNA on multiple nodes

To run LS-DYNA in the interactive mode, please read interactive first. Then follow the steps below.

$ ssh -X <netid>@login3.storrs.hpc.uconn.edu
$ fisbatch -N4 --ntasks-per-node=12 -p Westmere   ##(Do not use -c12 here; otherwise, it will only run one process on one node)
$ module load intelics/ifort/11.0.084 mpi/openmpi/1.6.5-ifort11 ls-dyna/R7.1.1_mpp lsopt

To check if executables can work properly, you can use command

$ ldd /apps2/ls-dyna/R7.1.1_mpp/ls-dyna_mpp_d_r7_1_1_88920_x64_redhat54_ifort131_sse2_openmpi165

If there is any library showing 'not found' on the screen, then it won't be working properly, you will need to unload all the modules and load them again.

$ cd <directory of *.lsopt>

Create a script submit_slurm as following:

#!/bin/csh -f
#
# Run jobs on a remote processor, remote disk
set newdir=`pwd | sed -n 's/.*\/\(.*\)\/\(.*\)/\1\/\2/p'`
# Run jobs on a remote processor, local disk (no transmission)
# set newdir=`pwd`
echo $newdir
setenv LSDYNA971_MPP "/apps2/ls-dyna/R7.1.1_mpp/ls-dyna_mpp_d_r7_1_1_88920_x64_redhat54_ifort131_sse2_openmpi165"
setenv LSOPT_WRAPPER "/apps2/lsopt/5.1.1/LSOPT_EXE/wrapper"
cat > dynscr << EOF
#!/bin/csh -f
#
# Define LSDYNA971_MPP environment variables in lsopt input
# or shell command ("setenv").
# $1 represents i=DynaOpt.inp and is automatically
# tagged on as the last argument of the lsopt "solver command".
#
setenv EXE "$LSDYNA971_MPP $1"

setenv LSOPT_HOST $LSOPT_HOST
setenv LSOPT_PORT $LSOPT_PORT
# Run jobs on a remote processor, remote disk
mkdir -p lsopt/$newdir
cd lsopt/$newdir
#
# This actually executes the job
#
$LSOPT_WRAPPER srun --mpi=openmpi \$EXE
EOF
# ============== E N D O F S C R I P T ===================
/bin/csh dynscr

Make sure script submit_slurm is executable. Otherwise, use command 'chmod +x submit_slurm' to make it runnable. This script can be changed according to your own needs.

$ lsoptui < *.lsopt file>

In the screen of lsopt, double click 'Stage' box, 'Setup'->'Command', browse to $ <path to>/submit_slurm; 'Input File', browse to *.k file; setup 'units per job' and 'global limit' as you need; Select 'Use Queuing'->'Slurm'->'OK'. Then click on 'Normal Run'.

LS-DYNA license check

If your LS-DYNA process is pending for a long time, you can check the license usage of ls-dyna. The licenses might be occupied by your previous jobs although those jobs are not running on nodes. It happens sometimes when your previous processes are not ended normally. To check process using license:

$ lstc_qrun

It will show info like this:

                    Running Programs

   User             Host          Program              Started       # procs
-----------------------------------------------------------------------------
hpc-pei    23507@cn19             MPPDYNA          Thu Feb  4 12:53    48

If you want release the occupied 48 licenses above, you can use:

$ lstc_qkill 23507@cn19