LAMMPS

From arccwiki
Jump to: navigation, search

Homepage: LAMMPS: Molecular Dynamics Simulator: LAMMPS is a classical molecular dynamics code with a focus on materials modeling. It's an acronym for Large-scale Atomic/Molecular Massively Parallel Simulator. LAMMPS has potentials for solid-state materials (metals, semiconductors) and soft matter (biomolecules, polymers) and coarse-grained or mesoscopic systems. It can be used to model atoms or, more generically, as a parallel particle simulator at the atomic, meso, or continuum scale.

  • Documentation
  • LAMMPS is built with a number of packages. If a required package isn't installed please contact ARCC and request what you need.
    • Latest Version 20200324: Built with packages: CPM, KSPACE, MANYBODY, MOLECULE, RIGID, USER-ADIOS, USER-ATC, USER-MEAMC, USER-REAXC and VORONOI


Update History:

    • Version 20190807: has been built with packages: MOLECULE and USER-REAXC
    • Version 20200227: has been built with packages: MANYBODY, MOLECULE, VORONOI, USER-ATC, USER-MEAMC, USER-REAXC and USER-ADIOS
    • Version 20200305: extends version 20200227 with additional packages: KSPACE and Constant Potential Method
    • Version 20200324: extends version 20200305 with additional package: RIGID

Module: Example

[]$ module spider LAMMPS
------------------------------------------------------------
  lammps:
------------------------------------------------------------
     Versions:
        lammps/2016.02
        lammps/20190807
        lammps/20200227
        lammps/20200305
        lammps/20200324
------------------------------------------------------------
  For detailed information about a specific "lammps" module (including how to load the modules) use the module's full name.
  For example:
     $ module spider lammps/20200324
------------------------------------------------------------
[]$ module load lammps/20200324

Using:

  • LAMMPS can be run to use OpenMP and MPI functionality across the teton cluster.
  • Below are some bare bone example batch files demonstrating various scenarios.
  • You will need to complete with appropriate job-name, time, mail, output and account related parameters:
  • Notice below that the two difference versions of 20190807 versus 20200227 require different methods to run.

Version: 20190807

Single Node with 16 Tasks:
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=16

module load lammps/20190807

mpirun -np 16 lmp -in SC.in

Within the output you will read:

using 1 OpenMP thread(s) per MPI task
2 by 2 by 4 MPI processor grid
16 MPI tasks x 1 OpenMP threads
Two Nodes with 32 Tasks:
#SBATCH --nodes=2
#SBATCH --ntasks-per-node=32

module load lammps/20190807

mpirun -np 64 lmp -in SC.in

Within the output you will read:

using 1 OpenMP thread(s) per MPI task
4 by 4 by 4 MPI processor grid
64 MPI tasks x 1 OpenMP threads
Four Nodes with each Node Running 16 Tasks, with 2 OpenMP Threads per Task:
  • Each node has 16 tasks/MPI processes running. Each MPI process has two OpenMP threads.
  • Each task requires two cores, one for each of the two OpenMP threads.
  • Each node is using 16 x 2 = 32 cores.
#SBATCH --nodes=4
#SBATCH --ntasks-per-node=16
#SBATCH --cpus-per-task=2

export OMP_NUM_THREADS=2

echo "SLURM_JOB_NUM_NODES:" $SLURM_JOB_NUM_NODES
echo "SLURM_CPUS_PER_TASK:" $SLURM_CPUS_PER_TASK
echo "SLURM_CPUS_ON_NODE:" $SLURM_CPUS_ON_NODE

module load lammps/20190807

TOTAL_NP=$(($SLURM_JOB_NUM_NODES*$SLURM_CPUS_ON_NODE/$SLURM_CPUS_PER_TASK))
echo "Total NPs: " $TOTAL_NP

mpirun -np $TOTAL_NP lmp -in SC.in

Within the output you will read:

using 2 OpenMP thread(s) per MPI task
4 by 4 by 4 MPI processor grid
64 MPI tasks x 2 OpenMP threads
Four Nodes with each Node Running 8 Tasks, with 4 OpenMP Threads per Task:
#SBATCH --nodes=4
#SBATCH --ntasks-per-node=8
#SBATCH --cpus-per-task=4

export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK

module load lammps/20190807

TOTAL_NP=$(($SLURM_JOB_NUM_NODES*$SLURM_CPUS_ON_NODE/$SLURM_CPUS_PER_TASK))
echo "Total NPs: " $TOTAL_NP

mpirun -np $TOTAL_NP lmp -in SC.in

Within the output you will read:

using 4 OpenMP thread(s) per MPI task
4 by 2 by 4 MPI processor grid
32 MPI tasks x 4 OpenMP threads

Version: 20200227 / 20200305 / 20200324

These versions use exactly the same #SBATCH configuration and will generate the same expected output.
Note: The difference is in how you call LAMMPS. Instead of using mpirun you will use srun. Regardless of the node/task/cpu configuration you choose, in every case you will call LAMMPS using the following:

module load lammps/20200227

srun lmp -in SC.in

General Rules:

  • You have to request, allocate and match the number of MPI slots for LAMMPS.
  • Use the three parameters: nodes, ntasks-per-node and cpus-per-task:
    • nodes x ntasks-per-node defines the number of slots.
    • cpus-per-task defines the number of OpenMP threads.
  • Make sure you understand the core structure of the partition you use. In general, teton nodes have 32 cores.
  • Define an appropriate configuration that can be allocated on your chosen partition. For example, on teton ntasks-per-node and cpus-per-task values of 8 x 4, 16 x 2, 32 x 1 should work, but 6 x 6 will fail since it requires 36 cores on a node.
  • By default only one OpenMP thread is used. If you wish to increase this you must define and set the OMP_NUM_THREADS environment variable.
  • You will need to experiment/analyse what node and task configuration works best for your data. Following the rules above simply gives you a starting point.
  • Do NOT assume more nodes will automatically make your simulation run faster. For one data set, the following four different configurations all took 31 minutes to complete, with only a 30 second difference between them:
    • nodes=4 / ntasks-per-node=16 / cpus-per-task=2
    • nodes=4 / ntasks-per-node=8 / cpus-per-task=4
    • nodes=4 / ntasks-per-node=4 / cpus-per-task=8
    • nodes=8 / ntasks-per-node=8 / cpus-per-task=4
  • Please don't hesitate to contact arcc with any questions.

Back to HPC Installed Software