LAMMPS from scratch · 20 min

Running in parallel

LAMMPS runs on many processors by cutting the box into bricks, one per processor. You predict by hand which grid of bricks LAMMPS chooses, how many atoms and ghost atoms each processor holds, and why five processors get uneven shares, and check every number against real mpirun runs. Then you read the speed-up and the cost of communication from the timing table.

1 · Understand: one box, many bricks

Run mpirun -np 4 lmp -in in.melt and four copies of LAMMPS start. Each one gets a brick of the box (src/domain.cpp:371: sublo = boxlo + xprd·xsplit), stores only the atoms inside it plus ghosts around it (page B.6), and computes their forces. Every step they talk through MPI (src/verlet.cpp:264):

WhenCallWhat moves between processors
every stepcomm->forward_comm() (src/verlet.cpp:268)new positions of the atoms other processors hold as ghosts
every stepcomm->reverse_comm() (src/verlet.cpp:341)forces computed on ghosts, sent back to their owners (newton on)
when the neighbour list is rebuiltcomm->exchange() (src/verlet.cpp:284)atoms that left a brick, handed to the new owner
samecomm->borders() (src/verlet.cpp:286)the new list of ghosts, as on page B.6

The force loop itself is the same code as in serial: each processor just loops over fewer atoms. The browser runs one processor only, so the parallel numbers on this page were measured with real mpirun on a 2-core Linux computer (LAMMPS 22 Jul 2025, MPICH 5.0.2) and are matched by hand below.

2 · Predict the processor grid

For P processors LAMMPS tries every px × py × pz = P and keeps the one where the bricks have the smallest surface, because the surface is where ghosts live and communication happens (src/procmap.cpp:875). For a cube of side L the surface per brick, divided by L², is 1/(px·py) + 1/(px·pz) + 1/(py·pz). A tie keeps the first one found (surf < bestsurf, src/procmap.cpp:878).

3 · Match: atoms and ghosts on each processor

At step 0 the atoms sit on 20 planes per direction, 0.8397981 apart, 200 atoms per plane. Along each split direction, a brick owns the planes between its two walls; its ghosts are the planes within the communication cutoff (2.5 + 0.3 = 2.8) beyond each wall, taken from the neighbouring bricks (or periodic images). Count the FCC sites in the grown brick, subtract the owned atoms: that is Nghost (page B.6).

4 · Observe: is it faster?

32 000 atoms (20 cells per side), 500 steps, on the same 2-core computer:

Ploop time (s)speed-up = t₁/t_Pefficiency = speed-up/PPair %Comm %
16.103931.00100 %80.581.26
23.167321.9396 %76.057.31
44.012851.5238 %41.5237.66

On your own computer

mpirun -np 4 lmp -in in.melt                 # LAMMPS picks the grid
# force a grid in the script, before create_box:
processors      4 1 1

The log prints the grid (1 by 2 by 2 MPI processor grid), and Nlocal and Nghost with ave, max and min over the processors. A large gap between max and min means uneven work: the slowest processor sets the speed (the balance command moves the walls to even it out; page F.5 covers performance).

Your turn

1. Which grid does LAMMPS choose for 6 processors, and what is its surface?

2. With 3 processors, why do two get 1400 atoms and one 1200?

3. You have a 32-core node and 4000 atoms. Is mpirun -np 32 a good idea?