SuperLU is a linear algebra package designed to solve large, sparse and nonsymmetric systems of linear equations. SuperLU is designed to solve these types of systems directly using LU Factorization with partial pivoting. The program can run in parallel either using shared-memory (OpenMP) or distributed memory (MPI) and can also run on GPU systems that support CUDA as well.
Using SuperLU on RCC Resources
The distributed version of SuperLU is installed on the listed RCC resources and the library itself does not require the use of a module. Since this library uses Intel Open MPI or Intel MVAPICH2, those modules are required. The Open MPI version of the library is located in the
/opt/hpc/intel/openmpi/lib64/ directory and requires the Open MPI module
intel openmpi. The MVAPICH2 version is located in the
/opt/hpc/intel/mvapich2/lib64/ directory and requires the MVAPICH2 module
The header files for OpenMPI are located in the
/opt/hpc/intel/openmpi/include directory, and the header files for MVAPICH2 are located in the
Here we compile and run an example described in the documentation pdf. Further described on page 59, we compile the PDDRIVE program that solves a linear system. First we create a new directory for SuperLU, then copy the relevant files needed into that folder.
$ mkdir ~/SuperLU $ cp /gpfs/research/software/userfiles/SuperLU_examples/pddrive.c ~/SuperLU/ $ cp /gpfs/research/software/userfiles/SuperLU_examples/dcreate_matrix.c ~/SuperLU/ $ cd ~/SuperLU
While in the SuperLU directory, compile the three source files, taking care to include either the Open MPI or the MVAPICH2 directory. In this example, we use the Open MPI version. Make sure that you load
intel openmpi or
intel mvapich2 modules first.
$ module load intel openmpi $ mpicc -std=c99 -O3 -Wall -w2 -mkl -openmp -DDEBUGlevel=0 -DPRNTlevel=1 -DPROFlevel=0 -DNoChange -DUSE_VENDOR_BLAS -I../SRC -c pddrive.c dcreate_matrix.c
Now link the library files to create the
$ mpicc -openmp pddrive.o dcreate_matrix.o /opt/hpc/intel/mvapich2/lib64/libsuperlu_dist_4.0.a \ -L/opt/hpc/intel/mvapich2/lib64 -lparmetis -L/opt/hpc/intel/lib64 -lmetis -L/gpfs/research/software/intel/mkl/lib/intel64 \ -lmkl_intel_ilp64 -lmkl_core -lmkl_intel_thread -lpthread -lm -o pddrive
Successful execution of this program requires the use of some command-line arguments, described in the comments in the
pddrive.c source file.
Additional source files for examples are available in the
/gpfs/research/software/userfiles/SuperLU_examples/ directory, and would be compiled in a similar manner. Any additional files would need to be downloaded from the SuperLU home page. Download the the 4.0 version (this is the distributed version) using the command
After extracting the files, some potentially necessary/helpful source files will be located in the
SRC directory. The library files will not be available, however.
The library files would need to be compiled, but we can instead use the library files already compiled and available on the HPC, using a method similar to that used in the linking stage described above.