Build HONEE on Summit

Building HONEE on Summit nodes

Overview

This page describes a Summit-specific workflow to build and test HONEE (main branch) on Summit compute nodes. HONEE depends on libCEED and PETSc.

• HONEE repo: https://gitlab.com/phypid/honee
• libCEED repo: https://github.com/CEED/libCEED
• PETSc repo: https://gitlab.com/petsc/petsc

Note: This guide assumes you build inside an interactive PBS job on a Summit compute node.

1) Start an interactive Summit node

From jumpgate, request a node:

qsub -I -l select=1:ncpus=24:mpiprocs=24 -l walltime=72:00:00 -l place=scatter:exclhost -q workq

You should land on a Summit node.

2) Create a single workspace for all repositories

Create a folder to hold all three code bases:

mkdir honee_stack
cd honee_stack

Recommended layout:

~/honee_stack/
  libCEED/
  petsc/
  honee/

3) Clone the repositories

Navigate into honee_stack and clone the 3 repos.

libCEED

git clone https://github.com/CEED/libCEED

PETSc (main branch)

Important: Use PETSc main, not the release branch.

git clone https://gitlab.com/petsc/petsc

HONEE (main branch)

git clone https://gitlab.com/phypid/honee

4) Build libCEED

Navigate into the libceed directory cd ~/honee_stack/libCEED

make -j

This produces libCEED libraries needed by HONEE.

5) Build PETSc

Navigate into the PETSc directory cd ~/honee_stack/PETSc

Example configuration

Below is a good baseline: MPI enabled, optimized build, and CGNS/HDF5 support via downloads.

./configure \
  --with-cc=mpicc \
  --with-cxx=mpicxx \
  --with-fc=0 \
  --with-mpi=1 \
  --with-mpiexec=mpirun \
  --download-f2cblaslapack \
  --download-hdf5 \
  --download-cgns \
  --with-debugging=0 \
  COPTFLAGS='-O3 -g --march=native' \
  CXXOPTFLAGS='-O3 -g --march=native'

Then build:

make -j all

Notes on the example PETSc settings

• --with-debugging=0 is recommended for performance runs.
• -O3 is appropriate for speed; keeping -g helps debugging without major runtime cost.
• Downloading hdf5 + cgns is recommended for CGNS output support.

6) Set environment variables for HONEE

HONEE requires CEED_DIR, PETSC_DIR, and PETSC_ARCH.

Example setup:

export CEED_DIR=$HOME/honee_stack/libCEED
export PETSC_DIR=$HOME/honee_stack/petsc
export PETSC_ARCH=arch-linux-c-opt

Important runtime note (recommended while using the example PETSc config):

export LD_LIBRARY_PATH=$PETSC_DIR/$PETSC_ARCH/lib:$LD_LIBRARY_PATH

7) Build HONEE

cd ~/honee_stack/honee
make -j

8) Run a sample problem

Run the Gaussian wave example:

cd ~/honee_stack/honee
build/navierstokes -options_file examples/gaussianwave.yaml

9) Test the installation

Run the test suite:

cd ~/honee_stack/honee
make test -j

Troubleshooting

Missing PETSc or libCEED

Verify environment variables:

echo $CEED_DIR
echo $PETSC_DIR
echo $PETSC_ARCH

Runtime linker errors (libpetsc.so not found)

Ensure:

export LD_LIBRARY_PATH=$PETSC_DIR/$PETSC_ARCH/lib:$LD_LIBRARY_PATH

CGNS/HDF5 issues

Reconfigure PETSc with:

--download-hdf5 --download-cgns

Reference

Official HONEE installation documentation:
https://gitlab.com/phypid/honee

Official PETSc configuration documentation:
https://petsc.org/main/install/