Overview

PerfSuite is an easy-to-use collection of tools, utilities, and libraries to support application software performance analysis on Linux-based systems (x86, x86-64, and ia64). It includes components to assist with a wide variety of performance-related tasks, ranging from assistance with compiler optimization reports to hardware performance counting, profiling, and MPI usage summarization.

PerfSuite is Open Source software, approved for licensing under the University of Illinois/NCSA Open Source License (OSI-approved).

You can find out more about PerfSuite at the project web sites, located at:

http://perfsuite.ncsa.uiuc.edu/
http://perfsuite.sourceforge.net/

The project web sites contain pointers to online documentation for PerfSuite and its individual components. Links to download pages for the current PerfSuite source distribution are also available at the project web sites.

NCSA-specific information

PerfSuite software is available for use on the Abe, Tungsten and Mercury Linux clusters and the Cobalt SGI Altix system.

On Abe, Tungsten and Cobalt, you can find PerfSuite software in the directory:

/usr/apps/tools/perfsuite

On Mercury, PerfSuite is available in the directory:

/usr/projects/perfsuite

To use PerfSuite on NCSA Linux systems, convenience scripts are available that modify your environment to include the proper PATH, LD_LIBRARY_PATH, and MANPATH settings. They are named psenv.sh and psenv.csh (for Bourne shell and C shell users, respectively) and are located in the bin subdirectory of the installation.

Examples of use:

Bourne shell (bash):: $ source /usr/apps/tools/perfsuite/bin/psenv.sh
C shell (tcsh):: % source /usr/apps/tools/perfsuite/bin/psenv.csh

If you experience problems when issuing the source command as shown above, please ensure that the psenv file name argument supplied has the proper suffix (".sh" for bash, ".csh" for tcsh).

The following UNIX man pages are available: psconfig, psenv, psinv, psprocess, psrun.

Using SoftEnv for PerfSuite

On Abe, Cobalt, Mercury, and Tungsten, PerfSuite is also available through the "SoftEnv" package. To set up your environment for PerfSuite with SoftEnv, use the following command:

soft add +perfsuite

If you are using the PerfSuite API directly from a Fortran program and you are compiling with the Intel compiler, you should substitute soft add +perfsuite-intel. You'll also want to link with Intel-compatible versions of PerfSuite that are in the directory /usr/apps/tools/perfsuite/lib/intel. Note that basic use of the PerfSuite command-line tools such as psrun can use either version.

You can arrange for PerfSuite to be included in your default environment by adding the line +perfsuite to the file $HOME/.soft

Please refer to NCSA's general user documentation for more information about using SoftEnv. If you use SoftEnv with PerfSuite, there is no need to use the psenv scripts described above.

Using psrun with mpirun

In most cases, you can use psrun with the MPI launch command mpirun on NCSA HPC platforms without modification. If you experience problems, you may find that using the full pathname to psrun will help. That is, try changing the command:

% mpirun -machinefile $MACHINEFILE -np 2 psrun mpiprogram

to:

% mpirun -machinefile $MACHINEFILE -np 2 /usr/local/bin/psrun mpiprogram

Note: you will want to change "/usr/local/bin" to the actual directory containing psrun on the system you are using.

If you are using the MPICH "ch_p4" device, you will not be able to use psrun with mpirun. This is because command-line arguments are changed by this variant of MPI. If you have questions, please refer to the contact information elsewhere on this page.

Using psrun with parallel programs on Cobalt

To use PerfSuite's psrun command with parallel programs on the SGI Altix (Cobalt), you should follow these steps:

For shared-memory (OpenMP/POSIX threads) programs:: psrun -p ompprogram
For shared-memory programs with dplace:: dplace [dplace-options] psrun -p ompprogram
For message-passing (MPI) programs:: mpirun -np P psrun -f mpiprogram
For hybrid (MPI + OpenMP) programs:: mpirun -np P psrun -f -p hybridprogram

Note: when using psrun with dplace, an additional process (psrun) must be accounted for when providing a "skip mask" through the dplace option -x. Please refer to the dplace man page for details.

psrun and the Intel compiler option -fast

Important note: the psrun command is only effective for executables that have been linked dynamically (the default linking mode). Statically-linked executables are produced when you add the option -static to your link line. The Intel compiler's -fast option, which selects a set of options likely to produce good performance, includes the -static option when your program is linked. Therefore, applications linked using this option will not produce performance data when run with psrun.

You can determine the option set implied by -fast by examining the "help" output of the compiler (-help): this will allow you to compile and link your application using the same optimization options as -fast (typically -O3 and -ipo) while omitting the option -static in order to produce a dynamically-linked executable that can be used with psrun.

For more information

If you have questions about or need assistance with PerfSuite at NCSA, please send email to:

consult (at) ncsa.uiuc.edu