Large sparse linear systems of equations arise in many computational science and engineering disciplines. One common scenario occurs when implicit methods are used for time-dependent partial differential equations, and a nonlinear system of equations has to be solved at each time step. With a Newton-type approach, this in turn leads to a linear system at each Newton iteration. In such situations, the linear solver can account for a large part of the overall simulation time. Demonstrating the effectiveness of linear solvers on large Linux clusters will help establish the usefulness of these platforms for applications in such diverse areas as CFD, supernova simulation, Bio-physics, earthquake engineering, and structural mechanics.
In this paper, we investigate the effectiveness of Linux clusters for state of the art software for parallel linear solvers such as PETSc and hypre. In particular, we study scaling to large processor counts, and large system sizes. The scalability of the algorithms, of the implementation, and of the cluster architecture are investigated. We focus on iterative Krylov subspace solvers like Conjugate Gradient, GMRES(k) and BiCGStab, coupled with parallel preconditioners, such as block Jacobi, domain decomposition methods (such as Additive Schwartz), and multigrid methods.
The structure of the remainder of this paper is as follows. We describe the Linux clusters at NCSA that were used in the tests reported here. The results are primarily for a Pentium III cluster, using up to 256 processors. Some data is included for an Itanium 1 Linux cluster, and for the SGI Origin 2000. We give a brief over view of the methods used in the scaling studies, namely Krylov subspace methods and multigrid methods, and of the software used in the study, PETSc and hypre. After describing the problems used in the symmetric and non-symmetric scaling tests, we present the results of our tests, for these two cases. In the process of investigating the scalability of linear solvers on Linux clusters, we include some comparisons of different methods and a comparison across architectures. We end with a summary of our findings, and future directions for investigation.