| NAS Parallel Benchmarks | |
|---|---|
| Original author | NASA Numerical Aerodynamic Simulation Program |
| Developer | NASA Advanced Supercomputing Division |
| Initial release | 1991 (1991) |
| Stable release | 3.4 |
| Website | nas |
NAS Parallel Benchmarks (NPB) are a set ofbenchmarks targeting performance evaluation of highlyparallelsupercomputers. They are developed and maintained by theNASAAdvanced Supercomputing (NAS) Division (formerly the NASA Numerical Aerodynamic Simulation Program) based at theNASA Ames Research Center. NAS solicits performance results for NPB from all sources.[1]
Traditional benchmarks that existed before NPB, such as theLivermore loops, theLINPACK Benchmark and theNAS Kernel Benchmark Program, were usually specialized for vector computers. They generally suffered from inadequacies including parallelism-impeding tuning restrictions and insufficient problem sizes, which rendered them inappropriate for highly parallel systems. Equally unsuitable were full-scale application benchmarks due to high porting cost and unavailability of automatic software parallelization tools.[2] As a result, NPB were developed in 1991[3] and released in 1992[4] to address the ensuing lack of benchmarks applicable to highly parallel machines.
The first specification of NPB recognized that the benchmarks should feature
In the light of these guidelines, it was deemed the only viable approach to use a collection of "paper-and-pencil" benchmarks that specified a set of problems only algorithmically and left most implementation details to the implementer's discretion under certain necessary limits.
NPB 1 defined eight benchmarks, each in two problem sizes dubbedClass A andClass B. Sample codes written inFortran 77 were supplied. They used a small problem sizeClass S and were not intended for benchmarking purposes.[2]
Since its release, NPB 1 displayed two major weaknesses. Firstly, due to its "paper-and-pencil" specification, computer vendors usually highly tuned their implementations so that their performance became difficult for scientific programmers to attain. Secondly, many of these implementation were proprietary and not publicly available, effectively concealing their optimizing techniques. Secondly, problem sizes of NPB 1 lagged behind the development of supercomputers as the latter continued to evolve.[3]
NPB 2, released in 1996,[5][6] came with source code implementations for five out of eight benchmarks defined in NPB 1 to supplement but not replace NPB 1. It extended the benchmarks with an up-to-date problem sizeClass C. It also amended the rules for submitting benchmarking results. The new rules included explicit requests for output files as well as modified source files and build scripts to ensure public availability of the modifications and reproducibility of the results.[3]
NPB 2.2 contained implementations of two more benchmarks.[5] NPB 2.3 of 1997 was the first complete implementation inMPI.[4] It shipped with serial versions of the benchmarks consistent with the parallel versions and defined a problem sizeClass W for small-memory systems.[7] NPB 2.4 of 2002 offered a new MPI implementation and introduced another still larger problem sizeClass D.[6] It also augmented one benchmark withI/O-intensive subtypes.[4]
NPB 3 retained the MPI implementation from NPB 2 and came in more flavors, namelyOpenMP,[8]Java[9] andHigh Performance Fortran.[10] These new parallel implementations were derived from the serial codes in NPB 2.3 with additional optimizations.[7] NPB 3.1 and NPB 3.2 added three more benchmarks,[11][12] which, however, were not available across all implementations; NPB 3.3 introduced aClass E problem size.[7] Based on the single-zone NPB 3, a set of multi-zone benchmarks taking advantage of the MPI/OpenMP hybrid programming model were released under the nameNPB-Multi-Zone (NPB-MZ) for "testing the effectiveness of multi-level and hybrid parallelization paradigms and tools".[1][13]
As of NPB 3.3, eleven benchmarks are defined as summarized in the following table.
| Benchmark | Name derived from[2] | Available since | Description[2] | Remarks |
|---|---|---|---|---|
| MG | MultiGrid | NPB 1[2] | Approximate the solution to a three-dimensionaldiscrete Poisson equation using the V-cyclemultigrid method | |
| CG | ConjugateGradient | Estimate the smallesteigenvalue of a largesparsesymmetricpositive-definite matrix using theinverse iteration with theconjugate gradient method as a subroutine for solvingsystems of linear equations | ||
| FT | FastFourierTransform | Solve a three-dimensionalpartial differential equation (PDE) using thefast Fourier transform (FFT) | ||
| IS | IntegerSort | Sort small integers using thebucket sort[5] | ||
| EP | EmbarrassinglyParallel | Generate independentGaussianrandom variates using theMarsaglia polar method | ||
| BT | BlockTridiagonal | Solve a synthetic system ofnonlinear PDEs using three different algorithms involvingblock tridiagonal, scalarpentadiagonal and symmetricsuccessive over-relaxation (SSOR) solver kernels, respectively | ||
| SP | ScalarPentadiagonal[6] | |||
| LU | Lower-Upper symmetricGauss-Seidel[6] | |||
| UA | UnstructuredAdaptive[11] | NPB 3.1[7] | SolveHeat equation with convection and diffusion from moving ball. Mesh is adaptive and recomputed at every 5th step. | |
| DC | DataCube operator[12] | |||
| DT | DataTraffic[7] | NPB 3.2[7] |