Basic Linear Algebra Subprograms (BLAS) is aspecification that prescribes a set of low-level routines for performing commonlinear algebra operations such asvector addition,scalar multiplication,dot products, linear combinations, andmatrix multiplication. They are thede facto standard low-level routines for linear algebra libraries; the routines have bindings for bothC ("CBLAS interface") andFortran ("BLAS interface"). Although the BLAS specification is general, BLAS implementations are often optimized for speed on a particular machine, so using them can bring substantial performance benefits. BLAS implementations will take advantage of special floating point hardware such as vector registers orSIMD instructions.

It originated as a Fortran library in 1979^[1] and its interface was standardized by the BLAS Technical (BLAST) Forum, whose latest BLAS report can be found on thenetlib website.^[2] This Fortran library is known as thereference implementation (sometimes confusingly referred to asthe BLAS library) and is not optimized for speed but is in thepublic domain.^[3][4]

Most libraries that offer linear algebra routines conform to the BLAS interface, allowing library users to develop programs that are indifferent to the BLAS library being used.

Many BLAS libraries have been developed, targeting various different hardware platforms. Examples includescuBLAS (NVIDIA GPU,GPGPU),rocBLAS (AMD GPU), andOpenBLAS. Examples of CPU-based BLAS library branches include:OpenBLAS,BLIS (BLAS-like Library Instantiation Software), Arm Performance Libraries,^[5]ATLAS, andIntel Math Kernel Library (iMKL). AMD maintains a fork of BLIS that is optimized for theAMD platform.^[6] ATLAS is a portable library that automatically optimizes itself for an arbitrary architecture. iMKL is a freeware^[7] and proprietary^[8] vendor library optimized for x86 and x86-64 with a performance emphasis onIntel processors.^[9] OpenBLAS is an open-source library that is hand-optimized for many of the popular architectures. TheLINPACK benchmarks rely heavily on the BLAS routinegemm for its performance measurements.

Many numerical software applications use BLAS-compatible libraries to do linear algebra computations, includingLAPACK,LINPACK,Armadillo,GNU Octave,Mathematica,^[10]MATLAB,^[11]NumPy,^[12]R,Julia and Lisp-Stat.

TheC++std::linalg library, introduced inC++26, is based on BLAS.

With the advent of numerical programming, sophisticated subroutine libraries became useful. These libraries would contain subroutines for common high-level mathematical operations such as root finding, matrix inversion, and solving systems of equations. The language of choice wasFORTRAN. The most prominent numerical programming library wasIBM'sScientific Subroutine Package (SSP).^[13] These subroutine libraries allowed programmers to concentrate on their specific problems and avoid re-implementing well-known algorithms. The library routines would also be better than average implementations; matrix algorithms, for example, might use full pivoting to get better numerical accuracy. The library routines would also have more efficient routines. For example, a library may include a program to solve a matrix that is upper triangular. The libraries would include single-precision and double-precision versions of some algorithms.

Initially, these subroutines used hard-coded loops for their low-level operations. For example, if a subroutine needed to perform a matrix multiplication, then the subroutine would have three nested loops. Linear algebra programs have many common low-level operations (the so-called "kernel" operations, not related tooperating systems).^[14] Between 1973 and 1977, several of these kernel operations were identified.^[15][16] These kernel operations became defined subroutines that math libraries could call. The kernel calls had advantages over hard-coded loops: the library routine would be more readable, there were fewer chances for bugs, and the kernel implementation could be optimized for speed. A specification for these kernel operations usingscalars andvectors, the level-1 Basic Linear Algebra Subroutines (BLAS), was published in 1979.^[17] BLAS was used to implement the linear algebra subroutine libraryLINPACK.

The BLAS abstraction allows customization for high performance. For example, LINPACK is a general purpose library that can be used on many different machines without modification. LINPACK could use a generic version of BLAS. To gain performance, different machines might use tailored versions of BLAS. As computer architectures became more sophisticated,vector machines appeared. BLAS for a vector machine could use the machine's fast vector operations. (While vector processors eventually fell out of favor, vector instructions in modern CPUs are essential for optimal performance in BLAS routines.)

Other machine features became available and could also be exploited. Consequently, BLAS was augmented from 1984 to 1986 with level-2 kernel operations that concerned vector-matrix operations. Memory hierarchy was also recognized as something to exploit. Many computers havecache memory that is much faster than main memory; keeping matrix manipulations localized allows better usage of the cache. In 1987 and 1988, the level 3 BLAS were identified to do matrix-matrix operations. The level 3 BLAS encouraged block-partitioned algorithms. TheLAPACK library uses level 3 BLAS.^[18]

The original BLAS concerned only densely stored vectors and matrices. Further extensions to BLAS, such as for sparse matrices, have been addressed.^[19]

BLAS functionality is categorized into three sets of routines called "levels", which correspond to both the chronological order of definition and publication, as well as the degree of the polynomial in the complexities of algorithms; Level 1 BLAS operations typically takelinear time,O(n), Level 2 operations quadratic time and Level 3 operations cubic time.^[20] Modern BLAS implementations typically provide all three levels.

This level consists of all the routines described in the original presentation of BLAS (1979),^[1] which defined onlyvector operations onstrided arrays:dot products,vector norms, a generalized vector addition of the form

${\displaystyle \mathit{y}\leftarrow \alpha \mathit{x}+\mathit{y}}$

(called "axpy", "a x plus y") and several other operations.

This level containsmatrix-vector operations including, among other things, ageneralizedmatrix-vector multiplication (gemv):

${\displaystyle \mathit{y}\leftarrow \alpha \mathit{A}\mathit{x}+\beta \mathit{y}}$

as well as a solver forx in the linear equation

${\displaystyle \mathit{T}\mathit{x}=\mathit{y}}$

withT being triangular. Design of the Level 2 BLAS started in 1984, with results published in 1988.^[21] The Level 2 subroutines are especially intended to improve performance of programs using BLAS onvector processors, where Level 1 BLAS are suboptimal "because they hide the matrix-vector nature of the operations from the compiler."^[21]

This level, formally published in 1990,^[20] containsmatrix-matrix operations, including a "generalmatrix multiplication" (gemm), of the form

${\displaystyle \mathit{C}\leftarrow \alpha \mathit{A}\mathit{B}+\beta \mathit{C},}$

whereA andB can optionally betransposed orhermitian-conjugated inside the routine, and all three matrices may be strided. The ordinary matrix multiplicationA B can be performed by settingα to one andC to an all-zeros matrix of the appropriate size.

Also included in Level 3 are routines for computing

${\displaystyle \mathit{B}\leftarrow \alpha {\mathit{T}}^{-1}\mathit{B},}$

whereT is atriangular matrix, among other functionality.

Due to the ubiquity of matrix multiplications in many scientific applications, including for the implementation of the rest of Level 3 BLAS,^[22] and because faster algorithms exist beyond the obvious repetition of matrix-vector multiplication,gemm is a prime target of optimization for BLAS implementers. E.g., by decomposing one or both ofA,B intoblock matrices,gemm can beimplemented recursively. This is one of the motivations for including theβ parameter,^{[dubious –discuss]} so the results of previous blocks can be accumulated. Note that this decomposition requires the special caseβ = 1 which many implementations optimize for, thereby eliminating one multiplication for each value ofC. This decomposition allows for betterlocality of reference both in space and time of the data used in the product. This, in turn, takes advantage of thecache on the system.^[23] For systems with more than one level of cache, the blocking can be applied a second time to the order in which the blocks are used in the computation. Both of these levels of optimization are used in implementations such asATLAS. More recently, implementations byKazushige Goto have shown that blocking only for theL2 cache, combined with carefulamortizing of copying to contiguous memory to reduceTLB misses, is superior toATLAS.^[24] A highly tuned implementation based on these ideas is part of theGotoBLAS,OpenBLAS andBLIS.

A common variation ofgemm is thegemm3m, which calculates a complex product using "three real matrix multiplications and five real matrix additions instead of the conventional four real matrix multiplications and two real matrix additions", an algorithm similar toStrassen algorithm first described by Peter Ungar.^[25]

Accelerate

Apple's framework formacOS andiOS, which includes tuned versions ofBLAS andLAPACK.^[26][27]

Arm Performance Libraries

Arm Performance Libraries, supporting Arm 64-bitAArch64-based processors, available fromArm.^[5]

ATLAS

Automatically Tuned Linear Algebra Software, anopen source implementation of BLASAPIs forC andFortran 77.^[28]

BLIS

BLAS-like Library Instantiation Software framework for rapid instantiation. Optimized for most modern CPUs. BLIS is a complete refactoring of the GotoBLAS that reduces the amount of code that must be written for a given platform.^[29][30]

C++ AMP BLAS

TheC++ AMP BLAS Library is anopen source implementation of BLAS for Microsoft's AMP language extension for Visual C++.^[31]

cuBLAS

Optimized BLAS for Nvidia-based GPU cards, requiring few additional library calls.^[32]

NVBLAS

Optimized BLAS for Nvidia-based GPU cards, providing only Level 3 functions, but as direct drop-in replacement for other BLAS libraries.^[33]

clBLAS

AnOpenCL implementation of BLAS by AMD. Part of the AMD Compute Libraries.^[34]

clBLAST

A tunedOpenCL implementation of most of the BLAS api.^[35]

Eigen BLAS

AFortran 77 andC BLAS library implemented on top of theMPL-licensedEigen library, supportingx86,x86-64,ARM (NEON), andPowerPC architectures.

ESSL

IBM's Engineering and Scientific Subroutine Library, supporting thePowerPC architecture underAIX andLinux.^[36]

GotoBLAS

Kazushige Goto's BSD-licensed implementation of BLAS, tuned in particular forIntelNehalem/Atom,VIANanoprocessor,AMDOpteron.^[37]

GNU Scientific Library

Multi-platform implementation of many numerical routines. Contains a CBLAS interface.

HP MLIB

HP's Math library supportingIA-64,PA-RISC,x86 andOpteron architecture underHP-UX andLinux.

Intel MKL

TheIntelMath Kernel Library, supporting x86 32-bits and 64-bits, available free fromIntel.^[7] Includes optimizations for IntelPentium,Core and IntelXeon CPUs and IntelXeon Phi; support forLinux,Windows andmacOS.^[38]

MathKeisan

NEC's math library, supportingNEC SX architecture underSUPER-UX, andItanium underLinux^[39]

Netlib BLAS

The official reference implementation onNetlib, written inFortran 77.^[40]

Netlib CBLAS

ReferenceC interface to the BLAS. It is also possible (and popular) to call the Fortran BLAS from C.^[41]

OpenBLAS

Optimized BLAS based on GotoBLAS, supportingx86,x86-64,MIPS,ARM andRISC-V processors.^[42]

PDLIB/SX

NEC's Public Domain Mathematical Library for the NECSX-4 system.^[43]

rocBLAS

Implementation that runs onAMD GPUs viaROCm.^[44]

SCSL

SGI's Scientific Computing Software Library contains BLAS and LAPACK implementations for SGI'sIrix workstations.^[45]

Sun Performance Library

Optimized BLAS and LAPACK forSPARC,Core andAMD64 architectures underSolaris 8, 9, and 10 as well as Linux.^[46]

uBLAS

A genericC++ template class library providing BLAS functionality. Part of theBoost library. It provides bindings to many hardware-accelerated libraries in a unifying notation. Moreover, uBLAS focuses on correctness of the algorithms using advanced C++ features.^[47]

Armadillo

Armadillo is a C++ linear algebra library aiming towards a good balance between speed and ease of use. It employs template classes, and has optional links to BLAS/ATLAS and LAPACK. It is sponsored byNICTA (in Australia) and is licensed under a free license.^[48]

LAPACK

LAPACK is a higher level Linear Algebra library built upon BLAS. Like BLAS, a reference implementation exists, but many alternatives like libFlame and MKL exist.

Mir

AnLLVM-accelerated generic numerical library for science and machine learning written inD. It provides generic linear algebra subprograms (GLAS). It can be built on a CBLAS implementation.^[49]

Elemental

Elemental is an open source software fordistributed-memory dense and sparse-direct linear algebra and optimization.^[50]

HASEM

is a C++ template library, being able to solve linear equations and to compute eigenvalues. It is licensed under BSD License.^[51]

LAMA

The Library for Accelerated Math Applications (LAMA) is a C++ template library for writing numerical solvers targeting various kinds of hardware (e.g.GPUs throughCUDA orOpenCL) ondistributed memory systems, hiding the hardware specific programming from the program developer

MTL4

TheMatrix Template Library version 4 is a genericC++ template library providing sparse and dense BLAS functionality. MTL4 establishes an intuitive interface (similar toMATLAB) and broad applicability thanks togeneric programming.

Several extensions to BLAS for handlingsparse matrices have been suggested over the course of the library's history; a small set of sparse matrix kernel routines was finally standardized in 2002.^[52]

The traditional BLAS functions have been also ported to architectures that support large amounts of parallelism such asGPUs. Here, the traditional BLAS functions provide typically good performance for large matrices. However, when computing e.g., matrix-matrix-products of many small matrices by using the GEMM routine, those architectures show significant performance losses. To address this issue, in 2017 a batched version of the BLAS function has been specified.^[53]

Taking the GEMM routine from above as an example, the batched version performs the following computation simultaneously for many matrices:

${\displaystyle \mathit{C}[k]\leftarrow \alpha \mathit{A}[k]\mathit{B}[k]+\beta \mathit{C}[k]\mathrm{\forall }k}$

The index${\displaystyle k}$ in square brackets indicates that the operation is performed for all matrices${\displaystyle k}$ in a stack. Often, this operation is implemented for a strided batched memory layout where all matrices follow concatenated in the arrays${\displaystyle A}$,${\displaystyle B}$ and${\displaystyle C}$.

Batched BLAS functions can be a versatile tool and allow e.g. a fast implementation ofexponential integrators andMagnus integrators that handle long integration periods with many time steps.^[54] Here, thematrix exponentiation, the computationally expensive part of the integration, can be implemented in parallel for all time-steps by using Batched BLAS functions.

• List of numerical libraries
• Math Kernel Library, math library optimized for theIntel architecture; includes BLAS, LAPACK
• Numerical linear algebra, the type of problem BLAS solves

• BLAST Forum (2001-08-21),Basic Linear Algebra Subprograms Technical (BLAST) Forum Standard, Knoxville, TN: University of Tennessee
• Dodson, D. S.; Grimes, R. G. (1982), "Remark on algorithm 539: Basic Linear Algebra Subprograms for Fortran usage",ACM Trans. Math. Softw.,8 (4):403–404,doi:10.1145/356012.356020,S2CID 43081631
• Dodson, D. S. (1983), "Corrigendum: Remark on "Algorithm 539: Basic Linear Algebra Subroutines for FORTRAN usage"",ACM Trans. Math. Softw.,9: 140,doi:10.1145/356022.356032,S2CID 22163977
• J. J. Dongarra, J. Du Croz, S. Hammarling, and R. J. Hanson, Algorithm 656: An extended set of FORTRAN Basic Linear Algebra Subprograms, ACM Trans. Math. Softw., 14 (1988), pp. 18–32.
• J. J. Dongarra, J. Du Croz, I. S. Duff, and S. Hammarling, A set of Level 3 Basic Linear Algebra Subprograms, ACM Trans. Math. Softw., 16 (1990), pp. 1–17.
• J. J. Dongarra, J. Du Croz, I. S. Duff, and S. Hammarling, Algorithm 679: A set of Level 3 Basic Linear Algebra Subprograms, ACM Trans. Math. Softw., 16 (1990), pp. 18–28.

New BLAS

• L. S. Blackford, J. Demmel, J. Dongarra, I. Duff, S. Hammarling, G. Henry, M. Heroux, L. Kaufman, A. Lumsdaine, A. Petitet, R. Pozo, K. Remington, R. C. Whaley, An Updated Set of Basic Linear Algebra Subprograms (BLAS), ACM Trans. Math. Softw., 28-2 (2002), pp. 135–151.
• J. Dongarra, Basic Linear Algebra Subprograms Technical Forum Standard, International Journal of High Performance Applications and Supercomputing, 16(1) (2002), pp. 1–111, and International Journal of High Performance Applications and Supercomputing, 16(2) (2002), pp. 115–199.

• BLAS homepage on Netlib.org
• BLAS FAQ
• BLAS Quick Reference Guide from LAPACK Users' Guide
• Lawson Oral History One of the original authors of the BLAS discusses its creation in an oral history interview. Charles L. Lawson Oral history interview by Thomas Haigh, 6 and 7 November 2004, San Clemente, California. Society for Industrial and Applied Mathematics, Philadelphia, PA.
• Dongarra Oral History In an oral history interview, Jack Dongarra explores the early relationship of BLAS to LINPACK, the creation of higher level BLAS versions for new architectures, and his later work on the ATLAS system to automatically optimize BLAS for particular machines. Jack Dongarra, Oral history interview by Thomas Haigh, 26 April 2005, University of Tennessee, Knoxville TN. Society for Industrial and Applied Mathematics, Philadelphia, PA
• How does BLAS get such extreme performance? Ten naive 1000×1000 matrix multiplications (10¹⁰ floating point multiply-adds) takes 15.77 seconds on 2.6 GHz processor; BLAS implementation takes 1.32 seconds.
• An Overview of the Sparse Basic Linear Algebra Subprograms: The New Standard from the BLAS Technical Forum[2]

• Numerical linear algebra
• Numerical software
• Public-domain software with source code

• Webarchive template wayback links
• All articles with dead external links
• Articles with dead external links from October 2016
• Articles with permanently dead external links
• Articles with dead external links from September 2023
• CS1: unfit URL
• Articles with short description
• Short description matches Wikidata
• Use dmy dates from March 2020
• All accuracy disputes
• Articles with disputed statements from January 2015