In three-dimensionalEuclidean space, these three planes represent solutions to linear equations, and their intersection represents the set of common solutions: in this case, a unique point. The blue line is the common solution to two of these equations.
Linear algebra is central to almost all areas of mathematics. For instance, linear algebra is fundamental in modern presentations ofgeometry, including for defining basic objects such aslines,planes androtations. Also,functional analysis, a branch ofmathematical analysis, may be viewed as the application of linear algebra tofunction spaces.
Linear algebra is also used in most sciences and fields ofengineering because it allowsmodeling many natural phenomena, and computing efficiently with such models. Fornonlinear systems, which cannot be modeled with linear algebra, it is often used for dealing withfirst-order approximations, using the fact that thedifferential of amultivariate function at a point is the linear map that best approximates the function near that point.
Systems of linear equations arose in Europe with the introduction in 1637 byRené Descartes ofcoordinates ingeometry. In fact, in this new geometry, now calledCartesian geometry, lines and planes are represented by linear equations, and computing their intersections amounts to solving systems of linear equations.
The first systematic methods for solving linear systems useddeterminants and were first considered byLeibniz in 1693. In 1750,Gabriel Cramer used them for giving explicit solutions of linear systems, now calledCramer's rule. Later,Gauss further described the method of elimination, which was initially listed as an advancement ingeodesy.[5]
In 1844Hermann Grassmann published his "Theory of Extension" which included foundational new topics of what is today called linear algebra. In 1848,James Joseph Sylvester introduced the termmatrix, which is Latin forwomb.
Linear algebra grew with ideas noted in thecomplex plane. For instance, two numbersw andz in have a differencew –z, and the line segmentswz and0(w −z) are of the same length and direction. The segments areequipollent. The four-dimensional system ofquaternions was discovered byW.R. Hamilton in 1843.[6] The termvector was introduced asv =xi +yj +zk representing a point in space. The quaternion differencep –q also produces a segment equipollent topq. Otherhypercomplex number systems also used the idea of a linear space with abasis.
Arthur Cayley introducedmatrix multiplication and theinverse matrix in 1856, making possible thegeneral linear group. The mechanism ofgroup representation became available for describing complex and hypercomplex numbers. Crucially, Cayley used a single letter to denote a matrix, thus treating a matrix as an aggregate object. He also realized the connection between matrices and determinants and wrote "There would be many things to say about this theory of matrices which should, it seems to me, precede the theory of determinants".[5]
The first modern and more precise definition of a vector space was introduced byPeano in 1888;[5] by 1900, a theory of linear transformations of finite-dimensional vector spaces had emerged. Linear algebra took its modern form in the first half of the twentieth century when many ideas and methods of previous centuries were generalized asabstract algebra. The development of computers led to increased research in efficientalgorithms for Gaussian elimination and matrix decompositions, and linear algebra became an essential tool for modeling and simulations.[5]
Until the 19th century, linear algebra was introduced throughsystems of linear equations andmatrices. In modern mathematics, the presentation throughvector spaces is generally preferred, since it is moresynthetic, more general (not limited to the finite-dimensional case), and conceptually simpler, although more abstract.
A vector space over afieldF (often the field of thereal numbers or of thecomplex numbers) is asetV equipped with twobinary operations.Elements ofV are calledvectors, and elements ofF are calledscalars. The first operation,vector addition, takes any two vectorsv andw and outputs a third vectorv +w. The second operation,scalar multiplication, takes any scalara and any vectorv and outputs a newvectorav. The axioms that addition and scalar multiplication must satisfy are the following. (In the list below,u,v andw are arbitrary elements ofV, anda andb are arbitrary scalars in the fieldF.)[8]
The first four axioms mean thatV is anabelian group under addition.
The elements of a specific vector space may have various natures; for example, they could betuples,sequences,functions,polynomials, or amatrices. Linear algebra is concerned with the properties of such objects that are common to all vector spaces.
Linear maps aremappings between vector spaces that preserve the vector-space structure. Given two vector spacesV andW over a fieldF, a linear map (also called, in some contexts, linear transformation or linear mapping) is amap
that is compatible with addition and scalar multiplication, that is
for any vectorsu,v inV and scalara inF.
An equivalent condition is that for any vectorsu,v inV and scalarsa,b inF, one has
.
WhenV =W are the same vector space, a linear mapT :V →V is also known as alinear operator onV.
Abijective linear map between two vector spaces (that is, every vector from the second space is associated with exactly one in the first) is anisomorphism. Because an isomorphism preserves linear structure, two isomorphic vector spaces are "essentially the same" from the linear algebra point of view, in the sense that they cannot be distinguished by using vector space properties. An essential question in linear algebra is testing whether a linear map is an isomorphism or not, and, if it is not an isomorphism, finding itsrange (or image) and the set of elements that are mapped to the zero vector, called thekernel of the map. All these questions can be solved by usingGaussian elimination or some variant of thisalgorithm.
The study of those subsets of vector spaces that are in themselves vector spaces under the induced operations is fundamental, similarly as for many mathematical structures. These subsets are calledlinear subspaces. More precisely, a linear subspace of a vector spaceV over a fieldF is asubsetW ofV such thatu +v andau are inW, for everyu,v inW, and everya inF. (These conditions suffice for implying thatW is a vector space.)
For example, given a linear mapT :V →W, theimageT(V) ofV, and theinverse imageT−1(0) of0 (calledkernel or null space), are linear subspaces ofW andV, respectively.
Another important way of forming a subspace is to considerlinear combinations of a setS of vectors: the set of all sums
wherev1,v2, ...,vk are inS, anda1,a2, ...,ak are inF form a linear subspace called thespan ofS. The span ofS is also the intersection of all linear subspaces containingS. In other words, it is the smallest (for the inclusion relation) linear subspace containingS.
A set of vectors islinearly independent if none is in the span of the others. Equivalently, a setS of vectors is linearly independent if the only way to express the zero vector as a linear combination of elements ofS is to take zero for every coefficientai.
A set of vectors that spans a vector space is called aspanning set orgenerating set. If a spanning setS islinearly dependent (that is not linearly independent), then some elementw ofS is in the span of the other elements ofS, and the span would remain the same if one were to removew fromS. One may continue to remove elements ofS until getting alinearly independent spanning set. Such a linearly independent set that spans a vector spaceV is called abasis ofV. The importance of bases lies in the fact that they are simultaneously minimal-generating sets and maximal independent sets. More precisely, ifS is a linearly independent set, andT is a spanning set such thatS ⊆T, then there is a basisB such thatS ⊆B ⊆T.
If any basis ofV (and therefore every basis) has a finite number of elements,V is afinite-dimensional vector space. IfU is a subspace ofV, thendimU ≤ dimV. In the case whereV is finite-dimensional, the equality of the dimensions impliesU =V.
Matrices allow explicit manipulation of finite-dimensional vector spaces andlinear maps. Their theory is thus an essential part of linear algebra.
LetV be a finite-dimensional vector space over a fieldF, and(v1,v2, ...,vm) be a basis ofV (thusm is the dimension ofV). By definition of a basis, the map
is abijection fromFm, the set of thesequences ofm elements ofF, ontoV. This is anisomorphism of vector spaces, ifFm is equipped with its standard structure of vector space, where vector addition and scalar multiplication are done component by component.
IfW is another finite dimensional vector space (possibly the same), with a basis(w1, ...,wn), a linear mapf fromW toV is well defined by its values on the basis elements, that is(f(w1), ...,f(wn)). Thus,f is well represented by the list of the corresponding column matrices. That is, if
forj = 1, ...,n, thenf is represented by the matrix
withm rows andn columns.
Matrix multiplication is defined in such a way that the product of two matrices is the matrix of thecomposition of the corresponding linear maps, and the product of a matrix and a column matrix is the column matrix representing the result of applying the represented linear map to the represented vector. It follows that the theory of finite-dimensional vector spaces and the theory of matrices are two different languages for expressing the same concepts.
Two matrices that encode the same linear transformation in different bases are calledsimilar. It can be proved that two matrices are similar if and only if one can transform one into the other byelementary row and column operations. For a matrix representing a linear map fromW toV, the row operations correspond to change of bases inV and the column operations correspond to change of bases inW. Every matrix is similar to anidentity matrix possibly bordered by zero rows and zero columns. In terms of vector spaces, this means that, for any linear map fromW toV, there are bases such that a part of the basis ofW is mapped bijectively on a part of the basis ofV, and that the remaining basis elements ofW, if any, are mapped to zero.Gaussian elimination is the basic algorithm for finding these elementary operations, and proving these results.
A finite set of linear equations in a finite set of variables, for example,x1,x2, ...,xn, orx,y, ...,z is called a system of linear equations or alinear system.[11][12][13][14][15]
Systems of linear equations form a fundamental part of linear algebra. Historically, linear algebra and matrix theory have been developed for solving such systems. In the modern presentation of linear algebra through vector spaces and matrices, many problems may be interpreted in terms of linear systems.
for putting it inreduced row echelon form. These row operations do not change the set of solutions of the system of equations. In the example, the reduced echelon form is
showing that the system (S) has the unique solution
More generally, a system of linear equations in variables can be written aswhere
If and the matrix is invertible, then the system has the unique solution.
It follows from this matrix interpretation of linear systems that the same methods can be applied for solving linear systems and for many operations on matrices and linear transformations, which include the computation of theranks,kernels,matrix inverses.
A linearendomorphism is a linear map that maps a vector spaceV to itself. IfV has a basis ofn elements, such an endomorphism is represented by a square matrix of sizen.
Concerning general linear maps, linear endomorphisms, and square matrices have some specific properties that make their study an important part of linear algebra, which is used in many parts of mathematics, includinggeometric transformations,coordinate changes,quadratic forms, and many other parts of mathematics.
Thedeterminant of a square matrixA is defined to be[16]
whereSn is thegroup of all permutations ofn elements,σ is a permutation, and(−1)σ theparity of the permutation. A matrix isinvertible if and only if the determinant is invertible (i.e., nonzero if the scalars belong to a field).
Thedeterminant of an endomorphism is the determinant of the matrix representing the endomorphism in terms of some ordered basis. This definition makes sense since this determinant is independent of the choice of the basis.
Iff is a linear endomorphism of a vector spaceV over a fieldF, aneigenvector off is a nonzero vectorv ofV such thatf(v) =av for some scalara inF. This scalara is aneigenvalue off.
If the dimension ofV is finite, and a basis has been chosen,f andv may be represented, respectively, by a square matrixM and a column matrixz; the equation defining eigenvectors and eigenvalues becomes
Using theidentity matrixI, whose entries are all zero, except those of the main diagonal, which are equal to one, this may be rewritten
Asz is supposed to be nonzero, this means thatM –aI is asingular matrix, and thus that its determinantdet (M −aI) equals zero. The eigenvalues are thus theroots of thepolynomial
IfV is of dimensionn, this is amonic polynomial of degreen, called thecharacteristic polynomial of the matrix (or of the endomorphism), and there are, at most,n eigenvalues.
If a basis exists that consists only of eigenvectors, the matrix off on this basis has a very simple structure: it is adiagonal matrix such that the entries on themain diagonal are eigenvalues, and the other entries are zero. In this case, the endomorphism and the matrix are said to bediagonalizable. More generally, an endomorphism and a matrix are also said diagonalizable, if they become diagonalizable afterextending the field of scalars. In this extended sense, if the characteristic polynomial issquare-free, then the matrix is diagonalizable.
Asymmetric matrix is always diagonalizable. There are non-diagonalizable matrices, the simplest being
(it cannot be diagonalizable since its square is thezero matrix, and the square of a nonzero diagonal matrix is never zero).
When an endomorphism is not diagonalizable, there are bases on which it has a simple form, although not as simple as the diagonal form. TheFrobenius normal form does not need to extend the field of scalars and makes the characteristic polynomial immediately readable on the matrix. TheJordan normal form requires to extension of the field of scalar for containing all eigenvalues and differs from the diagonal form only by some entries that are just above the main diagonal and are equal to 1.
Alinear form is a linear map from a vector spaceV over a fieldF to the field of scalarsF, viewed as a vector space over itself. Equipped bypointwise addition and multiplication by a scalar, the linear forms form a vector space, called thedual space ofV, and usually denotedV*[17] orV′.[18][19]
Ifv1, ...,vn is a basis ofV (this implies thatV is finite-dimensional), then one can define, fori = 1, ...,n, a linear mapvi* such thatvi*(vi) = 1 andvi*(vj) = 0 ifj ≠i. These linear maps form a basis ofV*, called thedual basis ofv1, ...,vn. (IfV is not finite-dimensional, thevi* may be defined similarly; they are linearly independent, but do not form a basis.)
Forv inV, the map
is a linear form onV*. This defines thecanonical linear map fromV into(V*)*, the dual ofV*, called thedouble dual orbidual ofV. This canonical map is anisomorphism ifV is finite-dimensional, and this allows identifyingV with its bidual. (In the infinite-dimensional case, the canonical map is injective, but not surjective.)
There is thus a complete symmetry between a finite-dimensional vector space and its dual. This motivates the frequent use, in this context, of thebra–ket notation
be a linear map. For every linear formh onW, thecomposite functionh ∘f is a linear form onV. This defines a linear map
between the dual spaces, which is called thedual or thetranspose off.
IfV andW are finite-dimensional, andM is the matrix off in terms of some ordered bases, then the matrix off* over the dual bases is thetransposeMT ofM, obtained by exchanging rows and columns.
If elements of vector spaces and their duals are represented by column vectors, this duality may be expressed inbra–ket notation by
To highlight this symmetry, the two members of this equality are sometimes written
Besides these basic concepts, linear algebra also studies vector spaces with additional structure, such as aninner product. The inner product is an example of abilinear form, and it gives the vector space a geometric structure by allowing for the definition of length and angles. Formally, aninner product is a map.
that satisfies the following threeaxioms for all vectorsu,v,w inV and all scalarsa inF:[20][21]
and so we can call this quantity the cosine of the angle between the two vectors.
Two vectors are orthogonal if⟨u,v⟩ = 0. An orthonormal basis is a basis where all basis vectors have length 1 and are orthogonal to each other. Given any finite-dimensional vector space, an orthonormal basis could be found by theGram–Schmidt procedure. Orthonormal bases are particularly easy to deal with, since ifv =a1v1 + ⋯ +anvn, then
The inner product facilitates the construction of many useful concepts. For instance, given a transformT, we can define itsHermitian conjugateT* as the linear transform satisfying
IfT satisfiesTT* =T*T, we callTnormal. It turns out that normal matrices are precisely the matrices that have an orthonormal system of eigenvectors that spanV.
There is a strong relationship between linear algebra andgeometry, which started with the introduction byRené Descartes, in 1637, ofCartesian coordinates. In this new (at that time) geometry, now calledCartesian geometry, points are represented byCartesian coordinates, which are sequences of three real numbers (in the case of the usualthree-dimensional space). The basic objects of geometry, which arelines andplanes are represented by linear equations. Thus, computing intersections of lines and planes amounts to solving systems of linear equations. This was one of the main motivations for developing linear algebra.
Until the end of the 19th century, geometric spaces were defined byaxioms relating points, lines, and planes (synthetic geometry). Around this date, it appeared that one may also define geometric spaces by constructions involving vector spaces (see, for example,Projective space andAffine space). It has been shown that the two approaches are essentially equivalent.[22] In classical geometry, the involved vector spaces are vector spaces over the reals, but the constructions may be extended to vector spaces over any field, allowing considering geometry over arbitrary fields, includingfinite fields.
Presently, most textbooks introduce geometric spaces from linear algebra, and geometry is often presented, at the elementary level, as a subfield of linear algebra.
Linear algebra is used in almost all areas of mathematics, thus making it relevant in almost all scientific domains that use mathematics. These applications may be divided into several wide categories.
Nearly allscientific computations involve linear algebra. Consequently, linear algebra algorithms have been highly optimized.BLAS andLAPACK are the best known implementations. For improving efficiency, some of them configure the algorithms automatically, at run time, to adapt them to the specificities of the computer (cache size, number of availablecores, ...).
Since the 1960s there have been processors with specialized instructions[23] for optimizing the operations of linear algebra, optional array processors[24] under the control of a conventional processor, supercomputers[25][26][27] designed for array processing and conventional processors augmented[28] with vector registers.
Some contemporaryprocessors, typicallygraphics processing units (GPU), are designed with a matrix structure, for optimizing the operations of linear algebra.[29]
In all these applications,synthetic geometry is often used for general descriptions and a qualitative approach, but for the study of explicit situations, one must compute withcoordinates. This requires the heavy use of linear algebra.
Most physical phenomena are modeled bypartial differential equations. To solve them, one usually decomposes the space in which the solutions are searched into small, mutually interactingcells. Forlinear systems this interaction involveslinear functions. Fornonlinear systems, this interaction is often approximated by linear functions.[b]This is called a linear model or first-order approximation. Linear models are frequently used for complex nonlinear real-world systems because they makeparametrization more manageable.[30] In both cases, very large matrices are generally involved.Weather forecasting (or more specifically,parametrization for atmospheric modeling) is a typical example of a real-world application, where the whole Earthatmosphere is divided into cells of, say, 100 km of width and 100 km of height.
Fluid mechanics, fluid dynamics, and thermal energy systems
Linear algebra, a branch of mathematics dealing withvector spaces andlinear mappings between these spaces, plays a critical role in various engineering disciplines, includingfluid mechanics,fluid dynamics, andthermal energy systems. Its application in these fields is multifaceted and indispensable for solving complex problems.
Influid mechanics, linear algebra is integral to understanding and solving problems related to the behavior of fluids. It assists in the modeling and simulation of fluid flow, providing essential tools for the analysis offluid dynamics problems. For instance, linear algebraic techniques are used to solve systems ofdifferential equations that describe fluid motion. These equations, often complex andnon-linear, can be linearized using linear algebra methods, allowing for simpler solutions and analyses.
In the field of fluid dynamics, linear algebra finds its application incomputational fluid dynamics (CFD), a branch that usesnumerical analysis anddata structures to solve and analyze problems involving fluid flows. CFD relies heavily on linear algebra for the computation of fluid flow andheat transfer in various applications. For example, theNavier–Stokes equations, fundamental influid dynamics, are often solved using techniques derived from linear algebra. This includes the use ofmatrices andvectors to represent and manipulate fluid flow fields.
Furthermore, linear algebra plays a crucial role inthermal energy systems, particularly inpower systems analysis. It is used to model and optimize the generation,transmission, anddistribution of electric power. Linear algebraic concepts such as matrix operations andeigenvalue problems are employed to enhance the efficiency, reliability, and economic performance ofpower systems. The application of linear algebra in this context is vital for the design and operation of modernpower systems, includingrenewable energy sources andsmart grids.
Overall, the application of linear algebra influid mechanics,fluid dynamics, andthermal energy systems is an example of the profound interconnection betweenmathematics andengineering. It provides engineers with the necessary tools to model, analyze, and solve complex problems in these domains, leading to advancements in technology and industry.
This section presents several related topics that do not appear generally in elementary textbooks on linear algebra but are commonly considered, in advanced mathematics, as parts of linear algebra.
The existence of multiplicative inverses in fields is not involved in the axioms defining a vector space. One may thus replace the field of scalars by aringR, and this gives the structure called amodule overR, orR-module.
The concepts of linear independence, span, basis, and linear maps (also calledmodule homomorphisms) are defined for modules exactly as for vector spaces, with the essential difference that, ifR is not a field, there are modules that do not have any basis. The modules that have a basis are thefree modules, and those that are spanned by a finite set are thefinitely generated modules. Module homomorphisms between finitely generated free modules may be represented by matrices. The theory of matrices over a ring is similar to that of matrices over a field, except thatdeterminants exist only if the ring iscommutative, and that a square matrix over a commutative ring isinvertible only if its determinant has amultiplicative inverse in the ring.
Vector spaces are completely characterized by their dimension (up to an isomorphism). In general, there is not such a complete classification for modules, even if one restricts oneself to finitely generated modules. However, every module is acokernel of a homomorphism of free modules.
Modules over the integers can be identified withabelian groups, since the multiplication by an integer may be identified as a repeated addition. Most of the theory of abelian groups may be extended to modules over aprincipal ideal domain. In particular, over a principal ideal domain, every submodule of a free module is free, and thefundamental theorem of finitely generated abelian groups may be extended straightforwardly to finitely generated modules over a principal ring.
There are many rings for which there are algorithms for solving linear equations and systems of linear equations. However, these algorithms have generally acomputational complexity that is much higher than similar algorithms over a field. For more details, seeLinear equation over a ring.
Inmultilinear algebra, one considers multivariable linear transformations, that is, mappings that are linear in each of several different variables. This line of inquiry naturally leads to the idea of thedual space, the vector spaceV* consisting of linear mapsf :V →F whereF is the field of scalars. Multilinear mapsT :Vn →F can be described viatensor products of elements ofV*.
If, in addition to vector addition and scalar multiplication, there is a bilinear vector productV ×V →V, the vector space is called analgebra; for instance, associative algebras are algebras with an associate vector product (like the algebra of square matrices, or the algebra of polynomials).
Vector spaces that are not finite-dimensional often require additional structure to be tractable. Anormed vector space is a vector space along with a function called anorm, which measures the "size" of elements. The norm induces ametric, which measures the distance between elements, and induces atopology, which allows for a definition of continuous maps. The metric also allows for a definition oflimits andcompleteness – a normed vector space that is complete is known as aBanach space. A complete metric space along with the additional structure of aninner product (a conjugate symmetricsesquilinear form) is known as aHilbert space, which is in some sense a particularly well-behaved Banach space.Functional analysis applies the methods of linear algebra alongside those ofmathematical analysis to study various function spaces; the central objects of study in functional analysis areLp spaces, which are Banach spaces, and especially theL2 space of square-integrable functions, which is the only Hilbert space among them. Functional analysis is of particular importance to quantum mechanics, the theory of partial differential equations, digital signal processing, and electrical engineering. It also provides the foundation and theoretical framework that underlies the Fourier transform and related methods.
^This axiom is not asserting the associativity of an operation, since there are two operations in question, scalar multiplicationbv; and field multiplication:ab.
^This may have the consequence that some physically interesting solutions are omitted.
^Banerjee, Sudipto; Roy, Anindya (2014).Linear Algebra and Matrix Analysis for Statistics. Texts in Statistical Science (1st ed.). Chapman and Hall/CRC.ISBN978-1420095388.
^Strang, Gilbert (July 19, 2005).Linear Algebra and Its Applications (4th ed.). Brooks Cole.ISBN978-0-03-010567-8.
^Benjamin Peirce (1872)Linear Associative Algebra, lithograph, new edition with corrections, notes, and an added 1875 paper by Peirce, plus notes by his sonCharles Sanders Peirce, published in theAmerican Journal of Mathematics v. 4, 1881, Johns Hopkins University, pp. 221–226,GoogleEprint and as an extract, D. Van Nostrand, 1882,GoogleEprint.
Grassmann, Hermann (1844),Die lineale Ausdehnungslehre ein neuer Zweig der Mathematik: dargestellt und durch Anwendungen auf die übrigen Zweige der Mathematik, wie auch auf die Statik, Mechanik, die Lehre vom Magnetismus und die Krystallonomie erläutert, Leipzig: O. Wigand
Anton, Howard (2005),Elementary Linear Algebra (Applications Version) (9th ed.), Wiley International
Banerjee, Sudipto; Roy, Anindya (2014),Linear Algebra and Matrix Analysis for Statistics, Texts in Statistical Science (1st ed.), Chapman and Hall/CRC,ISBN978-1420095388
Bretscher, Otto (2004),Linear Algebra with Applications (3rd ed.), Prentice Hall,ISBN978-0-13-145334-0
Essence of linear algebra, a video presentation from3Blue1Brown of the basics of linear algebra, with emphasis on the relationship between the geometric, the matrix and the abstract points of view
