In the field ofmathematics known asfunctional analysis, theinvariant subspace problem is a partially unresolved problem asking whether everybounded operator on acomplexBanach space sends some non-trivialclosedsubspace to itself. Many variants of the problem have been solved, by restricting the class of bounded operators considered or by specifying a particular class of Banach spaces. The problem is stillopen forseparableHilbert spaces (in other words, each example, found so far, of an operator with no non-trivial invariant subspaces is an operator that acts on a Banach space that is notisomorphic to a separable Hilbert space).

The problem seems to have been stated in the mid-20th century after work byBeurling andvon Neumann,^[1] who found (but never published) a positive solution for the case ofcompact operators. It was then posed byPaul Halmos for the case of operators${\displaystyle T}$ such that${\displaystyle T^2}$ is compact. This was resolved affirmatively, for the more general class of polynomially compact operators (operators${\displaystyle T}$ such that${\displaystyle p(T)}$ is a compact operator for a suitably chosen nonzeropolynomial${\displaystyle p}$), byAllen R. Bernstein andAbraham Robinson in 1966 (seeNon-standard analysis § Invariant subspace problem for a summary of the proof).

ForBanach spaces, the first example of an operator without an invariant subspace was constructed byPer Enflo. He proposed acounterexample to the invariant subspace problem in 1975, publishing an outline in 1976. Enflo submitted the full article in 1981 and the article's complexity and length delayed its publication to 1987.^[2] Enflo's long "manuscript had a world-wide circulation among mathematicians"^[1] and some of its ideas were described in publications besides Enflo (1976).^[3] Enflo's works inspired a similar construction of an operator without an invariant subspace for example byBernard Beauzamy, who acknowledged Enflo's ideas.^[2]

In the 1990s, Enflo developed a "constructive" approach to the invariant subspace problem on Hilbert spaces.^[4]

In May 2023, a preprint of Enflo appeared on arXiv,^[5] which, if correct, solves the problem for Hilbert spaces and completes the picture.^{[non-primary source needed]}

In July 2023, a second and independent preprint of Neville appeared on arXiv,^[6] claiming the solution of the problem for separable Hilbert spaces.^{[non-primary source needed]}

Formally, theinvariant subspace problem for a complexBanach space${\displaystyle H}$ ofdimension > 1 is the question whether everybounded linear operator${\displaystyle T:H\to H}$ has a non-trivialclosed${\displaystyle T}$-invariant subspace: a closedlinear subspace${\displaystyle W}$ of${\displaystyle H}$, which is different from${\displaystyle \{0\}}$ and from${\displaystyle H}$, such that${\displaystyle T(W)\subset W}$.

A negative answer to the problem is closely related to properties of the orbits${\displaystyle T}$. If${\displaystyle x}$ is an element of the Banach space${\displaystyle H}$, the orbit of${\displaystyle x}$ under the action of${\displaystyle T}$, denoted by${\displaystyle [x]}$, is the subspace generated by the sequence${\displaystyle \{T^n(x):n\ge 0\}}$. This is also called the${\displaystyle T}$-cyclic subspace generated by${\displaystyle x}$. From the definition it follows that${\displaystyle [x]}$ is a${\displaystyle T}$-invariant subspace. Moreover, it is theminimal${\displaystyle T}$-invariant subspace containing${\displaystyle x}$: if${\displaystyle W}$ is another invariant subspace containing${\displaystyle x}$, then necessarily${\displaystyle T^n(x)\in W}$ for all${\displaystyle n\ge 0}$ (since${\displaystyle W}$ is${\displaystyle T}$-invariant), and so${\displaystyle [x]\subset W}$. If${\displaystyle x}$ is non-zero, then${\displaystyle [x]}$ is not equal to${\displaystyle \{0\}}$, so its closure is either the whole space${\displaystyle H}$ (in which case${\displaystyle x}$ is said to be acyclic vector for${\displaystyle T}$) or it is a non-trivial${\displaystyle T}$-invariant subspace. Therefore, a counterexample to the invariant subspace problem would be a Banach space${\displaystyle H}$ and a bounded operator${\displaystyle T:H\to H}$ for which every non-zero vector${\displaystyle x\in H}$ is acyclic vector for${\displaystyle T}$. (Where a "cyclic vector"${\displaystyle x}$ for an operator${\displaystyle T}$ on a Banach space${\displaystyle H}$ means one for which the orbit${\displaystyle [x]}$ of${\displaystyle x}$ is dense in${\displaystyle H}$.)

While the case of the invariant subspace problem for separable Hilbert spaces is still open, several other cases have been settled fortopological vector spaces (over thefield of complex numbers):

• For finite-dimensional complex vector spaces, every operator admits an eigenvector, so it has a 1-dimensional invariant subspace.
• The conjecture is true if the Hilbert space${\displaystyle H}$ is notseparable (i.e. if it has anuncountableorthonormal basis). In fact, if${\displaystyle x}$ is a non-zero vector in${\displaystyle H}$, the norm closure of the linear orbit${\displaystyle [x]}$ is separable (by construction) and hence a proper subspace and also invariant.
• von Neumann showed^[7] that any compact operator on a Hilbert space of dimension at least 2 has a non-trivial invariant subspace.
• Thespectral theorem shows that allnormal operators admit invariant subspaces.
• Aronszajn & Smith (1954) proved that everycompact operator on any Banach space of dimension at least 2 has an invariant subspace.
• Bernstein & Robinson (1966) proved usingnon-standard analysis that if the operator${\displaystyle T}$ on a Hilbert space is polynomially compact (in other words${\displaystyle p(T)}$ is compact for some nonzero polynomial${\displaystyle p}$) then${\displaystyle T}$ has an invariant subspace. Their proof uses the original idea of embedding the infinite-dimensional Hilbert space in ahyperfinite-dimensional Hilbert space (seeNon-standard analysis#Invariant subspace problem).
• Halmos (1966), after having seen Robinson's preprint, eliminated the non-standard analysis from it and provided a shorter proof in the same issue of the same journal.
• Lomonosov (1973) gave a very short proof using theSchauder fixed point theorem that if the operator${\displaystyle T}$ on a Banach space commutes with a non-zero compact operator then${\displaystyle T}$ has a non-trivial invariant subspace. This includes the case of polynomially compact operators because an operator commutes with any polynomial in itself. More generally, he showed that if${\displaystyle S}$ commutes with a non-scalar operator${\displaystyle T}$ that commutes with a non-zero compact operator, then${\displaystyle S}$ has an invariant subspace.^[8]
• The first example of an operator on a Banach space with no non-trivial invariant subspaces was found byPer Enflo (1976,1987), and his example was simplified byBeauzamy (1985).
• The first counterexample on a "classical" Banach space was found byCharles Read (1984,1985), who described an operator on the classical Banach space${\displaystyle l_1}$ with no invariant subspaces.
• LaterCharles Read (1988) constructed an operator on${\displaystyle l_1}$ without even a non-trivial closed invariantsubset, that is that for every vector${\displaystyle x}$ theset${\displaystyle \{T^n(x):n\ge 0\}}$ is dense, in which case the vector is calledhypercyclic (the difference with the case of cyclic vectors is that we are not taking the subspace generated by the points${\displaystyle \{T^n(x):n\ge 0\}}$ in this case).
• Atzmon (1983) gave an example of an operator without invariant subspaces on anuclearFréchet space.
• Śliwa (2008) proved that any infinite-dimensional Banach space of countable type over anon-Archimedean field admits a bounded linear operator without a non-trivial closed invariant subspace. This completely solves the non-Archimedean version of this problem, posed by van Rooij and Schikhof in 1992.
• Argyros & Haydon (2011) gave the construction of an infinite-dimensional Banach space such that every continuous operator is the sum of a compact operator and a scalar operator, so in particular every operator has an invariant subspace.

• Abramovich, Yuri A.; Aliprantis, Charalambos D. (2002),An Invitation to Operator Theory,Graduate Studies in Mathematics, vol. 50, Providence, RI: American Mathematical Society,doi:10.1090/gsm/050,ISBN 978-0-8218-2146-6,MR 1921782
• Argyros, Spiros A.; Haydon, Richard G. (2011), "A hereditarily indecomposable L_∞-space that solves the scalar-plus-compact problem",Acta Math.,206 (1):1–54,arXiv:0903.3921,doi:10.1007/s11511-011-0058-y,MR 2784662,S2CID 119532059
• Aronszajn, N.; Smith, K. T. (1954), "Invariant subspaces of completely continuous operators",Annals of Mathematics, Second Series,60 (2):345–350,doi:10.2307/1969637,JSTOR 1969637,MR 0065807
• Atzmon, Aharon (1983), "An operator without invariant subspaces on a nuclear Fréchet space",Annals of Mathematics, Second Series,117 (3):669–694,doi:10.2307/2007039,JSTOR 2007039,MR 0701260
• Beauzamy, Bernard (1985), "Un opérateur sans sous-espace invariant: simplification de l'exemple de P. Enflo" [An operator with no invariant subspace: simplification of the example of P. Enflo],Integral Equations and Operator Theory (in French),8 (3):314–384,doi:10.1007/BF01202903,MR 0792905,S2CID 121418247
• Beauzamy, Bernard (1988),Introduction to operator theory and invariant subspaces, North-Holland Mathematical Library, vol. 42, Amsterdam: North-Holland,ISBN 978-0-444-70521-1,MR 0967989
• Bernstein, Allen R.;Robinson, Abraham (1966),"Solution of an invariant subspace problem of K. T. Smith and P. R. Halmos",Pacific Journal of Mathematics,16 (3):421–431,doi:10.2140/pjm.1966.16.421,MR 0193504
• Enflo, Per (1976), "On the invariant subspace problem in Banach spaces",Séminaire Maurey--Schwartz (1975--1976) Espaces L^p, applications radonifiantes et géométrie des espaces de Banach, Exp. Nos. 14-15, Centre Math., École Polytech., Palaiseau, p. 7,MR 0473871
• Enflo, Per (1987), "On the invariant subspace problem for Banach spaces",Acta Mathematica,158 (3):213–313,doi:10.1007/BF02392260,MR 0892591
• Enflo, Per;Lomonosov, Victor (2001), "Some aspects of the invariant subspace problem",Handbook of the geometry of Banach spaces, vol. I, Amsterdam: North-Holland, pp. 533–559,doi:10.1016/S1874-5849(01)80015-2,ISBN 9780444828422,MR 1863701
• Halmos, Paul R. (1966),"Invariant subspaces of polynomially compact operators",Pacific Journal of Mathematics,16 (3):433–437,doi:10.2140/pjm.1966.16.433,MR 0193505
• Lomonosov, V. I. (1973), "Invariant subspaces of the family of operators that commute with a completely continuous operator",Akademija Nauk SSSR. Funkcional' Nyi Analiz I Ego Prilozenija,7 (3):55–56,doi:10.1007/BF01080698,MR 0420305,S2CID 121421267
• Pearcy, Carl; Shields, Allen L. (1974), "A survey of the Lomonosov technique in the theory of invariant subspaces", in C. Pearcy (ed.),Topics in operator theory, Mathematical Surveys, Providence, R.I.: American Mathematical Society, pp. 219–229,MR 0355639
• Read, C. J. (1984), "A solution to the invariant subspace problem",The Bulletin of the London Mathematical Society,16 (4):337–401,doi:10.1112/blms/16.4.337,MR 0749447
• Read, C. J. (1985), "A solution to the invariant subspace problem on the space l₁",The Bulletin of the London Mathematical Society,17 (4):305–317,doi:10.1112/blms/17.4.305,MR 0806634
• Read, C. J. (1988), "The invariant subspace problem for a class of Banach spaces, 2: hypercyclic operators",Israel Journal of Mathematics,63 (1):1–40,doi:10.1007/BF02765019,MR 0959046,S2CID 123651876
• Radjavi, Heydar;Rosenthal, Peter (1982),"The invariant subspace problem",The Mathematical Intelligencer,4 (1):33–37,doi:10.1007/BF03022994,MR 0678734,S2CID 122811130
• Radjavi, Heydar; Rosenthal, Peter (2003),Invariant Subspaces (Second ed.), Mineola, NY: Dover,ISBN 978-0-486-42822-2,MR 2003221
• Radjavi, Heydar; Rosenthal, Peter (2000),Simultaneous triangularization, Universitext, New York: Springer-Verlag, pp. xii+318,doi:10.1007/978-1-4612-1200-3,ISBN 978-0-387-98467-4,MR 1736065
• Śliwa, Wiesław (2008),"The Invariant Subspace Problem for Non-Archimedean Banach Spaces"(PDF),Canadian Mathematical Bulletin,51 (4):604–617,doi:10.4153/CMB-2008-060-9,hdl:10593/798,MR 2462465,S2CID 40430187
• Yadav, B. S. (2005), "The present state and heritages of the invariant subspace problem",Milan Journal of Mathematics,73 (1):289–316,doi:10.1007/s00032-005-0048-7,MR 2175046,S2CID 121068326

• Invariant subspaces
• Operator theory
• Functional analysis
• Unsolved problems in mathematics
• Mathematical problems

• Articles with short description
• Short description is different from Wikidata
• All pages needing factual verification
• Wikipedia articles needing factual verification from December 2025
• CS1 French-language sources (fr)