Movatterモバイル変換

[0]ホーム

Jump to content

Tsallis entropy

Edit links

From Wikipedia, the free encyclopedia

(Redirected fromTsallis Entropy)

Generalization of the standard Boltzmann–Gibbs entropy

In physics, theTsallis entropy is a generalization of the standardBoltzmann–Gibbs entropy. It is proportional to the expectation of theq-logarithm of a distribution.

History

[edit]

The concept was introduced in 1988 byConstantino Tsallis^[1] as a basis for generalizing the standardstatistical mechanics and is identical in form toHavrda–Charvát structural α-entropy,^[2] introduced in 1967 withininformation theory.

Definition

[edit]

Given a discrete set of probabilities $\{p_{i}\}$ with the condition $\sum _{i}p_{i}=1$ , and $q {\displaystyle q}$ any real number, theTsallis entropy is defined as

S_{q}({p_{i}})=k\cdot {\frac {1}{q-1}}\left(1-\sum _{i}p_{i}^{q}\right),

where $q {\displaystyle q}$ is a real parameter sometimes calledentropic-index and $k {\displaystyle k}$ a positive constant.

In the limit as $q\to 1$ , the usual Boltzmann–Gibbs entropy is recovered, namely

S_{\text{BG}}=S_{1}(p)=-k\sum _{i}p_{i}\ln p_{i},

where one identifies $k {\displaystyle k}$ with theBoltzmann constant $k_{B}$ .

For continuous probability distributions, we define the entropy as

S_{q}[p]={1 \over q-1}\left(1-\int (p(x))^{q}\,dx\right),

where $p(x)$ is aprobability density function.

q-exponential

[edit]

Generalizations of the exponential family using theq-exponential function have been widely explored in statistical physics and information geometry. In the machine learning literature, theq-exponential family has garnered interest due to its flexibility in modeling tail behavior by tuning the parameterq.

Theq-exponential function and its inverse, theq-logarithm, are defined as follows:

\exp _{q}(x)={\begin{cases}\exp(x),&{\text{if }}q=1\\\left[1+(1-q)x\right]_{+}^{\frac {1}{1-q}},&{\text{if }}q\neq 1\end{cases}}

\ln _{q}(x)={\begin{cases}\ln(x),&{\text{if }}q=1\\{\frac {x^{1-q}-1}{1-q}},&{\text{if }}q\neq 1\end{cases}}

where $[z]_{+}:=\max\{z,0\}$ .

These functions recover the standard exponential and logarithmic functions in the limit $q\to 1$ , i.e., $\lim _{q\to 1}\exp _{q}(x)=\exp(x)$ and $\lim _{q\to 1}\ln _{q}(x)=\ln(x)$ . Like the ordinary exponential, $\exp _{q}(x)$ is monotonic and convex for $q>0$ , and satisfies $\exp _{q}(0)=1$ . However, a key distinction is that $\exp _{q}(a+b)\neq \exp _{q}(a)\cdot \exp _{q}(b)$ unless $q=1$ .

Tsallis entropy can be defined using theq-exponential:

S_{q}({p_{i}})=k\sum _{i}p_{i}\ln _{q}{\frac {1}{p_{i}}}.

Cross-entropy

[edit]

The cross-entropy pendant is the expectation of the negative q-logarithm with respect to a second distribution, $r {\displaystyle r}$ . So ${\tfrac {1}{q-1}}(1-{\textstyle \sum _{i}}p_{i}^{q}\cdot {\tfrac {r_{i}}{p_{i}}})$ .

Using $t=q-1$ , this may be written $(1-E_{r}[p^{t}])/t$ . For smaller $t {\displaystyle t}$ , values $p_{i}^{t}$ all tend towards $1 {\displaystyle 1}$ .

The limit $q\to 1$ computes the negative of the slope of $E_{r}[p^{t}]$ at $t=0$ and one recovers $-{\textstyle \sum _{i}}r_{i}\ln p_{i}$ . So for fixed small $t {\displaystyle t}$ , raising this expectation relates tolog-likelihood maximalization.

Properties

[edit]

Identities

[edit]

A logarithm can be expressed in terms of a slope through ${\tfrac {d}{dx}}p^{x}=p^{x}\ln p$ resulting in the following formula for the standard entropy:

S=-\lim _{x\rightarrow 1}{\tfrac {d}{dx}}\sum _{i}p_{i}^{x}=-{\textstyle \sum _{i}}p_{i}\ln p_{i}

Likewise, the discrete Tsallis entropy satisfies

S_{q}=-\lim _{x\rightarrow 1}D_{q}\sum _{i}p_{i}^{x}

whereD_q is theq-derivative with respect tox.

Non-additivity

[edit]

Given two independent systemsA andB, for which the jointprobability density satisfies

p(A,B)=p(A)p(B),\,

the Tsallis entropy of this system satisfies

S_{q}(A,B)=S_{q}(A)+S_{q}(B)+(1-q)S_{q}(A)S_{q}(B).\,

From this result, it is evident that the parameter $|1-q|$ is a measure of the departure from additivity. In the limit whenq = 1,

S(A,B)=S(A)+S(B),\,

which is what is expected for an additive system. This property is sometimes referred to as "pseudo-additivity".

Exponential families

[edit]

Many common distributions like the normal distribution belongs to the statisticalexponential families.Tsallis entropy for an exponential family can be written^[3] as

H_{q}^{T}(p_{F}(x;\theta ))={\frac {1}{1-q}}\left((e^{F(q\theta )-qF(\theta )})E_{p}[e^{(q-1)k(x)}]-1\right)

whereF is log-normalizer andk the term indicating the carrier measure.For multivariate normal, termk is zero, and therefore the Tsallis entropy is in closed-form.

Applications

[edit]

The Tsallis Entropy has been used along with thePrinciple of maximum entropy to derive theTsallis distribution.

In scientific literature, the physical relevance of the Tsallis entropy has been debated.^[4]^[5]^[6] However, from the years 2000 on, an increasingly wide spectrum of natural, artificial and socialcomplex systems have been identified which confirm the predictions and consequences that are derived from this nonadditive entropy, such as nonextensive statistical mechanics,^[7] which generalizes the Boltzmann–Gibbs theory.

Among the various experimental verifications and applications presently available in the literature, the following ones deserve a special mention:

The distribution characterizing the motion of cold atoms in dissipativeoptical lattices predicted in 2003^[8] and observed in 2006.^[9]
The fluctuations of the magnetic field in thesolar wind enabled the calculation of the q-triplet (or Tsallis triplet).^[10]
The velocity distributions in a driven dissipativedusty plasma.^[11]
Spin glass relaxation.^[12]
Trapped ion interacting with a classicalbuffer gas.^[13]
High energy collisional experiments at LHC/CERN (CMS, ATLAS and ALICE detectors)^[14]^[15] and RHIC/Brookhaven (STAR and PHENIX detectors).^[16]

Among the various available theoretical results which clarify the physical conditions under which Tsallis entropy and associated statistics apply, the following ones can be selected:

Anomalous diffusion.^[17]^[18]
Uniqueness theorem.^[19]
Sensitivity toinitial conditions and entropy production at the edge of chaos.^[20]^[21]
Probability sets that make the nonadditive Tsallis entropy to be extensive in the thermodynamical sense.^[22]
Stronglyquantum entangled systems and thermodynamics.^[23]
Thermostatistics ofoverdamped motion of interacting particles.^[24]^[25]
Nonlinear generalizations of the Schrödinger,Klein–Gordon andDirac equations.^[26]
Black hole entropy calculation.^[27]

For further details a bibliography is available athttp://tsallis.cat.cbpf.br/biblio.htm

Generalized entropies

[edit]

Several interesting physical systems^[28] abide by entropicfunctionals that are more general than the standard Tsallis entropy. Therefore, several physically meaningful generalizations have been introduced. The two most general of these are notably:Superstatistics, introduced by C. Beck and E. G. D. Cohen in 2003^[29] andSpectral Statistics, introduced by G. A. Tsekouras andConstantino Tsallis in 2005.

References

[edit]

^Tsallis, C. (1988). "Possible generalization of Boltzmann-Gibbs statistics".Journal of Statistical Physics.52 (1–2):479–487.Bibcode:1988JSP....52..479T.doi:10.1007/BF01016429.hdl:10338.dmlcz/142811.S2CID 16385640.
^Havrda, J.; Charvát, F. (1967)."Quantification method of classification processes. Concept of structural α-entropy"(PDF).Kybernetika.3 (1):30–35.
^Nielsen, F.; Nock, R. (2012). "A closed-form expression for the Sharma–Mittal entropy of exponential families".Journal of Physics A: Mathematical and Theoretical.45 (3) 032003.arXiv:1112.4221.Bibcode:2012JPhA...45c2003N.doi:10.1088/1751-8113/45/3/032003.S2CID 8653096.
^Cho, A. (2002). "A Fresh Take on Disorder, Or Disorderly Science?".Science.297 (5585):1268–1269.doi:10.1126/science.297.5585.1268.PMID 12193769.S2CID 5441957.
^Abe, S.; Rajagopal, A.K. (2003). "Revisiting Disorder and Tsallis Statistics".Science.300 (5617):249–251.doi:10.1126/science.300.5617.249d.PMID 12690173.S2CID 39719500.
^Pressé, S.; Ghosh, K.; Lee, J.; Dill, K. (2013). "Nonadditive Entropies Yield Probability Distributions with Biases not Warranted by the Data".Phys. Rev. Lett.111 (18) 180604.arXiv:1312.1186.Bibcode:2013PhRvL.111r0604P.doi:10.1103/PhysRevLett.111.180604.PMID 24237501.S2CID 2577710.
^Tsallis, Constantino (2009).Introduction to nonextensive statistical mechanics: approaching a complex world (Online-Ausg. ed.). New York: Springer.ISBN 978-0-387-85358-1.
^Lutz, E. (2003). "Anomalous diffusion and Tsallis statistics in an optical lattice".Physical Review A.67 (5) 051402.arXiv:cond-mat/0210022.Bibcode:2003PhRvA..67e1402L.doi:10.1103/PhysRevA.67.051402.S2CID 119403353.
^Douglas, P.; Bergamini, S.; Renzoni, F. (2006)."Tunable Tsallis Distributions in Dissipative Optical Lattices"(PDF).Physical Review Letters.96 (11) 110601.Bibcode:2006PhRvL..96k0601D.doi:10.1103/PhysRevLett.96.110601.PMID 16605807.
^Burlaga, L. F.; - Viñas, A. F. (2005). "Triangle for the entropic index q of non-extensive statistical mechanics observed by Voyager 1 in the distant heliosphere".Physica A: Statistical Mechanics and Its Applications.356 (2–4): 375.arXiv:physics/0507212.Bibcode:2005PhyA..356..375B.doi:10.1016/j.physa.2005.06.065.S2CID 18823047.
^Liu, B.; Goree, J. (2008). "Superdiffusion and Non-Gaussian Statistics in a Driven-Dissipative 2D Dusty Plasma".Physical Review Letters.100 (5) 055003.arXiv:0801.3991.Bibcode:2008PhRvL.100e5003L.doi:10.1103/PhysRevLett.100.055003.PMID 18352381.S2CID 2022402.
^Pickup, R.; Cywinski, R.; Pappas, C.; Farago, B.; Fouquet, P. (2009). "Generalized Spin-Glass Relaxation".Physical Review Letters.102 (9) 097202.arXiv:0902.4183.Bibcode:2009PhRvL.102i7202P.doi:10.1103/PhysRevLett.102.097202.PMID 19392558.S2CID 6454082.
^Devoe, R. (2009). "Power-Law Distributions for a Trapped Ion Interacting with a Classical Buffer Gas".Physical Review Letters.102 (6) 063001.arXiv:0903.0637.Bibcode:2009PhRvL.102f3001D.doi:10.1103/PhysRevLett.102.063001.PMID 19257583.S2CID 15945382.
^Khachatryan, V.; Sirunyan, A.; Tumasyan, A.; Adam, W.; Bergauer, T.; Dragicevic, M.; Erö, J.; Fabjan, C.; Friedl, M.; Frühwirth, R.; Ghete, V. M.; Hammer, J.; Hänsel, S.; Hoch, M.; Hörmann, N.; Hrubec, J.; Jeitler, M.; Kasieczka, G.; Kiesenhofer, W.; Krammer, M.; Liko, D.; Mikulec, I.; Pernicka, M.; Rohringer, H.; Schöfbeck, R.; Strauss, J.; Taurok, A.; Teischinger, F.; Waltenberger, W.; et al. (2010). "Transverse-Momentum and Pseudorapidity Distributions of Charged Hadrons in pp Collisions at√s=7 TeV".Physical Review Letters.105 (2) 022002.arXiv:1005.3299.Bibcode:2010PhRvL.105b2002K.doi:10.1103/PhysRevLett.105.022002.PMID 20867699.S2CID 119196941.
^Chatrchyan, S.; Khachatryan, V.; Sirunyan, A. M.; Tumasyan, A.; Adam, W.; Bergauer, T.; Dragicevic, M.; Erö, J.; Fabjan, C.; Friedl, M.; Frühwirth, R.; Ghete, V. M.; Hammer, J.; Hänsel, S.; Hoch, M.; Hörmann, N.; Hrubec, J.; Jeitler, M.; Kiesenhofer, W.; Krammer, M.; Liko, D.; Mikulec, I.; Pernicka, M.; Rohringer, H.; Schöfbeck, R.; Strauss, J.; Taurok, A.; Teischinger, F.; Wagner, P.; et al. (2011). "Charged particle transverse momentum spectra in pp collisions at $√s= 0.9 and 7 TeV".Journal of High Energy Physics.2011 (8): 86.arXiv:1104.3547.Bibcode:2011JHEP...08..086C.doi:10.1007/JHEP08(2011)086.S2CID 122835798.
^Adare, A.; Afanasiev, S.; Aidala, C.; Ajitanand, N.; Akiba, Y.; Al-Bataineh, H.; Alexander, J.; Aoki, K.; Aphecetche, L.; Armendariz, R.; Aronson, S. H.; Asai, J.; Atomssa, E. T.; Averbeck, R.; Awes, T. C.; Azmoun, B.; Babintsev, V.; Bai, M.; Baksay, G.; Baksay, L.; Baldisseri, A.; Barish, K. N.; Barnes, P. D.; Bassalleck, B.; Basye, A. T.; Bathe, S.; Batsouli, S.; Baublis, V.; Baumann, C.; et al. (2011). "Measurement of neutral mesons inp+p collisions at√s=200 GeV and scaling properties of hadron production".Physical Review D.83 (5) 052004.arXiv:1005.3674.Bibcode:2011PhRvD..83e2004A.doi:10.1103/PhysRevD.83.052004.S2CID 85560021.
^Plastino, A. R.; Plastino, A. (1995). "Non-extensive statistical mechanics and generalized Fokker-Planck equation".Physica A: Statistical Mechanics and Its Applications.222 (1–4):347–354.Bibcode:1995PhyA..222..347P.doi:10.1016/0378-4371(95)00211-1.
^Tsallis, C.; Bukman, D. (1996). "Anomalous diffusion in the presence of external forces: Exact time-dependent solutions and their thermostatistical basis".Physical Review E.54 (3):R2197–R2200.arXiv:cond-mat/9511007.Bibcode:1996PhRvE..54.2197T.doi:10.1103/PhysRevE.54.R2197.PMID 9965440.S2CID 16272548.
^Abe, S. (2000). "Axioms and uniqueness theorem for Tsallis entropy".Physics Letters A.271 (1–2):74–79.arXiv:cond-mat/0005538.Bibcode:2000PhLA..271...74A.doi:10.1016/S0375-9601(00)00337-6.S2CID 119513564.
^Lyra, M.; Tsallis, C. (1998). "Nonextensivity and Multifractality in Low-Dimensional Dissipative Systems".Physical Review Letters.80 (1):53–56.arXiv:cond-mat/9709226.Bibcode:1998PhRvL..80...53L.doi:10.1103/PhysRevLett.80.53.S2CID 15039078.
^Baldovin, F.; Robledo, A. (2004). "Nonextensive Pesin identity: Exact renormalization group analytical results for the dynamics at the edge of chaos of the logistic map".Physical Review E.69 (4) 045202.arXiv:cond-mat/0304410.Bibcode:2004PhRvE..69d5202B.doi:10.1103/PhysRevE.69.045202.PMID 15169059.S2CID 30277614.
^Tsallis, C.; Gell-Mann, M.; Sato, Y. (2005)."Asymptotically scale-invariant occupancy of phase space makes the entropy Sq extensive".Proceedings of the National Academy of Sciences.102 (43):15377–82.arXiv:cond-mat/0502274.Bibcode:2005PNAS..10215377T.doi:10.1073/pnas.0503807102.PMC 1266086.PMID 16230624.
^Caruso, F.; Tsallis, C. (2008). "Nonadditive entropy reconciles the area law in quantum systems with classical thermodynamics".Physical Review E.78 (2) 021102.arXiv:cond-mat/0612032.Bibcode:2008PhRvE..78b1102C.doi:10.1103/PhysRevE.78.021102.PMID 18850781.S2CID 18006627.
^Andrade, J.; Da Silva, G.; Moreira, A.; Nobre, F.; Curado, E. (2010). "Thermostatistics of Overdamped Motion of Interacting Particles".Physical Review Letters.105 (26) 260601.arXiv:1008.1421.Bibcode:2010PhRvL.105z0601A.doi:10.1103/PhysRevLett.105.260601.PMID 21231636.S2CID 14831948.
^Ribeiro, M.; Nobre, F.; Curado, E. M. (2012)."Time evolution of interacting vortices under overdamped motion"(PDF).Physical Review E.85 (2) 021146.Bibcode:2012PhRvE..85b1146R.doi:10.1103/PhysRevE.85.021146.PMID 22463191.S2CID 25200027.
^Nobre, F.; Rego-Monteiro, M.; Tsallis, C. (2011). "Nonlinear Relativistic and Quantum Equations with a Common Type of Solution".Physical Review Letters.106 (14) 140601.arXiv:1104.5461.Bibcode:2011PhRvL.106n0601N.doi:10.1103/PhysRevLett.106.140601.PMID 21561176.S2CID 12679518.
^Majhi, Abhishek (2017). "Non-extensive statistical mechanics and black hole entropy from quantum geometry".Physics Letters B.775:32–36.arXiv:1703.09355.Bibcode:2017PhLB..775...32M.doi:10.1016/j.physletb.2017.10.043.S2CID 119397503.
^García-Morales, V.; Krischer, K. (2011)."Superstatistics in nanoscale electrochemical systems".Proceedings of the National Academy of Sciences.108 (49):19535–19539.Bibcode:2011PNAS..10819535G.doi:10.1073/pnas.1109844108.PMC 3241754.PMID 22106266.
^Beck, C.; Cohen, E. G. D. (2003). "Superstatistics".Physica A: Statistical Mechanics and Its Applications.322:267–275.arXiv:cond-mat/0205097.Bibcode:2003PhyA..322..267B.doi:10.1016/S0378-4371(03)00019-0.S2CID 261331784.