A timeline of events related to
information theory,
quantum information theory andstatistical physics,
data compression,
error correcting codes and related subjects.
–Ludwig Boltzmann presents hisH-theorem, and with it the formula Σpi logpi for the entropy of a single gas particle –J. Willard Gibbs defines theGibbs entropy: the probabilities in the entropy formula are now taken as probabilities of the state of thewhole system –Harry Nyquist discusses quantifying"intelligence" and the speed at which it can be transmitted by a communication system –John von Neumann defines thevon Neumann entropy, extending the Gibbs entropy to quantum mechanics –Ralph Hartley introducesHartley information as the logarithm of the number of possible messages, with information being communicated when the receiver can distinguish one sequence of symbols from any other (regardless of any associated meaning) –Leó Szilárd analysesMaxwell's demon, showing how aSzilard engine can sometimes transform information into the extraction of useful work –Alan Turing introduces thedeciban as a measure of information inferred about the GermanEnigma machine cypher settings by theBanburismus process –Claude Shannon's theory of information is substantially complete –Richard W. Hamming inventsHamming codes for error detection and correction (to protect patent rights, the result is not published until 1950) –Claude E. Shannon publishesA Mathematical Theory of Communication –Claude E. Shannon publishesCommunication in the Presence of Noise –Nyquist–Shannon sampling theorem andShannon–Hartley law –Claude E. Shannon'sCommunication Theory of Secrecy Systems is declassified –Robert M. Fano publishesTransmission of Information. M.I.T. Press, Cambridge, Massachusetts –Shannon–Fano coding – Leon G. Kraft discoversKraft's inequality, which shows the limits ofprefix codes –Marcel J. E. Golay introducesGolay codes forforward error correction –Richard Hamming publishesHamming code paper, creates a new field of studyCoding theory –Solomon Kullback andRichard Leibler introduce theKullback–Leibler divergence –David A. Huffman inventsHuffman encoding, a method of finding optimalprefix codes forlosslessdata compression –August Albert Sardinas and George W. Patterson devise theSardinas–Patterson algorithm, a procedure to decide whether a givenvariable-length code is uniquely decodable –Irving S. Reed andDavid E. Muller proposeReed–Muller codes –Peter Elias introducesconvolutional codes –Eugene Prange first discussescyclic codes –Alexis Hocquenghem, and independently the next yearRaj Chandra Bose andDwijendra Kumar Ray-Chaudhuri, discoverBCH codes –Irving S. Reed andGustave Solomon proposeReed–Solomon codes –Robert G. Gallager proposeslow-density parity-check codes; they are unused for 30 years due to technical limitations –Dave Forney discussesconcatenated codes –Fumitada Itakura (Nagoya University) and Shuzo Saito (Nippon Telegraph and Telephone) developlinear predictive coding (LPC), a form ofspeech coding[1] –Andrew Viterbi reveals theViterbi algorithm, making decoding of convolutional codes practicable –Elwyn Berlekamp invents theBerlekamp–Massey algorithm; its application to decoding BCH and Reed–Solomon codes is pointed out byJames L. Massey the following year –Chris Wallace and David M. Boulton publish the first of many papers onMinimum Message Length (MML) statistical and inductive inference –Valerii Denisovich Goppa introducesGoppa codes –Jørn Justesen proposesJustesen codes, an improvement of Reed–Solomon codes –Nasir Ahmed proposes thediscrete cosine transform (DCT), which he develops with T. Natarajan andK. R. Rao in 1973;[2] the DCT later became the most widely usedlossy compression algorithm, the basis for multimedia formats such asJPEG,MPEG andMP3 –David Slepian andJack Wolf discover and prove theSlepian–Wolf coding limits for distributedsource coding[3] –Gottfried Ungerboeck gives the first paper ontrellis modulation; a more detailed exposition in 1982 leads to a raising of analogue modemPOTS speeds from 9.6 kbit/s to 33.6 kbit/s – Richard Pasco andJorma J. Rissanen develop effectivearithmetic coding techniques –Abraham Lempel andJacob Ziv develop Lempel–Ziv compression (LZ77) –Valerii Denisovich Goppa introducesalgebraic geometry codes –Phil Katz publishesthe.zip format includingDEFLATE (LZ77 + Huffman coding); later to become the most widely used archive container –Claude Berrou,Alain Glavieux andPunya Thitimajshima introduceTurbo codes –Michael Burrows andDavid Wheeler publish theBurrows–Wheeler transform, later to find use inbzip2 –Benjamin Schumacher coins the termqubit and proves the quantum noiseless coding theorem –David J. C. MacKay shows the connection between information theory, inference and machine learning in his book. –Jarosław Duda introduces firstAsymmetric numeral systems entropy coding: since 2014 popular replacement ofHuffman andarithmetic coding in compressors likeFacebook Zstandard,Apple LZFSE,CRAM orJPEG XL –Erdal Arıkan introducespolar codes, the first practical construction of codes that achieves capacity for a wide array of channels| By topic |
|
|---|---|
| Numeral systems | |
| By ancient cultures | |
| Controversies | |
| Other |
|
