 | This articleneeds additional citations forverification. Please helpimprove this article byadding citations to reliable sources. Unsourced material may be challenged and removed. Find sources: "Substitution cipher" – news ·newspapers ·books ·scholar ·JSTOR(March 2009) (Learn how and when to remove this message) |
| General information | |
|---|---|
| Field | Cryptography |
| First published | Around 850 CE |
| Technical details | |
| Related methods | Transposition cipher,polyalphabetic cipher,homophonic substitution cipher,one-time pad |
| Key size | Varies (typically 88 bits for mixed alphabet simple substitution) |
| Cryptanalysis | Frequency analysis |
Incryptography, asubstitution cipher is a method ofencrypting that creates theciphertext (its output) by replacing units of theplaintext (its input) in a defined manner, with the help of a key; the "units" may be single letters (the most common), pairs of letters, triplets of letters, mixtures of the above, and so forth. The receiver deciphers the text by performing the inverse substitution process to extract the original message.
Substitution ciphers can be compared withtransposition ciphers. In a transposition cipher, the units of the plaintext are rearranged in a different and usually quite complex order, but the units themselves are left unchanged. By contrast, in a substitution cipher, the units of the plaintext are retained in the same sequence in the ciphertext, but the units themselves are altered.
There are a number of different types of substitution cipher. If the cipher operates on single letters, it is termed asimple substitution cipher; a cipher that operates on larger groups of letters is termedpolygraphic. Amonoalphabetic cipher uses fixed substitution over the entire message, whereas apolyalphabetic cipher uses a number of substitutions at different positions in the message, where a unit from the plaintext is mapped to one of several possibilities in the ciphertext and vice versa.
The first ever published description of how to crack simple substitution ciphers was given byAl-Kindi inA Manuscript on Deciphering Cryptographic Messages written around 850 AD. The method he described is now known asfrequency analysis.
The simplest substitution ciphers are theCaesar cipher andAtbash cipher. Here single letters are substituted (referred to assimple substitution). It can be demonstrated by writing out the alphabet twice, once in regular order and again with the letters shifted by some number of steps or reversed to represent theciphertext alphabet (or substitution alphabet).
The substitution alphabet could also be scrambled in a more complex fashion, in which case it is called amixed alphabet orderanged alphabet. Traditionally, mixed alphabets may be created by first writing out a keyword, removing repeated letters in it, then writing all the remaining letters in the alphabet in the usual order.
Using this system, the keyword "zebras" gives us the following alphabets:
| Plaintext alphabet | ABCDEFGHIJKLMNOPQRSTUVWXYZ |
|---|---|
| Ciphertext alphabet | ZEBRASCDFGHIJKLMNOPQTUVWXY |
A message
flee at once. we are discovered!
enciphers to
SIAA ZQ LKBA. VA ZOA RFPBLUAOAR!
And the keyword "grandmother" gives us the following alphabets:
| Plaintext alphabet | ABCDEFGHIJKLMNOPQRSTUVWXYZ |
|---|---|
| Ciphertext alphabet | GRANDMOTHEBCFIJKLPQSUVWXYZ |
The same message
flee at once. we are discovered!
enciphers to
MCDD GS JIAD. WD GPD NHQAJVDPDN!
Usually the ciphertext is written out in blocks of fixed length, omitting punctuation and spaces; this is done to disguise word boundaries from theplaintext and to help avoid transmission errors. These blocks are called "groups", and sometimes a "group count" (i.e. the number of groups) is given as an additional check. Five-letter groups are often used, dating from when messages used to be transmitted bytelegraph:
SIAAZ QLKBA VAZOA RFPBL UAOAR
If the length of the message happens not to be divisible by five, it may be padded at the end with "nulls". These can be any characters that decrypt to obvious nonsense, so that the receiver can easily spot them and discard them.
The ciphertext alphabet is sometimes different from the plaintext alphabet; for example, in thepigpen cipher, the ciphertext consists of a set of symbols derived from a grid. For example:
Such features make little difference to the security of a scheme, however; at the very least, any set of strange symbols can be transcribed back into an A-Z alphabet and dealt with as normal.
In lists and catalogues for salespeople, a very simple encryption is sometimes used to replace numeric digits by letters.
| Plaintext digits | 1234567890 |
|---|---|
| Ciphertext alphabets | MAKEPROFIT [1] |
Examples: MAT would be used to represent 120, PAPR would be used for 5256, and OFTK would be used for 7803.
Although the traditional keyword method for creating a mixed substitution alphabet is simple, a serious disadvantage is that the last letters of the alphabet (which are mostly low frequency) tend to stay at the end. A stronger way of constructing a mixed alphabet is to generate the substitution alphabet completely randomly.
Although the number of possible substitution alphabets is very large (26! ≈ 288.4, or about88 bits), this cipher is not very strong, and is easily broken. Provided the message is of reasonable length (see below), thecryptanalyst can deduce the probable meaning of the most common symbols by analyzing thefrequency distribution of the ciphertext. This allows formation of partial words, which can be tentatively filled in, progressively expanding the (partial) solution (seefrequency analysis for a demonstration of this). In some cases, underlying words can also be determined from the pattern of their letters; for example, theEnglish wordstater,ninth, andpaper all have the patternABACD. Many people solve such ciphers for recreation, as withcryptogram puzzles in the newspaper.
According to theunicity distance ofEnglish, 27.6 letters of ciphertext are required to crack a mixed alphabet simple substitution. In practice, typically about 50 letters are needed, although some messages can be broken with fewer if unusual patterns are found. In other cases, the plaintext can be contrived to have a nearly flat frequency distribution, and much longer plaintexts will then be required by the cryptanalyst.
One once-common variant of the substitution cipher is thenomenclator. Named after the public official who announced the titles of visiting dignitaries, thiscipher uses a smallcode sheet containing letter, syllable and word substitution tables, sometimes homophonic, that typically converted symbols into numbers. Originally the code portion was restricted to the names of important people, hence the name of the cipher; in later years, it covered many common words and place names as well. The symbols for whole words (codewords in modern parlance) and letters (cipher in modern parlance) were not distinguished in the ciphertext. TheRossignols'Great Cipher used byLouis XIV of France was one.
Nomenclators were the standard fare ofdiplomatic correspondence,espionage, and advanced politicalconspiracy from the early fifteenth century to the late eighteenth century; most conspirators were and have remained less cryptographically sophisticated. Althoughgovernmentintelligencecryptanalysts were systematically breaking nomenclators by the mid-sixteenth century, and superior systems had been available since 1467, the usual response tocryptanalysis was simply to make the tables larger. By the late eighteenth century, when the system was beginning to die out, some nomenclators had 50,000 symbols.[citation needed]
Nevertheless, not all nomenclators were broken; today, cryptanalysis of archived ciphertexts remains a fruitful area ofhistorical research.
An early attempt to increase the difficulty of frequency analysis attacks on substitution ciphers was to disguise plaintext letter frequencies byhomophony. In these ciphers, plaintext letters map to more than one ciphertext symbol. Usually, the highest-frequency plaintext symbols are given more equivalents than lower frequency letters. In this way, the frequency distribution is flattened, making analysis more difficult.
Since more than 26 characters will be required in the ciphertext alphabet, various solutions are employed to invent larger alphabets. Perhaps the simplest is to use a numeric substitution 'alphabet'. Another method consists of simple variations on the existing alphabet; uppercase, lowercase, upside down, etc. More artistically, though not necessarily more securely, some homophonic ciphers employed wholly invented alphabets of fanciful symbols.
Thebook cipher is a type of homophonic cipher, one example being theBeale ciphers. This is a story of buried treasure that was described in 1819–21 by use of a ciphered text that was keyed to the Declaration of Independence. Here each ciphertext character was represented by a number. The number was determined by taking the plaintext character and finding a word in the Declaration of Independence that started with that character and using the numerical position of that word in the Declaration of Independence as the encrypted form of that letter. Since many words in the Declaration of Independence start with the same letter, the encryption of that character could be any of the numbers associated with the words in the Declaration of Independence that start with that letter. Deciphering the encrypted text characterX (which is a number) is as simple as looking up the Xth word of the Declaration of Independence and using the first letter of that word as the decrypted character.
Another homophonic cipher was described by Stahl in 1973[2][3][4] and was one of the first[citation needed] attempts to provide for computer security of data systems in computers through encryption. Stahl constructed the cipher in such a way that the number of homophones for a given character was in proportion to the frequency of the character, thus making frequency analysis much more difficult.
Francesco I Gonzaga,Duke of Mantua, used the earliest known example of a homophonic substitution cipher in 1401 for correspondence with one Simone de Crema.[5][6]
Mary, Queen of Scots, while imprisoned by Elizabeth I, during the years from 1578 to 1584 used homophonic ciphers with additional encryption using a nomenclator for frequent prefixes, suffixes, and proper names while communicating with her allies includingMichel de Castelnau.[7]
The work ofAl-Qalqashandi (1355–1418), based on the earlier work ofIbn al-Durayhim (1312–1359), contained the first published discussion of the substitution and transposition of ciphers, as well as the first description of a polyalphabetic cipher, in which each plaintext letter is assigned more than one substitute.[8] Polyalphabetic substitution ciphers were later described in 1467 byLeone Battista Alberti in the form of disks.Johannes Trithemius, in his bookSteganographia (Ancient Greek for "hidden writing") introduced the now more standard form of atableau (see below; ca. 1500 but not published until much later). A more sophisticated version using mixed alphabets was described in 1563 byGiovanni Battista della Porta in his book,De Furtivis Literarum Notis (Latin for "On concealed characters in writing").
In a polyalphabetic cipher, multiple cipher alphabets are used. To facilitate encryption, all the alphabets are usually written out in a largetable, traditionally called atableau. The tableau is usually 26×26, so that 26 full ciphertext alphabets are available. The method of filling the tableau, and of choosing which alphabet to use next, defines the particular polyalphabetic cipher. All such ciphers are easier to break than once believed, as substitution alphabets are repeated for sufficiently large plaintexts.
One of the most popular was that ofBlaise de Vigenère. First published in 1585, it was considered unbreakable until 1863, and indeed was commonly calledle chiffre indéchiffrable (French for "indecipherable cipher").
In theVigenère cipher, the first row of the tableau is filled out with a copy of the plaintext alphabet, and successive rows are simply shifted one place to the left. (Such a simple tableau is called atabula recta, and mathematically corresponds to adding the plaintext and key letters,modulo 26.) A keyword is then used to choose which ciphertext alphabet to use. Each letter of the keyword is used in turn, and then they are repeated again from the beginning. So if the keyword is 'CAT', the first letter of plaintext is enciphered under alphabet 'C', the second under 'A', the third under 'T', the fourth under 'C' again, and so on, or if the keyword is 'RISE', the first letter of plaintext is enciphered under alphabet 'R', the second under 'I', the third under 'S', the fourth under 'E', and so on. In practice, Vigenère keys were often phrases several words long.
In 1863,Friedrich Kasiski published a method (probably discovered secretly and independently before theCrimean War byCharles Babbage) which enabled the calculation of the length of the keyword in a Vigenère ciphered message. Once this was done, ciphertext letters that had been enciphered under the same alphabet could be picked out and attacked separately as a number of semi-independent simple substitutions - complicated by the fact that within one alphabet letters were separated and did not form complete words, but simplified by the fact that usually atabula recta had been employed.
As such, even today a Vigenère type cipher should theoretically be difficult to break if mixed alphabets are used in the tableau, if the keyword is random, and if the total length of ciphertext is less than 27.67 times the length of the keyword.[9] These requirements are rarely understood in practice, and so Vigenère enciphered message security is usually less than might have been.
Other notable polyalphabetics include:
Modernstream ciphers can also be seen, from a sufficiently abstract perspective, to be a form of polyalphabetic cipher in which all the effort has gone into making thekeystream as long and unpredictable as possible.
In a polygraphic substitution cipher, plaintext letters are substituted in larger groups, instead of substituting letters individually. The first advantage is that the frequency distribution is much flatter than that of individual letters (though not actually flat in real languages; for example, 'OS' is much more common than 'RÑ' in Spanish). Second, the larger number of symbols requires correspondingly more ciphertext to productively analyze letter frequencies.
To substitutepairs of letters would take a substitution alphabet 676 symbols long (). In the sameDe Furtivis Literarum Notis mentioned above, della Porta actually proposed such a system, with a 20 x 20 tableau (for the 20 letters of the Italian/Latin alphabet he was using) filled with 400 uniqueglyphs. However, the system was impractical and probably never actually used.
The earliest practicaldigraphic cipher (pairwise substitution), was the so-calledPlayfair cipher, invented by SirCharles Wheatstone in 1854. In this cipher, a 5 x 5 grid is filled with the letters of a mixed alphabet (two letters, usually I and J, are combined). A digraphic substitution is then simulated by taking pairs of letters as two corners of a rectangle, and using the other two corners as the ciphertext (see thePlayfair cipher main article for a diagram). Special rules handle double letters and pairs falling in the same row or column. Playfair was in military use from theBoer War throughWorld War II.
Several other practical polygraphics were introduced in 1901 byFelix Delastelle, including thebifid andfour-square ciphers (both digraphic) and thetrifid cipher (probably the first practical trigraphic).
TheHill cipher, invented in 1929 byLester S. Hill, is a polygraphic substitution which can combine much larger groups of letters simultaneously usinglinear algebra. Each letter is treated as a digit inbase 26: A = 0, B =1, and so on. (In a variation, 3 extra symbols are added to make thebasisprime.) A block of n letters is then considered as avector of ndimensions, and multiplied by a n x nmatrix,modulo 26. The components of the matrix are the key, and should berandom provided that the matrix is invertible in (to ensure decryption is possible). A mechanical version of the Hill cipher of dimension 6 was patented in 1929.[10]
The Hill cipher is vulnerable to aknown-plaintext attack because it is completelylinear, so it must be combined with somenon-linear step to defeat this attack. The combination of wider and wider weak, lineardiffusive steps like a Hill cipher, with non-linear substitution steps, ultimately leads to asubstitution–permutation network (e.g. aFeistel cipher), so it is possible – from this extreme perspective – to consider modernblock ciphers as a type of polygraphic substitution.
| This sectionneeds additional citations forverification. Please helpimprove this article byadding citations to reliable sources in this section. Unsourced material may be challenged and removed.(February 2017) (Learn how and when to remove this message) |
Between aroundWorld War I and the widespread availability ofcomputers (for some governments this was approximately the 1950s or 1960s; for other organizations it was a decade or more later; for individuals it was no earlier than 1975), mechanical implementations of polyalphabetic substitution ciphers were widely used. Several inventors had similar ideas about the same time, androtor cipher machines were patented four times in 1919. The most important of the resulting machines was theEnigma, especially in the versions used by theGerman military from approximately 1930. TheAllies also developed and used rotor machines (e.g.,SIGABA andTypex).
All of these were similar in that the substituted letter was chosenelectrically from amongst the huge number of possible combinations resulting from the rotation of several letter disks. Since one or more of the disks rotated mechanically with each plaintext letter enciphered, the number of alphabets used was astronomical. Early versions of these machine were, nevertheless, breakable.William F. Friedman of the US Army'sSIS early found vulnerabilities inHebern's rotor machine, and theGovernment Code and Cypher School'sDillwyn Knox solved versions of the Enigma machine (those without the "plugboard") well beforeWWII began. Traffic protected by essentially all of the German military Enigmas was broken by Allied cryptanalysts, most notably those atBletchley Park, beginning with the German Army variant used in the early 1930s. This version was broken by inspired mathematical insight byMarian Rejewski inPoland.
As far as is publicly known, no messages protected by theSIGABA andTypex machines were ever broken during or near the time when these systems were in service.
One type of substitution cipher, theone-time pad, is unique. It was invented near the end of World War I byGilbert Vernam andJoseph Mauborgne in the US. It was mathematically proven unbreakable byClaude Shannon, probably duringWorld War II; his work was first published in the late 1940s. In its most common implementation, the one-time pad can be called a substitution cipher only from an unusual perspective; typically, the plaintext letter is combined (not substituted) in some manner (e.g.,XOR) with the key material character at that position.
The one-time pad is, in most cases, impractical as it requires that the key material be as long as the plaintext,actuallyrandom, used once andonly once, and kept entirely secret from all except the sender and intended receiver. When these conditions are violated, even marginally, the one-time pad is no longer unbreakable.Soviet one-time pad messages sent from the US for a brief time during World War II usednon-random key material. US cryptanalysts, beginning in the late 40s, were able to, entirely or partially, break a few thousand messages out of several hundred thousand. (SeeVenona project)
In a mechanical implementation, rather like theRockex equipment, the one-time pad was used for messages sent on theMoscow-Washingtonhot line established after theCuban Missile Crisis.
Substitution ciphers as discussed above, especially the older pencil-and-paper hand ciphers, are no longer in serious use. However, the cryptographic concept of substitution carries on even today. From an abstract perspective, modern bit-orientedblock ciphers (e.g.,DES, orAES) can be viewed as substitution ciphers on a largebinary alphabet. In addition, block ciphers often include smaller substitution tables calledS-boxes. See alsosubstitution–permutation network.
