![]() Shellsort with gaps 23, 10, 4, 1 in action | |
| Class | Sorting algorithm |
|---|---|
| Data structure | Array |
| Worst-caseperformance | O(n2) (worst known worst case gap sequence) O(n log2n) (best known worst case gap sequence)[1] |
| Best-caseperformance | O(n logn) (most gap sequences) O(n log2n) (best known worst-case gap sequence)[2] |
| Averageperformance | depends on gap sequence |
| Worst-casespace complexity | О(n) total, O(1) auxiliary |
| Optimal | No |

Shellsort, also known asShell sort orShell's method, is anin-placecomparison sort. It can be understood as either a generalization of sorting by exchange (bubble sort) or sorting by insertion (insertion sort).[3] The method starts by sorting pairs of elements far apart from each other, then progressively reducing the gap between elements to be compared. By starting with far-apart elements, it can move some out-of-place elements into the position faster than a simple nearest-neighbor exchange. The running time of Shellsort is heavily dependent on the gap sequence it uses. For many practical variants, determining theirtime complexity remains anopen problem.
The algorithm was first published byDonald Shell in 1959, and has nothing to do with shells.[4][5]
Shellsort is an optimization ofinsertion sort that allows the exchange of items that are far apart. The idea is to arrange the list of elements so that, starting anywhere, taking everyhth element produces a sorted list. Such a list is said to beh-sorted. It can also be thought of ash interleaved lists, each individually sorted.[6] Beginning with large values ofh allows elements to move long distances in the original list, reducing large amounts of disorder quickly, and leaving less work for smallerh-sort steps to do.[7] If the list is thenk-sorted for some smaller integerk, then the list remainsh-sorted. A final sort withh = 1 ensures the list is fully sorted at the end,[6] but a judiciously chosen decreasing sequence ofh values leaves very little work for this final pass to do.
In simplistic terms, this means if we have an array of 1024 numbers, our first gap (h) could be 512. We then run through the list comparing each element in the first half to the element in the second half. Our second gap (k) is 256, which breaks the array into four sections (starting at 0, 256, 512, 768), and we make sure the first items in each section are sorted relative to each other, then the second item in each section, and so on. In practice the gap sequence could be anything, but the last gap is always 1 to finish the sort (effectively finishing with an ordinary insertion sort).
An example run of Shellsort with gaps 5, 3 and 1 is shown below.
| a1 | a2 | a3 | a4 | a5 | a6 | a7 | a8 | a9 | a10 | a11 | a12 | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Input data | 62 | 83 | 18 | 53 | 07 | 17 | 95 | 86 | 47 | 69 | 25 | 28 |
| After 5-sorting | 17 | 28 | 18 | 47 | 07 | 25 | 83 | 86 | 53 | 69 | 62 | 95 |
| After 3-sorting | 17 | 07 | 18 | 47 | 28 | 25 | 69 | 62 | 53 | 83 | 86 | 95 |
| After 1-sorting | 07 | 17 | 18 | 25 | 28 | 47 | 53 | 62 | 69 | 83 | 86 | 95 |
The first pass, 5-sorting, performs insertion sort on five separate subarrays (a1,a6,a11), (a2,a7,a12), (a3,a8), (a4,a9), (a5,a10). For instance, it changes the subarray (a1,a6,a11) from (62, 17, 25) to (17, 25, 62). The next pass, 3-sorting, performs insertion sort on the three subarrays (a1,a4,a7,a10), (a2,a5,a8,a11), (a3,a6,a9,a12). The last pass, 1-sorting, is an ordinary insertion sort of the entire array (a1,...,a12).
As the example illustrates, the subarrays that Shellsort operates on are initially short; later they are longer but almost ordered. In both cases insertion sort works efficiently.
Unlikeinsertion sort, Shellsort is not astable sort since gapped insertions transport equal elements past one another and thus lose their original order. It is anadaptive sorting algorithm in that it executes faster when the input is partially sorted.
This is aC# example using Marcin Ciura's gap sequence, with an inner insertion sort.
usingSystem.Collections.Generic;// Sort an array a[0...n-1].List<int>gaps=[701,301,132,57,23,10,4,1];// Ciura gap sequence// Start with the largest gap and work down to a gap of 1// similar to insertion sort but instead of 1, gap is being used in each stepforeach(intgapingaps){// Do a gapped insertion sort for every element in gaps// Each loop leaves a[0..gap-1] in gapped orderfor(inti=gap;i<n;++i){// save a[i] in temp and make a hole at position iinttemp=a[i];// shift earlier gap-sorted elements up until the correct location for a[i] is foundfor(intj=i;(j>=gap)&&(a[j-gap]>temp);j-=gap){a[j]=a[j-gap];}// put temp (the original a[i]) in its correct locationa[j]=temp;}}
The question of deciding which gap sequence to use is difficult. Every gap sequence that contains 1 yields a correct sort (as this makes the final pass an ordinary insertion sort); however, the properties of thus obtained versions of Shellsort may be very different. Too few gaps slows down the passes, and too many gaps produces an overhead.
The table below compares most proposed gap sequences published so far. Some of them have decreasing elements that depend on the size of the sorted array (N). Others are increasing infinite sequences, whose elements less thanN should be used in reverse order.
| OEIS | General term (k ≥ 1) | Concrete gaps | Worst-case time complexity | Author and year of publication |
|---|---|---|---|---|
| [e.g. whenN = 2p] | Shell, 1959[4] | |||
| Frank & Lazarus, 1960[8] | ||||
| A000225 | Hibbard, 1963[9] | |||
| A083318 | , prefixed with 1 | Papernov & Stasevich, 1965[10] | ||
| A003586 | Successive numbers of the form (3-smooth numbers) | Pratt, 1971[1] | ||
| A003462 | , not greater than | Knuth, 1973,[3] based onPratt, 1971[1] | ||
| A036569 | Incerpi &Sedgewick, 1985,[11]Knuth[3] | |||
| A036562 | , prefixed with 1 | Sedgewick, 1982[6] | ||
| A033622 | Sedgewick, 1986[12] | |||
| Unknown | Gonnet &Baeza-Yates, 1991[13] | |||
| A108870 | (or equivalently,) | Unknown | Tokuda, 1992[14] (misquote per OEIS) | |
| A102549 | Unknown (experimentally derived) | Unknown | Ciura, 2001[15] | |
| A366726 | Unknown | Lee, 2021[16] | ||
| Unknown | Skean, Ehrenborg, Jaromczyk, 2023[17] |
When the binary representation ofN contains many consecutive zeroes, Shellsort using Shell's original gap sequence makes Θ(N2) comparisons in the worst case. For instance, this case occurs forN equal to a power of two when elements greater and smaller than the median occupy odd and even positions respectively, since they are compared only in the last pass.
Although it has higher complexity than theO(N log N) that is optimal for comparison sorts, Pratt's version lends itself tosorting networks and has the same asymptotic gate complexity as Batcher'sbitonic sorter.
Gonnet and Baeza-Yates observed that Shellsort makes the fewest comparisons on average when the ratios of successive gaps are roughly equal to 2.2.[13] This is why their sequence with ratio 2.2 and Tokuda's sequence with ratio 2.25 prove efficient. However, it is not known why this is so. Sedgewick recommends using gaps which have lowgreatest common divisors or are pairwisecoprime.[18][failed verification]
With respect to the average number of comparisons, Ciura's sequence[15] has the best known performance; gaps greater than 701 were not determined but the sequence can be further extended according to the recursive formula.
Tokuda's sequence, defined by the simple formula, where,, can be recommended for practical applications.
If the maximum input size is small, as may occur if Shellsort is used on small subarrays by another recursive sorting algorithm such asquicksort ormerge sort, then it is possible to tabulate an optimal sequence for each input size.[19][20] For N = 128 and N = 1000, Ciura found that (1, 4, 9, 24, 85) and (1, 4, 10, 23, 57, 156, 409, 995) made the fewest number of comparisons on average respectively.[15]
The following property holds: afterh2-sorting of anyh1-sorted array, the array remainsh1-sorted.[21] Everyh1-sorted andh2-sorted array is also (a1h1+a2h2)-sorted, for any nonnegative integersa1 anda2. The worst-case complexity of Shellsort is therefore connected with theFrobenius problem: for given integersh1,...,hn with gcd = 1, the Frobenius numberg(h1,...,hn) is the greatest integer that cannot be represented asa1h1+ ... +anhn with nonnegative integera1,...,an. Using known formulae for Frobenius numbers, we can determine the worst-case complexity of Shellsort for several classes of gap sequences.[22] Proven results are shown in the above table.
Mark Allen Weiss proved that Shellsort runs inO(N logN) time when the input array is in reverse order.[23]
With respect to the average number of operations, none of the proven results concerns a practical gap sequence. For gaps that are powers of two, Espelid computed this average as.[24]Knuth determined the average complexity of sorting anN-element array with two gaps (h, 1) to be.[3] It follows that a two-pass Shellsort withh = Θ(N1/3) makes on averageO(N5/3) comparisons/inversions/running time.Yao found the average complexity of a three-pass Shellsort.[25] His result was refined byJanson and Knuth:[26] the average number of comparisons/inversions/running time made during a Shellsort with three gaps (ch,cg, 1), whereh andg are coprime, is in the first pass, in the second pass and in the third pass.ψ(h,g) in the last formula is a complicated function asymptotically equal to. In particular, whenh = Θ(N7/15) andg = Θ(N1/5), the average time of sorting isO(N23/15).
Based on experiments, it is conjectured that Shellsort withHibbard's gap sequence runs inO(N5/4) average time,[3] and that Gonnet and Baeza-Yates's sequence requires on average 0.41N ln N (ln ln N + 1/6) element moves.[13] Approximations of the average number of operations formerly put forward for other sequences fail when sorted arrays contain millions of elements.
The graph below shows the average number of element comparisons use by various gap sequences, divided by thetheoretical lower bound, i.e. log2N!. Ciuria's sequence 1, 4, 10, 23, 57, 132, 301, 701 (labelled Ci01) has been extended according to the formula.

Applying the theory ofKolmogorov complexity, Jiang,Li, andVitányi[27] proved the following lower bound for the order of the average number of operations/running time in ap-pass Shellsort: Ω(pN1+1/p) whenp ≤ log2N and Ω(pN) whenp > log2N.Therefore, Shellsort has prospects of running in an average time that asymptotically grows likeN logN only when using gap sequences whose number of gaps grows in proportion to the logarithm of the array size. It is, however, unknown whether Shellsort can reach this asymptotic order of average-case complexity, which is optimal for comparison sorts. The lower bound was improved byVitányi[28] for every number of passes towhere. This result implies for example the Jiang-Li-Vitányi lower bound for all-pass increment sequences and improves that lower bound for particular increment sequences. In fact all bounds (lower and upper) currently known for the average case are precisely matched by this lower bound. For example, this gives the new result that the Janson-Knuth upper bound is matched by the resulting lower bound for the used increment sequence, showing that three pass Shellsort for this increment sequence uses comparisons/inversions/running time.The formula allows us to search for increment sequences that yield lower bounds which are unknown; for example an increment sequence for four passes which has a lower bound greater than for the increment sequence. The lower bound becomes
The worst-case complexity of any version of Shellsort is of higher order: Plaxton,Poonen, andSuel showed that it grows at least as rapidly as.[29][30]Robert Cypher proved a stronger lower bound: when for all.[31]
Shellsort performs more operations and has highercache miss ratio thanquicksort. However, since it can be implemented using little code and does not use thecall stack, some implementations of theqsort function in theC standard library targeted atembedded systems use it instead of quicksort. Shellsort is, for example, used in theuClibc library.[32] For similar reasons, in the past, Shellsort was used in theLinux kernel.[33]
Shellsort can also serve as a sub-algorithm ofintrospective sort, to sort short subarrays and to prevent a slowdown when the recursion depth exceeds a given limit. This principle is employed, for instance, in thebzip2 compressor.[34]
Extensive experiments indicate that the sequence defined byα = 0.45454 < 5/11 performs significantly better than other sequences. The easiest way to compute⌊0.45454n⌋ is by(5 *n — 1)/11 using integer arithmetic.{{cite book}}: CS1 maint: location missing publisher (link)