TheHutter Prize is a cash prize funded byMarcus Hutter which rewardsdata compression improvements on a specific 1GB English text file, with the goal of encouraging research inartificial intelligence (AI).
Launched in 2006, the prize awards 5,000 euros for each one percent improvement (with 500,000 euros total funding)[1] in the compressed size of the fileenwik9, which is the larger of two files used in theLarge Text Compression Benchmark (LTCB);[2] enwik9 consists of the first 109 bytes of a specific version ofEnglish Wikipedia.[3] The ongoing[4] competition is organized by Hutter, Matt Mahoney, and Jim Bowery.[1]
The prize was announced on August 6, 2006[1] with a smaller text file:enwik8 consisting of 100MB. On February 21, 2020 both the dataset and the total prize pool were expanded by a factor of 10: fromenwik8 of 100MB toenwik9 of 1GB; from 50,000 to 500,000 euros.
The goal of the Hutter Prize is to encourage research inartificial intelligence (AI). The organizers believe that text compression and AI are equivalent problems. Hutter proved that the optimal behavior of a goal-seeking agent in an unknown but computable environment is to guess at each step that the environment is probably controlled by one of the shortest programs consistent with all interaction so far.[5] However, there is no general solution becauseKolmogorov complexity is not computable. Hutter proved that in the restricted case (calledAIXItl) where the environment is restricted to timet and spacel, a solution can be computed in timeO(t2l), which is still intractable.
The organizers further believe that compressing natural language text is ahard AI problem, equivalent to passing theTuring test. Thus, progress toward one goal represents progress toward the other. They argue thatpredicting which characters are most likely to occur next in a text sequence requires vast real-world knowledge. A text compressor must solve the same problem in order to assign the shortest codes to the most likely text sequences.[6]
Models like ChatGPT are not ideal for the Hutter Prize for a variety of reasons, they might take more computational resources than those allowed by the competition (computational and storage space).
The contest is open-ended. It is open to everyone. To enter, a competitor must submit a compression program and a decompressor that decompresses to the fileenwik9 (formerlyenwik8 up to 2017).[3] It is also possible to submit a compressed file instead of the compression program. The total size of the compressed file and decompressor (as a Win32 or Linux executable) must be less than or equal 99% of the previous prize winning entry. For each one percent improvement, the competitor wins 5,000 euros. The decompression program must also meet execution time and memory constraints.
Submissions must be published in order to allow independent verification. There is a 30-day waiting period for public comment before awarding a prize. In 2017, the rules were changed to require the release of the source code under afree software license, out of concern that "past submissions [which did not disclose their source code] had been useless to others and the ideas in them may be lost forever."[4]
| Author (enwik9) | Date | Program | Total size | Award |
|---|---|---|---|---|
| Kaido Orav and Byron Knoll | September 3, 2024 | fx2-cmix | 110,793,128 | 7,950€ |
| Kaido Orav | February 2, 2024 | fx-cmix | 112,578,322 | 6,911€ |
| Saurabh Kumar | July 16, 2023 | fast cmix | 114,156,155 | 5,187€ |
| Artemiy Margaritov | May 31, 2021 | starlit | 115,352,938 | 9,000€ |
| Alexander Rhatushnyak | July 4, 2019 | phda9v1.8 | 116,673,681 | No prize |
| Author (enwik8) | Date | Program | Total size | Award |
|---|---|---|---|---|
| Alexander Rhatushnyak | November 4, 2017 | phda9 | 15,284,944 | 2,085€ |
| Alexander Rhatushnyak | May 23, 2009 | decomp8 | 15,949,688 | 1,614€ |
| Alexander Rhatushnyak | May 14, 2007 | paq8hp12 | 16,481,655 | 1,732€ |
| Alexander Rhatushnyak | September 25, 2006 | paq8hp5 | 17,073,018 | 3,416€ |
| Matt Mahoney | March 24, 2006 | paq8f | 18,324,887 | No prize |
