Inmathematical optimization andcomputer science,heuristic (from Greekεὑρίσκωeurísko "I find, discover"[1]) is a technique designed forproblem solving more quickly when classic methods are too slow for finding an exact or approximate solution, or when classic methods fail to find any exact solution in asearch space. This is achieved by trading optimality, completeness,accuracy, or precision for speed. In a way, it can be considered a shortcut.
Aheuristic function, also simply called aheuristic, is afunction that ranks alternatives insearch algorithms at each branching step based on available information to decide which branch to follow. For example, it may approximate the exact solution.[2]
The objective of a heuristic is to produce a solution in a reasonable time frame that is good enough for solving the problem at hand. This solution may not be the best of all the solutions to this problem, or it may simply approximate the exact solution. But it is still valuable because finding it does not require a prohibitively long time.
Heuristics may produce results by themselves, or they may be used in conjunction with optimization algorithms to improve their efficiency (e.g., they may be used to generate good seed values).
Results aboutNP-hardness in theoretical computer science make heuristics the only viable option for a variety of complex optimization problems that need to be routinely solved in real-world applications.
Heuristics underlie the whole field of Artificial Intelligence and the computer simulation of thinking, as they may be used in situations where there are no knownalgorithms.[3]
One way of achieving the computational performance gain expected of a heuristic consists of solving a simpler problem whose solution is also a solution to the initial problem.
An example of approximation is described byJon Bentley for solving thetravelling salesman problem (TSP):
so as to select the order to draw using apen plotter. TSP is known to beNP-hard so an optimal solution for even a moderate size problem is difficult to solve. Instead, thegreedy algorithm can be used to give a good but not optimal solution (it is an approximation to the optimal answer) in a reasonably short amount of time. The greedy algorithm heuristic says to pick whatever is currently the best next step regardless of whether that prevents (or even makes impossible) good steps later. It is a heuristic in the sense that practice indicates it is a good enough solution, while theory indicates that there are better solutions (and even indicates how much better, in some cases).[4]
Another example of heuristic making an algorithm faster occurs in certain search problems. Initially, the heuristic tries every possibility at each step, like the full-space search algorithm. But it can stop the search at any time if the current possibility is already worse than the best solution already found. In such search problems, a heuristic can be used to try good choices first so that bad paths can be eliminated early (seealpha–beta pruning). In the case ofbest-first search algorithms, such asA* search, the heuristic improves the algorithm's convergence while maintaining its correctness as long as the heuristic isadmissible.
In theirTuring Award acceptance speech,Allen Newell andHerbert A. Simon discuss the heuristic search hypothesis: a physical symbol system will repeatedly generate and modify known symbol structures until the created structure matches the solution structure. Each following step depends upon the step before it, thus the heuristic search learns what avenues to pursue and which ones to disregard by measuring how close the current step is to the solution. Therefore, some possibilities will never be generated as they are measured to be less likely to complete the solution.
A heuristic method can accomplish its task by using search trees. However, instead of generating all possible solution branches, a heuristic selects branches more likely to produce outcomes than other branches. It is selective at each decision point, picking branches that are more likely to produce solutions.[5]
Antivirus software often uses heuristic rules for detecting viruses and other forms ofmalware. Heuristic scanning looks for code and/or behavioral patterns common to a class or family of viruses, with different sets of rules for different viruses. If a file or executing process is found to contain matching code patterns and/or to be performing that set of activities, then the scanner infers that the file is infected. The most advanced part of behavior-based heuristic scanning is that it can work against highly randomized self-modifying/mutating (polymorphic) viruses that cannot be easily detected by simpler string scanning methods. Heuristic scanning has the potential to detect future viruses without requiring the virus to be first detected somewhere else, submitted to the virus scanner developer, analyzed, and a detection update for the scanner provided to the scanner's users.
Some heuristics have a strong underlying theory; they are either derived in a top-down manner from the theory or are arrived at based on either experimental or real world data. Others are justrules of thumb based on real-world observation or experience without even a glimpse of theory. The latter are exposed to a larger number of pitfalls.
When a heuristic is reused in various contexts because it has been seen to "work" in one context, without having been mathematically proven to meet a given set of requirements, it is possible that the current data set does not necessarily represent future data sets (see:overfitting) and that purported "solutions" turn out to be akin to noise.
Statistical analysis can be conducted when employing heuristics to estimate theprobability of incorrect outcomes. To use a heuristic for solving asearch problem or aknapsack problem, it is necessary to check that the heuristic isadmissible. Given a heuristic function meant to approximate the true optimaldistance to the goal node in adirected graph containing total nodes orvertices labeled, "admissible" means roughly that the heuristic underestimates the cost to the goal or formally that forall where.
If a heuristic is not admissible, it may never find the goal, either by ending up in a dead end of graph or by skipping back and forth between two nodes and where.
The word "heuristic" came into usage in the early 19th century. It is formed irregularly from theGreek wordheuriskein, meaning "to find".[6]