Inprogramming andinformation security, abuffer overflow orbuffer overrun is ananomaly whereby aprogram writesdata to abuffer beyond the buffer'sallocated memory, overwriting adjacentmemory locations.
Buffers are areas of memory set aside to hold data, often while moving it from one section of a program to another, or between programs. Buffer overflows can often be triggered by malformed inputs; if one assumes all inputs will be smaller than a certain size and the buffer is created to be that size, then an anomalous transaction that produces more data could cause it to write past the end of the buffer. If this overwrites adjacent data or executable code, this may result in erratic program behavior, includingmemory access errors, incorrect results, andcrashes.
Exploiting the behavior of a buffer overflow is a well-knownsecurity exploit. On many systems, the memory layout of a program, or the system as a whole, is well defined. By sending in data designed to cause a buffer overflow, it is possible to write into areas known to holdexecutable code and replace it withmalicious code, or to selectively overwrite data pertaining to the program's state, therefore causing behavior that was not intended by the original programmer. Buffers are widespread inoperating system (OS) code, so it is possible to make attacks that performprivilege escalation and gain unlimited access to the computer's resources. The famedMorris worm in 1988 used this as one of its attack techniques.
Programming languages commonly associated with buffer overflows includeC andC++, which provide no built-in protection against accessing or overwriting data in any part of memory and do not automatically check that data written to anarray (the built-in buffer type) is within the boundaries of that array.Bounds checking can prevent buffer overflows, but requires additional code and processing time. Modern operating systems use a variety of techniques to combat malicious buffer overflows, notably byrandomizing the layout of memory, or deliberately leaving space between buffers and looking for actions that write into those areas ("canaries").
A buffer overflow occurs whendata written to a buffer also corrupts data values inmemory addresses adjacent to the destination buffer due to insufficientbounds checking.[1]: 41 This can occur when copying data from one buffer to another without first checking that the data fits within the destination buffer.
In the following example expressed inC, a program has two variables which are adjacent in memory: an 8-byte-long string buffer, A, and a two-bytebig-endian integer, B.
charA[8]="";unsignedshortB=1979;
Initially, A contains nothing but zero bytes, and B contains the number 1979.
| variable name | A | B | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| value | [null string] | 1979 | ||||||||
| hex value | 00 | 00 | 00 | 00 | 00 | 00 | 00 | 00 | 07 | BB |
Now, the program attempts to store thenull-terminated string"excessive" withASCII encoding in the A buffer.
strcpy(A,"excessive");
"excessive" is 9 characters long and encodes to 10 bytes including the null terminator, but A can take only 8 bytes. By failing to check the length of the string, it also overwrites the value of B:
| variable name | A | B | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| value | 'e' | 'x' | 'c' | 'e' | 's' | 's' | 'i' | 'v' | 25856 | |
| hex | 65 | 78 | 63 | 65 | 73 | 73 | 69 | 76 | 65 | 00 |
B's value has now been inadvertently replaced by a number formed from part of the character string. In this example "e" followed by a zero byte would become 25856.
Writing data past the end of allocated memory can sometimes be detected by the operating system to generate asegmentation fault error that terminates the process.
To prevent the buffer overflow from happening in this example, the call tostrcpy could be replaced withstrlcpy, which takes the maximum capacity of A (including a null-termination character) as an additional parameter and ensures that no more than this amount of data is written to A:
strlcpy(A,"excessive",sizeof(A));
When available, thestrlcpy library function is preferred overstrncpy which does not null-terminate the destination buffer if the source string's length is greater than or equal to the size of the buffer (the third argument passed to the function). ThereforeA may not be null-terminated and cannot be treated as a valid C-style string.
The techniques toexploit a buffer overflow vulnerability vary byarchitecture,operating system, and memory region. For example, exploitation on theheap (used for dynamically allocated memory), differs markedly from exploitation on thecall stack. In general, heap exploitation depends on the heap manager used on the target system, while stack exploitation depends on the calling convention used by the architecture and compiler.
There are several ways in which one can manipulate a program by exploiting stack-based buffer overflows:
The attacker designs data to cause one of these exploits, then places this data in a buffer supplied to users by the vulnerable code. If the address of the user-supplied data used to affect the stack buffer overflow is unpredictable, exploiting a stack buffer overflow to cause remote code execution becomes much more difficult. One technique that can be used to exploit such a buffer overflow is called "trampolining". Here, an attacker will find a pointer to the vulnerable stack buffer and compute the location of theirshellcode relative to that pointer. The attacker will then use the overwrite to jump to aninstruction already in memory which will make a second jump, this time relative to the pointer. That second jump will branch execution into the shellcode. Suitable instructions are often present in large code. TheMetasploit Project, for example, maintains a database of suitable opcodes, though it lists only those found in theWindows operating system.[4]
A buffer overflow occurring in the heap data area is referred to as a heap overflow and is exploitable in a manner different from that of stack-based overflows. Memory on the heap is dynamically allocated by the application at run-time and typically contains program data. Exploitation is performed by corrupting this data in specific ways to cause the application to overwrite internal structures such as linked list pointers. The canonical heap overflow technique overwrites dynamic memory allocation linkage (such asmalloc meta data) and uses the resulting pointer exchange to overwrite a program function pointer.
Microsoft'sGDI+ vulnerability in handlingJPEGs is an example of the danger a heap overflow can present.[5]
Manipulation of the buffer, which occurs before it is read or executed, may lead to the failure of an exploitation attempt. These manipulations can mitigate the threat of exploitation, but may not make it impossible. Manipulations could include conversion to upper or lower case, removal ofmetacharacters and filtering out of non-alphanumeric strings. However, techniques exist to bypass these filters and manipulations, such asalphanumeric shellcode,polymorphic code,self-modifying code, andreturn-to-libc attacks. The same methods can be used to avoid detection byintrusion detection systems. In some cases, including where code is converted intoUnicode,[6] the threat of the vulnerability has been misrepresented by the disclosers as only Denial of Service when in fact the remote execution of arbitrary code is possible.
In real-world exploits there are a variety of challenges which need to be overcome for exploits to operate reliably. These factors include null bytes in addresses, variability in the location of shellcode, differences between environments, and various counter-measures in operation.
A NOP-sled is the oldest and most widely known technique for exploiting stack buffer overflows.[7] It solves the problem of finding the exact address of the buffer by effectively increasing the size of the target area. To do this, much larger sections of the stack are corrupted with theno-op machine instruction. At the end of the attacker-supplied data, after the no-op instructions, the attacker places an instruction to perform a relative jump to the top of the buffer where theshellcode is located. This collection of no-ops is referred to as the "NOP-sled" because if the return address is overwritten with any address within the no-op region of the buffer, the execution will "slide" down the no-ops until it is redirected to the actual malicious code by the jump at the end. This technique requires the attacker to guess where on the stack the NOP-sled is instead of the comparatively small shellcode.[8]
Because of the popularity of this technique, many vendors ofintrusion prevention systems will search for this pattern of no-op machine instructions in an attempt to detect shellcode in use. A NOP-sled does not necessarily contain only traditional no-op machine instructions. Any instruction that does not corrupt the machine state to a point where the shellcode will not run can be used in place of the hardware assisted no-op. As a result, it has become common practice for exploit writers to compose the no-op sled with randomly chosen instructions which will have no real effect on the shellcode execution.[9]
While this method greatly improves the chances that an attack will be successful, it is not without problems. Exploits using this technique still must rely on some amount of luck that they will guess offsets on the stack that are within the NOP-sled region.[10] An incorrect guess will usually result in the target program crashing and could alert thesystem administrator to the attacker's activities. Another problem is that the NOP-sled requires a much larger amount of memory in which to hold a NOP-sled large enough to be of any use. This can be a problem when the allocated size of the affected buffer is too small and the current depth of the stack is shallow (i.e., there is not much space from the end of the current stack frame to the start of the stack). Despite its problems, the NOP-sled is often the only method that will work for a given platform, environment, or situation, and as such it is still an important technique.
The "jump to register" technique allows for reliable exploitation of stack buffer overflows without the need for extra room for a NOP-sled and without having to guess stack offsets. The strategy is to overwrite the return pointer with something that will cause the program to jump to a known pointer stored within a register which points to the controlled buffer and thus the shellcode. For example, if register A contains a pointer to the start of a buffer then any jump or call taking that register as an operand can be used to gain control of the flow of execution.[11]
DbgPrint() routine contains thei386 machine opcode forjmp esp.In practice a program may not intentionally contain instructions to jump to a particular register. The traditional solution is to find an unintentional instance of a suitableopcode at a fixed location somewhere within the program memory. FigureE on the left contains an example of such an unintentional instance of the i386jmp esp instruction. The opcode for this instruction isFF E4.[12] This two-byte sequence can be found at a one-byte offset from the start of the instructioncall DbgPrint at address0x7C941EED. If an attacker overwrites the program return address with this address the program will first jump to0x7C941EED, interpret the opcodeFF E4 as thejmp esp instruction, and will then jump to the top of the stack and execute the attacker's code.[13]
When this technique is possible the severity of the vulnerability increases considerably. This is because exploitation will work reliably enough to automate an attack with a virtual guarantee of success when it is run. For this reason, this is the technique most commonly used inInternet worms that exploit stack buffer overflow vulnerabilities.[14]
This method also allows shellcode to be placed after the overwritten return address on theWindows platform. Since executables are mostly based at address0x00400000 and x86 is alittle endian architecture, the last byte of the return address must be a null, which terminates the buffer copy and nothing is written beyond that. This limits the size of the shellcode to the size of the buffer, which may be overly restrictive.DLLs are located in high memory (above0x01000000) and so have addresses containing no null bytes, so this method can remove null bytes (or other disallowed characters) from the overwritten return address. Used in this way, the method is often referred to as "DLL trampolining".
Various techniques have been used to detect or prevent buffer overflows, with various tradeoffs. The following sections describe the choices and implementations available.
Assembly,C, andC++ are popular programming languages that are vulnerable to buffer overflow in part because they allow direct access to memory and are notstrongly typed.[15] C provides no built-in protection against accessing or overwriting data in any part of memory. More specifically, it does not check that data written to a buffer is within the boundaries of that buffer. The standard C++ libraries provide many ways of safely buffering data, and C++'sStandard Template Library (STL) provides containers that can optionally perform bounds checking if the programmer explicitly calls for checks while accessing data. For example, avector's member functionat() performs a bounds check and throws anout_of_rangeexception if the bounds check fails.[16] However, C++ behaves just like C if the bounds check is not explicitly called. Techniques to avoid buffer overflows also exist for C.
Languages that are strongly typed and do not allow direct memory access, such as COBOL, Java, Eiffel, Python, and others, prevent buffer overflow in most cases.[15] Many programming languages other than C or C++ provide runtime checking and in some cases even compile-time checking which might send a warning or raise an exception, while C or C++ would overwrite data and continue to execute instructions until erroneous results are obtained, potentially causing the program to crash. Examples of such languages includeAda,Eiffel,Lisp,Modula-2,Smalltalk,OCaml and such C-derivatives asCyclone,Rust andD. TheJava and.NET Framework bytecode environments also require bounds checking on all arrays. Nearly everyinterpreted language will protect against buffer overflow, signaling a well-defined error condition. Languages that provide enough type information to do bounds checking often provide an option to enable or disable it.Static code analysis can remove many dynamic bound and type checks, but poor implementations and awkward cases can significantly decrease performance. Software engineers should carefully consider the tradeoffs of safety versus performance costs when deciding which language and compiler setting to use.
The problem of buffer overflows is common in the C and C++ languages because they expose low level representational details of buffers as containers for data types. Buffer overflows can be avoided by maintaining a high degree of correctness in code that performs buffer management. It has also long been recommended to avoid standard library functions that are not bounds checked, such asgets,scanf andstrcpy. TheMorris worm exploited agets call infingerd.[17]
Well-written and tested abstract data type libraries that centralize and automatically perform buffer management, including bounds checking, can reduce the occurrence and impact of buffer overflows. The primary data types in languages in which buffer overflows are common are strings and arrays. Thus, libraries preventing buffer overflows in these data types can provide the vast majority of the necessary coverage. However, failure to use these safe libraries correctly can result in buffer overflows and other vulnerabilities, and naturally anybug in the library is also a potential vulnerability. "Safe" library implementations include "The Better String Library",[18] Vstr[19] and Erwin.[20] TheOpenBSD operating system'sC library provides thestrlcpy andstrlcat functions, but these are more limited than full safe library implementations.
In September 2007, Technical Report 24731, prepared by the C standards committee, was published.[21] It specifies a set of functions that are based on the standard C library's string and IO functions, with additional buffer-size parameters. However, the efficacy of these functions for reducing buffer overflows is disputable. They require programmer intervention on a per function call basis that is equivalent to intervention that could make the analogous older standard library functions buffer overflow safe.[22]
Buffer overflow protection is used to detect the most common buffer overflows by checking that thestack has not been altered when a function returns. If it has been altered, the program exits with asegmentation fault. Three such systems are Libsafe,[23] and theStackGuard[24] andProPolice[25]gcc patches.
Microsoft's implementation ofData Execution Prevention (DEP) mode explicitly protects the pointer to theStructured Exception Handler (SEH) from being overwritten.[26]
Stronger stack protection is possible by splitting the stack in two: one for data and one for function returns. This split is present in theForth language, though it was not a security-based design decision. Regardless, this is not a complete solution to buffer overflows, as sensitive data other than the return address may still be overwritten.
This type of protection is also not entirely accurate because it does not detect all attacks. Systems like StackGuard are more centered around the behavior of the attacks, which makes them efficient and faster in comparison to range-check systems.[27]
Buffer overflows work by manipulatingpointers, including stored addresses. PointGuard was proposed as a compiler-extension to prevent attackers from reliably manipulating pointers and addresses.[28] The approach works by having the compiler add code to automatically XOR-encode pointers before and after they are used. Theoretically, because the attacker does not know what value will be used to encode and decode the pointer, one cannot predict what the pointer will point to if it is overwritten with a new value. PointGuard was never released, but Microsoft implemented a similar approach beginning inWindows XP SP2 andWindows Server 2003 SP1.[29] Rather than implement pointer protection as an automatic feature, Microsoft added an API routine that can be called. This allows for better performance (because it is not used all of the time), but places the burden on the programmer to know when its use is necessary.
Because XOR is linear, an attacker may be able to manipulate an encoded pointer by overwriting only the lower bytes of an address. This can allow an attack to succeed if the attacker can attempt the exploit multiple times or complete an attack by causing a pointer to point to one of several locations (such as any location within a NOP sled).[30] Microsoft added a random rotation to their encoding scheme to address this weakness to partial overwrites.[31]
Executable space protection is an approach to buffer overflow protection that prevents execution of code on the stack or the heap. An attacker may use buffer overflows to insert arbitrary code into the memory of a program, but with executable space protection, any attempt to execute that code will cause an exception.
Some CPUs support a feature calledNX ("No eXecute") orXD ("eXecute Disabled") bit, which in conjunction with software, can be used to markpages of data (such as those containing the stack and the heap) as readable and writable but not executable.
Some Unix operating systems (e.g.OpenBSD,macOS) ship with executable space protection (e.g.W^X). Some optional packages include:
Newer variants of Microsoft Windows also support executable space protection, calledData Execution Prevention.[35]Proprietary add-ons include:
Executable space protection does not generally protect againstreturn-to-libc attacks, or any other attack that does not rely on the execution of the attackers code. However, on64-bit systems usingASLR, as described below, executable space protection makes it far more difficult to execute such attacks.
CHERI (Capability Hardware Enhanced RISC Instructions) is a computer processor technology designed to improve security. It operates at a hardware level by providing a hardware-enforced type (a CHERI capability) that authorises access to memory. Traditional pointers are replaced by addresses accompanied by metadata that limit what can be accessed through any given pointer.
Address space layout randomization (ASLR) is a computer security feature that involves arranging the positions of key data areas, usually including the base of the executable and position of libraries, heap, and stack, randomly in a process' address space.
Randomization of thevirtual memory addresses at which functions and variables can be found can make exploitation of a buffer overflow more difficult, but not impossible. It also forces the attacker to tailor the exploitation attempt to the individual system, which foils the attempts ofinternet worms.[38] A similar but less effective method is torebase processes and libraries in the virtual address space.
The use of deep packet inspection (DPI) can detect, at the network perimeter, very basic remote attempts to exploit buffer overflows by use of attack signatures andheuristics. This technique can block packets that have the signature of a known attack. It was formerly used in situations in which a long series of No-Operation instructions (known as a NOP-sled) was detected and the location of the exploit'spayload was slightly variable.
Packet scanning is not an effective method since it can only prevent known attacks and there are many ways that a NOP-sled can be encoded.Shellcode used by attackers can be madealphanumeric,metamorphic, orself-modifying to evade detection by heuristic packet scanners andintrusion detection systems.
Checking for buffer overflows and patching the bugs that cause them helps prevent buffer overflows. One common automated technique for discovering them isfuzzing.[39] Edge case testing can also uncover buffer overflows, as can static analysis.[40] Once a potential buffer overflow is detected it should be patched. This makes the testing approach useful for software that is in development, but less useful for legacy software that is no longer maintained or supported.
Buffer overflows were understood and partially publicly documented as early as 1972, when the Computer Security Technology Planning Study laid out the technique: "The code performing this function does not check the source and destination addresses properly, permitting portions of the monitor to be overlaid by the user. This can be used to inject code into the monitor that will permit the user to seize control of the machine."[41] Today, the monitor would be referred to as the kernel.
The earliest documented hostile exploitation of a buffer overflow was in 1988. It was one of several exploits used by theMorris worm to propagate itself over the Internet. The program exploited was aservice onUnix calledfinger.[42] Later, in 1995, Thomas Lopatic independently rediscovered the buffer overflow and published his findings on theBugtraq security mailing list.[43] A year later, in 1996,Elias Levy (also known as Aleph One) published inPhrack magazine the paper "Smashing the Stack for Fun and Profit",[44] a step-by-step introduction to exploiting stack-based buffer overflow vulnerabilities.
Since then, at least two major internet worms have exploited buffer overflows to compromise a large number of systems. In 2001, theCode Red worm exploited a buffer overflow in Microsoft'sInternet Information Services (IIS) 5.0[45] and in 2003 theSQL Slammer worm compromised machines runningMicrosoft SQL Server 2000.[46]
In 2003, buffer overflows present in licensedXbox games have been exploited to allow unlicensed software, includinghomebrew games, to run on the console without the need for hardware modifications, known asmodchips.[47] ThePS2 Independence Exploit also used a buffer overflow to achieve the same for thePlayStation 2. The Twilight hack accomplished the same with theWii, using a buffer overflow inThe Legend of Zelda: Twilight Princess.
{{cite journal}}:Cite journal requires|journal= (help){{cite journal}}:Cite journal requires|journal= (help){{cite journal}}:Cite journal requires|journal= (help){{cite journal}}:Cite journal requires|journal= (help){{cite web}}: CS1 maint: archived copy as title (link)| Hardware | |
|---|---|
| Virtual memory | |
| Memory segmentation | |
| Memory allocator | |
| Manual memory management | |
| Garbage collection | |
| Memory safety | |
| Issues | |
| Other | |
Authority control databases: National |
|---|
