Dissecting DLL with Static and Dynamic Analysis

Shreethaar Arunagirinathan
@0x251e

./whoami

  • Shreethaar Arunagirinathan
  • Computer Science @ UUM
  • MCC 2024 Alumni
  • CTFs @ RE:UN10N
    • DFIR
    • RE
    • OSINT

./toc

  1. Case study of SEO#LURKER campaign
  2. Why reversing DLL matters
  3. What is DLL
  4. How DLL works
  5. Create simple DLL
  6. Structure of DLL
  7. Static Analysis of DLL
  8. Dynamic Analysis of DLL
  9. CTF Challenge Walkthrough
    Materials: https://github.com/shreethaar/RE-DLL

./case_study

  • Target: WinSCP users (430K searches/3 months) via fake Google Ads
  • User downloads WinSCP_v.6.1.zip → runs setup.exe
  • Appears legitimate: setup.exe = signed pythonw.exe ✅
  • Hidden threat: python311.dll = MALICIOUS ❌

The Deception

  • Traditional analysis: "Legitimate signed binary = safe"
  • Reality: Malicious DLL sideloaded by legitimate process
  • Full system compromise through DLL hijacking
  • Persistence via scheduled tasks + more DLL chains

right

References: https://www.iaesjournal.com/4980-2/

./why_reversing_dll_matters

  • Modern malware uses DLL to employ various tactics
  • Common in reflective and process injections
  • More stealthy behavior and modular payloads
  • DLL is one of the top 5 file types listed in MalwareBazaar
  • Vulnerabilities analysis that lead to supply chain risks
  • Understand how applications interacts with external libraries

./what_is_dll

  • DLL (Dynamic Link Library) contains code and data which used by more than one program at the same time
  • Contains exported functions to be called by other modules
  • Any process that uses Windows API uses dynamic linking:
    • Comdlg32 DLL use for dialog box related functions (GetOpenFileName, GetSaveFileName)
  • Dynamic linking allows a module to include only the information needed to locate an exported DLL function at loadtime or runtime
  • Benefits of using DLL:
    • Reduce memory and disk usage as duplication of code is reduced
    • Promotes modular architecture, eases deployment and installation
  • Security Matters with DLL:
    • Code sharing: Multiple applications can use the same DLL
    • Runtime Loading: DLLs loaded when needed (attack opportunity)
    • Modularity: Can be replaced or hijacked without changing main executable

./how_dll_works

Before understanding the working of DLL, understand the difference between static and dynamic linking:

  • If static linking is used, the linker copies the object code from the library directly into the final executable. This results in a larger binary that is self-contained
  • If dynamic linking is used, the linker does not copy the object code. Instead, it inserts references (or stubs) pointing to the external library (e.g., a DLL).

In windows the file extensions are as follows: Static libraries (.lib) and dynamic libraries (.dll). The main difference is that static libraries are linked to the executable at compile time; whereas dynamic linked libraries are not linked until run-time.

./how_dll_works

Static vs Dynamic Linking

DLL Loading with Relative Addressing

DLL Load Flow

./create_simple_dll

// simple-dll.c
#include <windows.h>
/* Export a very small function */
__declspec(dllexport) int add(int a, int b) {
    return a + b;
}
__declspec(dllexport) void CALLBACK getString(HWND hwnd, HINSTANCE hinst, LPSTR cmdLine, int nCmdShow) {
    MessageBoxA(hwnd, "Hello from DLL", "getString()", MB_OK | MB_ICONINFORMATION);
}
__declspec(dllexport) void processData(char* input, int length) {
    for (int i = 0; i < length; i++) {
        input[i] ^= 0x6F;
    }
}
BOOL WINAPI DllMain(HINSTANCE hinstDLL, DWORD fdwReason, LPVOID lpvReserved) {
    switch (fdwReason) {        // perform actions based on the reason of calling 
    case DLL_PROCESS_ATTACH:    // initialize once for each new process
        MessageBoxA(NULL, "DLL Loaded", "DLL Event", MB_OK | MB_ICONINFORMATION);
        break;
    case DLL_THREAD_ATTACH:     // do thread-specific initialization
        break;
    case DLL_THREAD_DETACH:     // do thread-specific cleanup
        break;
    case DLL_PROCESS_DETACH:    // do not do cleanup if process termination scenario
        MessageBoxA(NULL, "DLL Unloaded", "DLL Event", MB_OK | MB_ICONINFORMATION);
        break;
    }
    return TRUE;
}

./create_simple_dll

// simple-loader.c
#include <windows.h>
#include <stdio.h>
#include <string.h>
typedef int (*add_func_t)(int, int);
typedef void (*processData_func_t)(char*, int);

int main(void) {
    char string[] = "Hello from EXE";
    printf("%s\n", string);
    const char* dllName = "simple-dl.dll"; /* the DLL filename */
    HMODULE h = LoadLibraryA(dllName);
    if (!h) {
        DWORD err = GetLastError();
        printf("LoadLibraryA failed (error %lu). Make sure %s is in the same folder.\n", err, dllName);
        return 1;
    }
    add_func_t add = (add_func_t)GetProcAddress(h, "add");
    if (!add) {
        DWORD err = GetLastError();
        printf("GetProcAddress failed (error %lu).\n", err);
        FreeLibrary(h);
        return 1;
    }
    int a = 7, b = 5;
    int result = add(a, b);
    printf("add(%d, %d) = %d\n", a, b, result);

    processData_func_t processData = (processData_func_t)GetProcAddress(h, "processData");
    if (!processData) {
        printf("GetProcAddress for processData failed.\n");
    } else {
        char testData[] = "yappare is the best hacker\n";
        printf("Original: %s\n", testData);
        processData(testData, 6);
        printf("Processed: %s\n", testData);
    }
    FreeLibrary(h);
    return 0;
}

./structure_of_dll

DLL File Structure Overview
  1. DOS Stub - Legacy DOS compatibility header
  2. PE Signature - "PE\0\0" magic Signature
  3. COFF File Header - Machine type, number of sections, characteristics
  4. Optional Header - Magic number, entry point, base address, RVA
  5. Section Headers - Sections names, virtual address, offset

./structure_of_dll

  1. View COFF headers
dumpbin /headers simple-dll.dll
  1. Examine code sections
dumpbin /rawdata /section:.text simple-dll.dll
dumpbin /disasm /section:.text simple-dll.dll
  1. Export information
dumpbin /exports simple-dll.dll 

./structure_of_dll

  1. View dependecies
dumpbin /dependents loader.exe 
  1. View imports
dumpbin /imports loader.exe

./static_analysis_dll

static1

./static_analysis_dll

static2

./static_analysis_dll

static3

./dynamic_analysis_dll

We will be using the simple-loader.exe to debug the dll.
Yes, it is possible to use rundll32 to debug but the limitations are:

  • simple-dll.dll exported functions required arguments to be passed
  • rundll32 expect a callback function from DLL
__declspec(dllexport) void CALLBACK CustomFunc(HWND hwnd, HINSTANCE hinst, LPSTR cmdLine, int nCmdShow);
  • Calling convention of CALLBACK(__stdcall) with a return type of void
  • Is best use with Windows system DLL functions, like WinAPI

./dynamic_analysis_dll

dynamic1

./dynamic_analysis_dll

dynamic2

./dynamic_analysis_dll

Once in simple-dll.dll section, set breakpoint for exported functions

Run and you will switch how it switch from simple-loader.exe to simple-dll.dll to execute function, value being passed and return back to the loader.

./dynamic_analysis_dll

dynamic3

./ctf_challenge_walkthrough

ctf1

./ctf_challenge_walkthrough

ctf2

./ctf_challenge_walkthrough

ctf3

./ctf_challenge_walkthrough

ctf4

./ctf_challenge_walkthrough

Key information derived from IDA:

  1. Two exported functions: Run() and VoidFunc()
  2. VoidFunc() contains WinAPI that retrieved something from resource section of the DLL
  3. v1 byte block (32-bit) passed to function sub_6EB41648
  4. In sub_6EB41648 function, contain XOR operation
  5. while(v3) loop is clearing v0 which is EAX by replacing it will 0 (*v0 = 0)

./ctf_challenge_walkthrough

Next step:

  1. Run x32dbg with rundll32 (C:\Windows\SysWOW64\rundll32.exe)
  2. Command line: "C:\Windows\SysWOW64\rundll32.exe" C:\Users\flarevm\Desktop\RE-DLL\challenge\Helper.dll,VoidFunc
  3. Breakpoint before while loop and after sub_6EB41648
  4. Inspect memory what is contained

./ctf_challenge_walkthrough

ctf5

./ctf_challenge_walkthrough

ctf6

./ctf_challenge_walkthrough

ctf7

./ctf_challenge_walkthrough

ctf8

./ctf_challenge_walkthrough

ctf9

./ctf_challenge_walkthrough

ctf10

./ctf_challenge_walkthrough

ctf11

./ctf_challenge_walkthrough

ctf12

./references

  1. Everything You Ever Wanted to Know about DLLs - CppCon
  2. Reversing DLLs - CactusCon
  3. LoadLibraryA - MSDN
  4. GetProcAddress - MSDN
  5. Thomas Roccia - Medium
  6. Passing arguments via rundll32.exe to function exported by DLL
  7. Exporing DLLs Through Creation and Reversing

./the_end.sh

meme