283 words
1 minute
BITSCTF 2026 - gcc (Ghost C Compiler) - Reverse Engineering Writeup

Category: Reverse Engineering

Flag: BITSCTF{n4n0m1t3s_4nd_s3lf_d3struct_0ur0b0r0s}

Challenge Description#

Given README.md and chall.zip. A supposedly “safe and fast” C compiler wrapper.

Analysis#

ghost_compiler is a 64-bit stripped PIE ELF with RELRO/Canary/NX/PIE. Running it without args produces gcc: fatal error: no input files - it forwards to system gcc.

Core logic from reversing:

  1. Opens itself (argv[0]), finds embedded 8-byte marker

  2. Computes FNV-1a-like 64-bit hash over whole file except 0x40-byte window

  3. Derives key: key = 0xcafebabe00000000 ^ hash

  4. Decrypts using rolling XOR with ROR64 key schedule

  5. Validates decrypted prefix is BITSCTF{

Solution#

Brute-force candidate offsets for 0x40-byte encrypted block:

# !/usr/bin/env python3
from pathlib import Path
BIN_PATH = "ghost_compiler"
TARGET_PREFIX = b"BITSCTF{"
FNV_OFFSET = 0xCBF29CE484222325
FNV_PRIME = 0x100000001B3
MASK64 = (1 << 64) - 1
def ror64(x: int, n: int = 1) -> int:
return ((x >> n) | ((x << (64 - n)) & MASK64)) & MASK64
def derive_key(blob: bytes, skip_off: int, skip_len: int = 0x40) -> int:
h = FNV_OFFSET
for i, bt in enumerate(blob):
if skip_off <= i < skip_off + skip_len:
continue
h ^= bt
h = (h * FNV_PRIME) & MASK64
return (0xCAFEBABE00000000 ^ h) & MASK64
def decrypt_window(blob: bytes, off: int, key: int, n: int = 0x40) -> bytes:
out = bytearray()
k = key
for i in range(n):
out.append(blob[off + i] ^ (k & 0xFF))
k = ror64(k, 1)
return bytes(out)
def main() -> None:
blob = Path(BIN_PATH).read_bytes()
for off in range(0, len(blob) - 0x40 + 1):
key = derive_key(blob, off, 0x40)
dec = decrypt_window(blob, off, key, 0x40)
if dec.startswith(TARGET_PREFIX) and b"}" in dec:
flag = dec.split(b"\x00", 1)[0].decode("utf-8", errors="ignore")
print(f"[+] offset = {off}")
print(f"[+] key = {hex(key)}")
print(f"[+] flag = {flag}")
return
print("[-] Flag window not found")
if __name__ == "__main__":
main()

Run:

python3 solve_ghost_compiler.py

Output:

[+] offset = 12320
[+] key = 0x5145dd89c16375d8
[+] flag = BITSCTF{n4n0m1t3s_4nd_s3lf_d3struct_0ur0b0r0s}
BITSCTF 2026 - gcc (Ghost C Compiler) - Reverse Engineering Writeup
https://fuwari.vercel.app/posts/45/bitsctf-2026-gcc-ghost-c-compiler-reverse-engineering-writeup/
Author
Light
Published at
2026-02-22
License
CC BY-NC-SA 4.0