core (misc)

Extracting the archive gives 2 files:

  • chall
  • core
❯ file chall
chall: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/, BuildID[sha1]=b94dac66618a0a01fb827ad76fbd6ba5b8811c67, for GNU/Linux 3.2.0, not stripped

❯ file core
core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from './chall DEBUG', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: './chall', platform: 'x86_64'

core is a core dump generated from running ./chall with an argument DEBUG.

Static Analysis

First, it is useful to know what chall does. Opening it up in Ghidra gives the following code.

undefined8 main(int argc,long param_2)

  __stream = fopen("./pass","r");
  if (__stream == (FILE *)0x0) {
    fwrite("Missing password file!\n",1,0x17,stderr);
  else {
    // [1] Read the password, get the flag from stdin, and encrypt the flag with the password
    printf("Enter flag: ");
    printf("Encrypted flag: ");
    for (i = 0; i < 40; i++) {
      flag[i] = flag[i] ^ password[i % 16];
      if (i != 39) {
        flag[i + 1] = flag[i] ^ flag[i + 1];

    // [2] Clear the password from the stack

    // [3] Generate an abort signal.
    // This creates a core dump if the program is executed in GDB.
    if (argc == 2) {
      result = strcmp(argv[1],"DEBUG");
      if (result == 0) {


The program is quite short.

  1. First, it read the password from a file, and asks the user to provide the flag as input. Then it encrypts the flag with the password with a simple xor loop.
  2. Then, it clears the password from the stack using memset.
  3. Finally, it raises signal 6 (SIGABRT). If a program aborts while running in GDB, a core dump will be generated.

The core dump is given to us by the challenge. So, we can inspect the core dump to find the flag.

Inspecting the Core Dump

Before continuing, here is a brief overview of what a core dump. When a program crashes, a core dump is generated, and it contains the whole program state, i.e. register value and memory contents. This is useful for a developer to find out what caused the program to crash.

To use a core dump, we can load it with GDB, by running the command gdb chall core.

A useful thing to do first is to check where we are in the program.

gef➤  print/x $rip
$1 = 0x7f2d17452ce1

gef➤  info registers
rip            0x7f2d17452ce1      0x7f2d17452ce1 <setjmp+1>

It seems like the program stopped when it is in a setjmp function. For better understanding, we can run vmmap (or info proc mappings) to see the memory map. (vmmap is only available in GDB extensions like GEF, peda, pwndbg)

gef➤  info proc mapping
Mapped address spaces:

          Start Addr           End Addr       Size     Offset objfile
      0x5566a996c000     0x5566a996d000     0x1000        0x0 /core/chall
      0x5566a996d000     0x5566a996e000     0x1000     0x1000 /core/chall
      0x5566a996e000     0x5566a996f000     0x1000     0x2000 /core/chall
      0x5566a996f000     0x5566a9970000     0x1000     0x2000 /core/chall
      0x5566a9970000     0x5566a9971000     0x1000     0x3000 /core/chall
      0x7f2d1741a000     0x7f2d1743c000    0x22000        0x0 /lib/x86_64-linux-gnu/
      0x7f2d1743c000     0x7f2d17596000   0x15a000    0x22000 /lib/x86_64-linux-gnu/
      0x7f2d17596000     0x7f2d175e5000    0x4f000   0x17c000 /lib/x86_64-linux-gnu/
      0x7f2d175e5000     0x7f2d175e9000     0x4000   0x1ca000 /lib/x86_64-linux-gnu/
      0x7f2d175e9000     0x7f2d175eb000     0x2000   0x1ce000 /lib/x86_64-linux-gnu/
      0x7f2d175f3000     0x7f2d175f4000     0x1000        0x0 /lib/x86_64-linux-gnu/
      0x7f2d175f4000     0x7f2d17614000    0x20000     0x1000 /lib/x86_64-linux-gnu/
      0x7f2d17614000     0x7f2d1761c000     0x8000    0x21000 /lib/x86_64-linux-gnu/
      0x7f2d1761d000     0x7f2d1761e000     0x1000    0x29000 /lib/x86_64-linux-gnu/
      0x7f2d1761e000     0x7f2d1761f000     0x1000    0x2a000 /lib/x86_64-linux-gnu/
gef➤  vmmap
[ Legend:  Code | Heap | Stack ]
Start              End                Offset             Perm Path
0x005566a996d000 0x005566a996d01b 0x00000000001000 r-x .init
0x005566a996d020 0x005566a996d0e0 0x00000000001020 r-x .plt
0x005566a996d0e0 0x005566a996d0f0 0x000000000010e0 r-x
0x005566a996d0f0 0x005566a996d1a0 0x000000000010f0 r-x .plt.sec
0x005566a996d1a0 0x005566a996d4b5 0x000000000011a0 r-x .text
0x005566a996d4b8 0x005566a996d4c5 0x000000000014b8 r-x .fini

Both outputs above give the information in different formats, so we can’t just say one is better than the other. info proc mappings tells us that contents from chall are in the memory region with address starting with 0x5566, and libc has address that starts from 0x7f2d.

vmmap output is useful as it gets into finer details. It shows that the .text section is in the 0x005566a996d1a0-0x005566a996d4b5 memory range.

Remember that the value of rip is 0x7f2d17452ce1, so it is some code inside libc. This makes sense, as raise is a libc function. The program aborts inside libc, as expected.

Finding the stack frame

Now, the important step is to find the stack frame of the main function, so that we can get the encrypted flag.

gef➤  print/x $rbp
$3 = 0x7ffc1d759a90

According to the disassembly, the encrypted flag is stored in the stack, at rbp-0x30, and the password is stored at rbp-0x40. We can use the telescope command (only in GEF/peda/pwndbg) to check the contents starting from rbp-0x40. (I have annotated the memory contents below with their corresponding purposes.)

gef➤  tele $rbp-0x40
# password
0x007ffc1d759a50│+0x0000: 0x00000000000000
0x007ffc1d759a58│+0x0008: 0x00000000000000

# flag
0x007ffc1d759a60│+0x0010: 0x9364d7bb7e9a9eb9
0x007ffc1d759a68│+0x0018: 0x5a48f0f298bfa23a
0x007ffc1d759a70│+0x0020: 0xd829cef5738cd3a0
0x007ffc1d759a78│+0x0028: 0x4b52b0bb85f2e723
0x007ffc1d759a80│+0x0030: 0xd4419cb834cdc5bd

# canary
0x007ffc1d759a88│+0x0038: 0xf6df0028e1a91800

# return address
0x007ffc1d759a90│+0x0040: 0x005566a996d440

Everything above is as expected.

  1. rbp-0x40 contains 16 bytes of password, which was cleared using memset, so it only contains null bytes.
  2. Then, the 40 bytes of the flag (encrypted).
  3. According to Ghidra, after the flag is the stack cookie. This value looks like a stack cookie too, because its least significant byte is 00. If I’m not wrong, the stack cookie’s LSB is a null byte, to prevent it from being leaked, if a string is stored before it, and the string is not null-terminated. Think about it.
  4. Lastly, there is a value that starts with 0x5566. We have seen earlier that this maps to the .text section of the chall executable.

Recovery of the flag

Now, we can use GDB to give me the encrypted flag contents for scripting. The print-format command in GEF is useful to get the contents in Python list format.

gef➤  x/40bx $rbp-0x30
0x7ffc1d759a60:	0xb9	0x9e	0x9a	0x7e	0xbb	0xd7	0x64	0x93
0x7ffc1d759a68:	0x3a	0xa2	0xbf	0x98	0xf2	0xf0	0x48	0x5a
0x7ffc1d759a70:	0xa0	0xd3	0x8c	0x73	0xf5	0xce	0x29	0xd8
0x7ffc1d759a78:	0x23	0xe7	0xf2	0x85	0xbb	0xb0	0x52	0x4b
0x7ffc1d759a80:	0xbd	0xc5	0xcd	0x34	0xb8	0x9c	0x41	0xd4
gef➤  print-format --lang py --bitlen 8 -l 40 $rbp-0x30
buf = [0xb9, 0x9e, 0x9a, 0x7e, 0xbb, 0xd7, 0x64, 0x93, 0x3a, 0xa2, 0xbf, 0x98, 0xf2, 0xf0, 0x48, 0x5a, 0xa0, 0xd3, 0x8c, 0x73, 0xf5, 0xce, 0x29, 0xd8, 0x23, 0xe7, 0xf2, 0x85, 0xbb, 0xb0, 0x52, 0x4b, 0xbd, 0xc5, 0xcd, 0x34, 0xb8, 0x9c, 0x41, 0xd4]

Unfortunately, the password was already cleared from the stack, and all we get are null bytes. Let’s see what we can do from here.

By reversing the xor operations, and knowing that the flag starts with wgmy{, it is possible to recover part of the password. It should be simple enough so you can do it yourself :p. I will skip describing how to do so.

We can deduce that the password starts with '\xce@i\x9d'. But this is not enough. The password has a total of 16 bytes.

Here, I made a guess that maybe although the password is no longer in the stack, it may be in other sections, like the heap. I remember that functions like scanf may store the values in the heap during the process. Maybe fread (used by the program to read the password) does the same.

To look for the password in the heap, I can use the grep command by GEF. The syntax is as follows:

gef➤  help search-pattern
SearchPatternCommand: search a pattern in memory. If given an hex value (starting with 0x)
the command will also try to look for upwards cross-references to this address.
Syntax: search-pattern PATTERN [little|big] [section]

search-pattern AAAAAAAA
search-pattern 0x555555554000 little stack
search-pattern AAAA 0x600000-0x601000

I first tried to search by just running grep '\\xce@i\\x9d' (need the double backslash because of some parsing issues, I will make a PR to fix this soon). In a normal debugging session, this will search through all the memory regions of the process memory. However, it doesn’t work with this core dump. GEF is not commonly used with core dumps so this command doesnt work well here.

So, I have to manually provide the memory ranges. To do so, I used vmmap to find out the memory regions, in particular, I am intersted in the heap regions.

gef➤  vmmap
[ Legend:  Code | Heap | Stack ]
Start              End                Offset             Perm Path
0x005566a996c000 0x005566a996d000 0x00000000002000 r-- load1
0x005566a996d000 0x005566a996e000 0x00000000003000 r-x load2
0x005566a996e000 0x005566a996f000 0x00000000003000 r-- load3
0x005566a996f000 0x005566a9970000 0x00000000003000 r-- load4
0x005566a9970000 0x005566a9971000 0x00000000004000 --- load5
0x005566aaf07000 0x005566aaf28000 0x00000000005000 --- load6
0x007f2d1741a000 0x007f2d1741b000 0x00000000026000 r-- load7a
0x007f2d1741b000 0x007f2d1743c000 0x00000000027000 r-- load7b

I know that regions starting with 0x7f2d is libc, so I’m not interested in those regions. And regions from 0x005566a996c000 to 0x005566a9971000 belongs to chall, according to the output of info proc mappings.

So, I provide the memory range of load6 to grep.

gef➤  grep '\\xce@i\\x9d' little 0x005566aaf07000-0x005566aaf28000
[+] Searching '\xce@i\x9d' in 0x005566aaf07000-0x005566aaf28000
[+] In 'load6'(0x5566aaf07000-0x5566aaf28000), permission=---
  0x5566aaf07480 - 0x5566aaf0748a  →   "\xce@i\x9d[...]"

It found something at 0x5566aaf07480!

gef➤  x/16bx 0x5566aaf07480
0x5566aaf07480:	0xce	0x40	0x69	0x9d	0xbe	0x59	0xd7	0x95
0x5566aaf07488:	0x98	0xa1	0x2d	0x14	0x0f	0x33	0x81	0x20

This should be the password!

Using this password, the encrypted flag can be decrypted to get the flag.