Walking through a basic buffer overflow

I’m learning about buffer overflows in preparation for the PWK course and OSCP exam. I haven’t touched assembly language in more than 20 years, and the protections present in modern OSes just didn’t exist back when I first learned all this (let alone the fact that I was working on 680x0 and 650x assembly at the time).

After trying desperately to make the mental leap from the 1990s to modern operating systems and following along in Aleph One’s Smashing The Stack For Fun And Profit, I decided to use the material from Chapter 16 of Georgia Weidman’s excellent Penetration Testing: A Hands-On Introduction to Hacking, and write this to ensure I understand exactly what I’m doing and what’s going on in that chapter.

To start with, we need a program with some very basic functionality. She uses the following code in her examples:

#include <string.h>
#include <stdio.h>

void overflowed() {
  printf("%s\n", "Execution hijacked");

void function1(char *str) {
  char buffer[5];
  strcpy(buffer, str);

void main(int argc, char *argv[]) {
  printf("%s\n", "Executed normally");

This is some very basic (and from a security standpoint, quite dangerous, as we’ll soon learn) C code. There is the main() function, which simply takes an argument from the command line and stores it in a variable by passing it to the function function1(). It doesn’t do any bounds checking, so it’s possible to overflow the buffer area assigned to that variable.

There’s also a function named overflowed() which never gets called during the execution of the program. Our goal is to overflow the buffer, and cause the code to call this function instead of the printf() right after the call to function1() in main().

The stack

To better understand what we need to do, we need to understand the stack. The stack is a fixed-size area of memory assigned when the program is run, designed to store things associated with functions like argument values, local variables, and so forth. There are other areas, such as the heap, the data segment, and the text segment, but let’s focus on the stack.

The stack is exactly what it sounds like: It’s a stack of things. You put one thing on the stack, and the next thing you put on the stack goes on top of that thing, and so on. So when you take something off the stack, it’s the thing at the top of the stack – the last thing put on. This is called last-in, first-out (LIFO).

The stack is ordered from a maximum memory address to a minimum memory address. This means the first thing that goes on the stack is at the highest memory address in the stack, and the next thing, since it goes on top of that, goes to the next highest memory address in the stack, and so on.

So, for example, if we had memory addresses indexed by numbers 1 through 10, the first thing to go on the stack would go into address 10 – the highest memory address associated with the stack. The next thing on the stack would go on address 9, and the next at address 8, and so forth.

An example stack
      8 Third thing on stack
      9 Second thing on stack
     10 First thing on stack

Stack frames

There is also the concept of stack frames. Stack frames are chunks of information associated with a function, put onto the stack all at once. They’re put onto the stack (and removed from the stack) as needed. So, in the case of the example code above, the first stack frame put on the stack will be the stack frame for the main() function. Because there’s nothing on the stack initially when the program is run, the main() stack frame will go onto the stack starting at address 10.

The assembly language commands used to put things onto the stack and take them back off again are push and pop. Push puts something onto the stack, and pop takes it off again. This will be relevant when we begin looking at disassembly of code.


Stack frames are tracked using two pointers: ESP and EBP. ESP (the extended stack pointer) points to the top of the stack frame, and EBP (the extended base pointer) points to the bottom of the stack frame. Let’s take a look at an example: Assume main()’s stack frame is 3 things long (we’re using “things” to avoid getting into a discussion of word size, which we’ll mention later; right now it’ll just confuse things). If the main() stack frame is 3 things long, and we’re using our addressing scheme above, the stack would look like this:

An example stack with a stack frame
ESP   8 ,--------------------.
      9 | main() stack frame |
EBP  10 `--------------------'

The main() stack frame occupies addresses 8, 9, and 10 of the stack. The two pointers we just mentioned, ESP and EBP, point to the top and the bottom of the stack frame. In this case, ESP would contain the address 8, which is the top of the main() stack frame, and EBP would contain the address 10, which is the bottom of the main() stack frame.

Remember, the stack is used to store function-related things like local variables, function arguments and such. It doesn’t actually store the program’s instructions. Those are all in the text segment I mentioned earlier. The stack is sort of like scratch paper used to keep track of things necessary for program execution.

Return addresses and the instruction pointer (EIP)

When a function is called, its stack frame gets put onto the stack. When we started the program, main() was called automatically because that’s how C (and several other languages) work. But if you look at the source code above, you’ll see that main() makes a call to function1(). But it does so before main() is finished, so now we need to do a few things:

  • We need to keep the main() stack frame on the stack, because we’re not done with it
  • We need to put function1()’s stack frame on the stack, because we’re switching to it
  • We need to change ESP and EBP to point to the top and bottom of the current stack frame (in this case, the function1() stack frame)
  • We need some way of preserving the EBP from the main() stack frame
  • We need some way of knowing where to go back once we’re done with function1()

In main(), there’s a printf() statement after the call to function1(), so we’ll need some way of remembering that we’re supposed to do that right after we finish executing function1().

We do that by putting a return address on the stack, pointing to the location in memory where the instructions that represent the printf() statement begin.

So, continuing with our addressing scheme above, and assuming the function1() stack frame is also three things long, we’d have:

An example stack with two stack frames and a return address
ESP   4 ,-------------------------.
      5 | function1() stack frame |
EBP   6 `-------------------------'
      7 return address
      8 ,--------------------.
      9 | main() stack frame |
     10 `--------------------'

We preserve the EBP from the main() stack frame by storing it in the last address for the function1() stack frame. This means that EBP will now point to address 6 – the bottom of the function1() stack frame, but the value stored in address 6 will be address 10, which is where EBP needs to point when we finish with function1().

You may be wondering, “What about preserving main()’s ESP?” Well, we don’t have to. When we’re done with function1(), function1()’s stack frame will be removed from the stack. Which will include retrieving the value stored at function1()’s EBP, and setting EBP to that value – in this case, 10. Then all that’s left on the stack is main()’s stack frame and the return address telling the computer which instruction in main() to execute now that we’ve finished the call to function1(). The current instruction being executed is tracked by another pointer called EIP (the extended instruction pointer). Once we transfer the return address out of address 7 and store it in the EIP, we don’t need that anymore either, so it comes off the stack as well. And since ESP always points to the top of the stack, by the time we’re ready to continue executing code in main(), the only thing left in the stack is main()’s stack frame, and ESP will already be pointing to address 8, the top of main()’s stack frame, because we’ve removed everything else that was on the stack.

That bit about the return address is crucial to understanding what we’re about to attempt: the return address is stored on the stack, and gets transferred to the instruction pointer EIP when we leave function1() to return to main(). And that address is how we know where to go. It’s supposed to point to the next instruction in main(), but it doesn’t have to. It could point anywhere. And there’s the magic: Our goal is to somehow control what that return address is, so we can trick the computer into executing what we want it to, rather than what the program thinks it’s supposed to. As an added benefit, if we can get control of the EIP, whatever we get the computer to execute will run with whatever privileges the original program had. So if this program was running with root, admin, or system level privileges, the code we trick it into executing will have those same privileges. Not to give the ending away, but imagine if we could somehow get the EIP to point to a shell. We’d have full control of a shell running with whatever privileges the original program has.

Let’s do this

To see this in action, we’ll need to compile the source code at the beginning of this article. I’m going to assume you’re doing this on a modern Linux OS (where “modern” in this case means somewhere around 2015-2016). These days, operating systems have several tricks up their sleeves to prevent us from doing what we’re about to do…at least, from doing it so easily. Since this is supposed to be demonstrating the basics, not how to bypass these protections, we’re going to disable them for the time being.

On a modern Linux, that means doing the following:

sudo echo 0 > /proc/sys/kernel/randomize_va_space

This will disable Address Space Layout Randomization (ASLR) while we’re doing what we need to do here. To re-enable it once we’re done working on buffer overflows, execute:

sudo echo 2 > /proc/sys/kernel/randomize_va_space

We also need to disable data execution prevention (DEP). We’ll do that when we compile the program. In fact, let’s do that now. Save the source code at the beginning of the article as overflowtest.c and issue the following command:

gcc -g -fno-stack-protector -z execstack -m32 -o overflowtest overflowtest.c

In brief, this command tells gcc to compile the code as a 32-bit binary (-m32), add extra debugging information (-g) which we’ll use in the gdb debugger, disable stack protection (-fno-stack-protector), and allow executable code on the stack (-z execstack), then save the binary with the name overflowtest.

(…to be continued!)

OSCPRed Team