We know that when you type a command like
ls foo the shell is actually telling
the kernel to run the program
/usr/bin/ls with the argument
foo. But we
still don’t really see what goes on behind the scenes when that program runs.
Conceptually, we know that a program is a list of instructions for the computer,
but so far the only example we’ve seen of that was the instructions for the
simple, theoretical Turing machine back in Part 1. In this chapter we’ll
look at some real program instructions.
At the lowest, most fundamental level we have so-called machine code. Machine code is the basic instructions understood by the CPU. The CPU reads instructions byte-by-byte as pure digital data. We are going to create a very simple program in machine code by creating a file containing such data.
First you need to understand that while the kernel could technically run any sort of program on the computer, it expects certain standard behavior from most programs. For example, it expects that when a program is done running, the program will send it a number indicating how things went. This number is called the program’s “exit code”. It doesn’t really matter what the number is: like command arguments, the meaning of the exit code can vary from program to program. Most programs use very simple exit codes: an exit code of 0 means everything went alright and an exit code of 1 means something went wrong while the program was running.
The program we’re going to make is very simple: it will send the kernel the exit
code “7”. Nothing more. To write this program in machine code, we need to be
able to write some specific bytes to a file. We will do this using a tool called
hexedit which lets us edit files by changing the hexadecimal values of each
First we need to create a file to store the program instructions. We will do
this using a command called
dd which lets us create a file with a certain
number of bytes. We will do all of the work in this chapter in the
# cd /root # dd if=/dev/zero of=sevn bs=1 count=91
This creates the 91 byte file
sevn by copying the first 91 bytes of the file
/dev/zero, which is a special file containing just zero bytes. Now we can edit
the bytes of this file:
# hexedit sevn
You will see a screen like this:
00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 00000030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 00000040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 00000050 00 00 00 00 00 00 00 00 00 00 00 ...........
hexedit interface is split up into three columns. The column on the left
simply shows us byte numbers in hexadecimal to keep track of where we are in the
the file. So the first byte of the first line is byte number 0x0, the first byte
of the second line is byte number 0x10, the first byte of the third line is
number 0x20, etc. The middle column shows the actual bytes of the file in
hexadecimal, which are all zero. In hexadecimal a byte stores values from 0x00
to 0xFF, so each byte is represented by two characters. Each line has 16 bytes
and the spacing in between the bytes helps us visualize them in groups of 4. The
right column is a duplicate of the middle column but it shows the bytes of the
file represented as ASCII characters rather than hexadecimal values. Since
only the byte values 0x20–0x7E correspond to ASCII characters, all other
byte values are displayed as a
. in this column.
Start typing on the first line so that the first four bytes look like this:
7F 45 4C 46
You can use backspace to undo and the arrow keys to move the cursor as you might expect. Even though everything appears in uppercase, you don’t need to hold Shift when you type.
Notice that the corresponding bytes in the third column change because
46 are the hexadecimal values for “ELF” in ASCII.
Keep changing bytes until your columns look like this:
00000000 7F 45 4C 46 01 01 01 00 00 00 00 00 00 00 00 00 .ELF............ 00000010 02 00 03 00 01 00 00 00 54 80 04 08 34 00 00 00 ........T...4... 00000020 00 00 00 00 00 00 00 00 34 00 20 00 01 00 00 00 ........4. ..... 00000030 00 00 00 00 01 00 00 00 00 00 00 00 00 80 04 08 ................ 00000040 00 80 04 08 5B 00 00 00 5B 00 00 00 05 00 00 00 ....[...[....... 00000050 00 10 00 00 B3 07 31 C0 40 CD 80 ......1.@..
Once you’ve finished, press Ctrl+X to save the file and
then press Y to confirm and exit
I realize that it was really tedious to type in all of those bytes and you might
have made a typo or misread something. So let’s check our work. For this we can
use the command
md5sum, which prints a checksum of the bytes in our
# md5sum sevn 21deab879a3943cb640e7bfc9b702ca2 sevn
If the output you get doesn’t match that, you mistyped something. Open the file
hexedit and carefully look things over and edit until you get the
checksum to match. Once you have a match, we’re almost ready to go. Next we have
to tell the file system that this file is actually a program, not just a bunch
# chmod +x sevn
This changes the mode of the file to executable. “Executable” is yet another synonym for program. Now we can run the program:
Remember that when we type the name of a program as a shell command, the shell
looks for the program in certain directories like
/usr/bin. Since we didn’t
sevn program in one of these directories, we have to specify
exactly where the program is by including the
If everything went according to plan, your shell will simply print another prompt, meaning it finished running the program and is waiting for another command. To know that our program worked as we intended, we need to see what its exit code was. When you run a command, the shell stores its exit code for later use. You can tell the shell to show you that exit code with the following command:
# echo $? 7
Tada! That 7 means our program worked. Now that we have a working program, let’s take a look at what it’s doing. Just like most other files, programs store their data (their instructions) in a format. So not all of the bytes we just wrote are actual program instructions; many of them describe things like what kind of program this is, what sort of computers it works on, etc. We don’t really care about that stuff so we can just focus on the last seven bytes, which are the actual instructions:
B3 07 31 C0 40 CD 80
The CPU basically performs these instructions from left to right byte-by-byte. Some instructions are kind of like shell commands and have arguments, so they include multiple bytes. We can visually split up the instructions like so:
B3 07 31 C0 40 CD 80
So there are four instructions. Let’s talk about what they do.
- Recall from the hardware chapter that the CPU’s memory is its
registers: numbers with a set size that we can perform calculations
on. The first instruction
B3 07stores the byte value
07in a 8-bit register called BL. This is the exit code that we’re going to send to the kernel.
- The next instruction
31 C0sets a different register to 0. The
C0is what chooses which register gets zeroed. In this case it’s a 32-bit register named EAX.
- Next we get the single instruction
40. This simply adds 1 to the value in the EAX register. Since we previously set this register to 0, the EAX register now has the value 1.
- Finally we get the
CD 80instruction. The machine code
CDis called an “interrupt”. It means we are interrupting our program to hand control over to some other part of the computer. The second byte specifies what that “something else” is.
80specifies that this is an interrupt for a “system call”, which is another way of saying that we want to hand control over to the kernel. So when the processor sees the interrupt instruction it transfers control to the kernel, at which point the kernel looks at the values that we have placed in the CPU’s registers. These values work kind of like a shell command: the value in EAX tells the kernel what sort of action to perform and the value in BL is an argument for that action. A value of 1 in EAX is an “exit” action, and the value in BL says that the exit code should be 7.
So to recap, we store the exit code in BL, set EAX to 1 (which is the code for an exit), and give control to the kernel to perform the exit with the specified exit code.
So that was more than a little confusing and difficult. But this is hopefully the last time you’ll ever need to touch machine code; only crazy people actually write machine code by hand like we just did. I just wanted to demonstrate that you can type out instructions as bytes and make the computer do things.
sevn program to have a different exit code.
In practice, the closest that programmers get to writing machine code is a
slightly more readable style called “assembly”. Writing machine code is
difficult because you have to keep in mind all of the technical details of the
computer and the way its registers and instructions interact, but it also has an
additional layer of difficulty because every instruction is a bunch of
meaningless bytes. How do you remember that
31 C0 means “set EAX to 0”? That’s
just not easy to understand.
Assembly improves this one aspect of machine code by letting us write (slightly)
more human-readable instructions. For example,
40 in machine code becomes
incl %eax in assembly.
incl is short for increment long and the
eax is the name of the register we want to increment. It’s
still not super understandable, and you still need to have a good understanding
of the inner workings of the CPU, but it’s definitely easier to work with. Let’s
rewrite our program in assembly.
Unlike machine code which is pure digital data (usually viewed as hexadecimal),
assembly is entirely human-readable ASCII text. So instead of
hexedit we will
use a program called
nano, which is a text editor. Type
# nano sevn.s
This opens up a new text file called
sevn.s in a simple text editing
interface, somewhat similar to
hexedit. You can type to add text and use
backspace and the arrow keys as you would expect.
Type out the following text:
As in previous chapters the text is colored simply to make it easier to read.
Then press Ctrl+O and Enter to save the text in
Now let’s look at what this assembly says. Like the machine code, we can ignore the beginning because it’s just part of the assembly format. The last four lines are the actual instructions and they correspond to the four instructions we saw earlier:
B3 07has become
movb $7,%bl, which means “move the byte 7 into register BL”. The
7is a literal, numerical value. The
blis the name of a register. In other words, set BL to 7. Assembly language may not be “human readable” per se, but at least we can see the names of the registers in the instructions now.
31 C0has become
xorl %eax,%eax. Oddly, assembly doesn’t have an instruction for setting a register to 0. But we can achieve the same effect by XORing a register with itself. So
xorl %eax,%eaxsays “XOR EAX with the long value in EAX”. In assembly language, “long” means 32-bits. So this instruction sets all 32 bits of EAX to 0.
- As we mentioned earlier,
incl %eaxwhich means “increment the 32-bit register EAX by 1”. So now EAX contains a 1.
- And finally
int $0x80which is of course short for “interrupt code 0x80” i.e. a system call. So we see that even though this program is written differently it performs the exact same steps as our machine code program.
Once you’re done perusing your assembly code, press Ctrl+X
nano. If you want to check your work again:
# md5sum sevn.s bedeba8cdfade20ece6a3e37da749435 sevn.s
Getting your assembly to run is a little more involved than it was for your
machine code. You see, the CPU only understands machine code, so we need to
translate our assembly back into machine code before we can run it. This is a
two stage process. First we turn our assembly into an “object file” using the
as command (short for assemble):
# as -o sevn.o sevn.s
-o option tells
as to save the object file with the file name
Next we use the
ld command (short for link editor) to “link” the
object file and create the executable:
# ld -s -o sevn2 sevn.o
-o option tells
ld to save the executable file with the file name
sevn2 (so you can compare it with the machine code version of the program).
-s is optional, but it makes the resulting machine code a little easier to
ld is even nice enough to mark the file as executable for us, so we
don’t have to
chmod it like we did before. So now:
# ./sevn2 # echo $? 7
The reason why this is a two stage process is actually to save us time when we make really big, complicated programs. Suppose we’re working on a team to make a program and we want different team members to work on different parts of the program. We could all work on our assembly code separately, then put our code together in one big assembly file, and create the executable. But the big downside of this is that if later on we want to change even one instruction, we have to re-assemble the whole program!
What we can do instead is write our assembly separately and translate our assembly into machine code separately. The resulting pieces of machine code aren’t programs themselves, so instead we call them “object files”. Then we use another program, the link editor, to “link” our object files together into an actual program. That way if we want to change an instruction later, we only have to re-assemble that one piece and then re-link the pieces rather than re-assemble everything.
hexedit. There are a lot more bytes than before but again
most of it is formatting. That’s because
ld included more optional
parts of the executable format that we left out when we wrote the machine code
by hand. However the same familiar machine code instructions should be nestled
in here somewhere. See if you can find them.
While some programmers still write assembly today, it has many drawbacks. First of all the instructions are processor-specific. So a program written in assembly will only work with certain CPUs. If you want to run your program on other computers with different CPUs, you need to rewrite it in instructions that that computer understands. Second, while it is more human readable than machine code, assembly is still not very understandable and thus it can be notoriously difficult to write it without making mistakes. Third, assembly can be very tedious to write. Seemingly simple tasks can often take a surprising number of instructions to perform. In the next chapter we’ll learn about more advanced programming techniques that solve these problems.
Try changing a single byte of the
sevnfile so that this happens
# ./sevn -bash: ./sevn: cannot execute binary file: Exec format error
What do you think that means?
Try changing a single instruction of
sevn.sso that when you reassemble and run it this happens
# ./sevn2 Segmentation fault (core dumped)
What do you think that means?