6.5 The exec() Family of System Calls: Replacing Process Memory
💡 Core Intuition
🍳 The Everyday Analogy: The Total Stage Role Replacement
Imagine a theater production where an actor steps onto the stage under an assigned character ID badge (the Process ID):
The Theater Role Replacement Analogy Pipeline
Mapping dramatic role transformation to virtual address space replacement
Current Actor on Stage
An actor recites lines from an initial script.
Complete Script & Costume Swap
The actor's script, costume, and dialogue are completely replaced.
New Play Begins (Same PID)
The new performance begins directly from line 1 of the new script.
fork()vsexec():fork()makes a clone of the actor.exec()transforms the actor into an entirely different person without changing their badge number (PID).
💻 Bridging to Computer Science
In UNIX systems, process creation is intentionally decoupled into two orthogonal operations:
fork(): Clones the current address space to establish a new child process.exec(): Overwrites the process's virtual memory with a completely new executable binary loaded from disk.
📚 Core Deep-Dive & Concepts
1. The exec() System Call: Core Mechanics & The Non-Return Rule
Definition: The
exec()family of functions replaces the current process image with a new process image. It loads an executable file (such as an ELF binary) into the process's virtual address space, overwriting its Text, Data, Heap, and Stack segments.
The Fundamental Non-Return Rule
The Point of No Return: A successful call to
exec()NEVER RETURNS. Because the code segment containing the original instructions is completely destroyed and replaced, the CPU jumps directly to the entry point (main()) of the new program.
#include <stdio.h>
#include <unistd.h>
int main() {
printf("Starting execution...\n");
// Overwrite process with /bin/ls
execl("/bin/ls", "ls", "-l", NULL);
// ================= DEAD CODE =================
// If execl succeeds, this line NEVER executes!
perror("execl failed");
return 1;
}
- If
execl()succeeds,"Starting execution..."prints, followed by directory listings fromls. The lineperror("execl failed")is never reached. exec()returns if and only if an error occurs (e.g. file not found, permission denied), returning-1.
2. What is Preserved Across exec()?
Although memory is wiped, the kernel preserves essential operating system context:
Preserved Across exec() | Reset / Replaced by exec() |
|---|---|
| Process ID (PID): Retains identical numerical PID | Text Segment: Replaced by new compiled code |
| Parent PID (PPID): Preserved unchanged | Data Segment: Replaced by new global variables |
| User & Group IDs: Real UID and GID are preserved | Heap & Stack: Completely wiped and reinitialized |
| Current Working Directory (CWD): Preserved | CPU Registers & PC: Reset to new binary entry point |
| Open File Descriptors: Preserved by default! | Signal Handlers: Reset to default actions |
The Close-On-Exec Flag (FD_CLOEXEC)
By default, file descriptors remain open across exec(). If a program opens a sensitive database socket or file and calls exec(), the new program inherits that descriptor!
To prevent security leaks, programs set the close-on-exec flag:
fcntl(fd, F_SETFD, FD_CLOEXEC);
With FD_CLOEXEC, the kernel automatically closes that specific file descriptor during the exec() transition.
3. The 6 Functions in the exec() Family Decoded
The underlying kernel system call is execve(). The standard C library (glibc) provides five wrapper functions to simplify invocation:
The 6 Variants in the exec() Family
Deconstructing function suffixes: list vs vector, PATH lookup, and environment
1. execl()
List of Arguments- Arguments passed as a comma-separated list of strings.
- Terminated by a mandatory NULL pointer.
- Requires absolute or relative file path.
2. execv()
Vector Array- Arguments passed as an array of string pointers (char *argv[]).
- Last array element must be NULL.
- Requires absolute or relative file path.
3. execlp()
List + PATH Search- Arguments passed as a list of strings.
- Automatically searches the system $PATH environment variable.
- No need to specify /bin/ls; just write 'ls'.
4. execvp()
Vector + PATH Search- Arguments passed as a vector array.
- Automatically searches $PATH for binary location.
- The standard workhorse used by UNIX command shells.
5. execle()
List + Custom Env- Arguments passed as a list of strings.
- Accepts custom environment variable array (envp).
- Overrides default system environment.
6. execve()
The Core Syscall- The true, fundamental kernel system call.
- Accepts vector array and custom environment vector.
- All other 5 functions wrap execve().
Mnemonic Rules
l(List): Arguments passed individually:arg0, arg1, ..., NULL.v(Vector): Arguments passed as an array:char *argv[].p(PATH): Searches system$PATHenvironment variable.e(Environment): Accepts explicit custom environment array:char *envp[].
4. The Canonical UNIX Shell Architecture: fork() + exec() + wait()
How does an interactive terminal (such as Bash or Zsh) execute user commands like cat file.txt?
The UNIX Shell Execution Lifecycle
Tracing how shells combine fork, descriptor redirection, exec, and wait
Read User Command
Shell invokes fork()
Parent Shell invokes waitpid()
Child invokes execvp('cat', argv)
cat completes and calls exit(0)
Shell reaps child and resumes
🏭 In The Real World: Production Case Study
Docker Container Entrypoints and The exec Trap
In containerized microservices (Docker / Kubernetes), shell scripts are frequently used as startup wrappers:
#!/bin/sh
# WRONG ENTRYPOINT PATTERN (Shell stays PID 1)
./setup_configs.sh
node server.js
- The Bug (Signals Trapped):
- When Docker starts,
/bin/shruns as PID 1. - Running
node server.jswithoutexeccauses/bin/shto fork Node as a child (PID 2). - When Kubernetes sends
SIGTERMto stop the container gracefully, the signal is sent strictly to PID 1 (/bin/sh). - Standard
/bin/shdoes not forward signals to its children! Node never receivesSIGTERM, never closes database connections, and is forcibly killed viaSIGKILLafter a 30-second timeout.
- When Docker starts,
- The Production Fix (Using
exec):#!/bin/sh
# CORRECT ENTRYPOINT PATTERN
./setup_configs.sh
exec node server.js # Replaces /bin/sh with Node! Node is now PID 1!- Using
execreplaces the shell process entirely. Node becomes PID 1, receivesSIGTERMdirectly, and shuts down gracefully in milliseconds.
- Using
🎯 Exam & Interview Pitfall Check
Question 1: Does the Process ID (PID) of a process change after successfully executing an exec() family function?
Answer:
No.
exec() does not create a new process; it merely replaces the virtual address space (code, data, stack, heap) of the existing calling process with a new binary image. The Process ID (PID) and Parent Process ID (PPID) remain completely identical.
Question 2: Why must a command-line shell (such as Bash) call fork() before calling exec() to run an external program like ls?
Answer:
If the shell called exec() directly without forking first:
- The shell's own code, data, and memory space would be completely overwritten by the
lsprogram. - Once
lsfinished executing, it would callexit(), causing the entire process to terminate. - The user's terminal session would crash and close immediately.
- By calling
fork()first, the shell preserves itself in the parent process while the child process safely executesexec().
- The Code After
exec()Trap: Remember that any code placed immediately afterexec()will never run ifexec()succeeds. It executes only ifexec()returns an error (-1). - The
system()vsexec()Confusion:system("ls")internally runsfork(), launches a subshell (/bin/sh -c "ls"), and callswait(). It is slow and vulnerable to shell injection attacks. Theexec()family replaces the process directly without spawning a subshell. - Forgetting the NULL Terminator in
execl(): When callingexecl()orexeclp(), you must terminate the argument list with a cast NULL pointer ((char *)NULL). Omitting NULL leads to memory read segmentation faults.