Lost in a space

In this part we dig into the hardware mechanics that power x86 memory management. We explore the evolution from Real to Protected Mode and dissect the translation process from Logical to Linear addresses. You’ll learn how the Global Descriptor Table (GDT), Segment Selectors, and the Flat Memory Model function under the hood. This post builds the essential hardware foundation required to understand the complex world of memory internals

Previously on memory primitives

In the previous part of this series, we talked about some basics of memory and CPU such as how CPUs are designed, how memory is represented, and how memory is managed. In this part, we’ll take a closer look at memory-related topics and how the operating system makes use of physical memory. Primarily we will be discussing the models, modes and addressing of the intel architecture as well as discuss some concepts regarding memory management to help us later understand other mechanisms use to both optimize and protect the memory.

1. Memory models & Addresses

Modern operating systems have a variety of mechanisms and designs for managing and operating memory. To gain a clear understanding of how memory operates, it is necessary to comprehend the different existing models, modes, and addressing schemes.

Key terms at a glance

  • Memory address
    • A numeric “coordinate” that points to a specific location in memory. Different forms exist:
      • Logical: segment selector + offset
      • Virtual (linear): index within a process’s virtual space
      • Physical: location in actual RAM
  • Address space
    • The full range of addresses a program can use, the range is based on what the CPU and OS are supporting, this could be 32-bit or 64-bit, each process “virtually” has the same memory range starting at 0x0 and ending 0xFFFFFFFFFFFFFFFF ( 0xFFFFFFFF on 32-bit). Each process gets its own virtual address space, but only the needed regions are mapped to physical memory.
  • Virtual memory
    • An abstraction layer that lets software use a clean, isolated view of memory, while the OS maps parts of that view to physical RAM or disk as needed for protection and efficiency.

1.1 Memory Addresses

In modern computer architectures, software and hardware use a memory address to refer to location in memory. This is similar to the coordinate system, where the coordinates point to a specific location. In the same way, a memory address points to a specific memory location; Several types of memory addresses exist, yet they all fulfill the same purpose; The key difference between these types is their representation. The operating system interprets each representation differently to acquire the actual address. The following are some different ways to represent a memory address:

  • Logical address — It consists of two parts, segment selector & offset, an offset is a distance from the segment’s start to the actual address in question. This type of address is used in the segmented memory model as will be explained later.
  • Virtual address (Linear) — is a linear 32/64-bit unsigned integer that can be used to address the whole address space or “virtually” the whole ram depending on the size of the address space.
  • Physical address — is used to address memory locations in physical memory (RAM). They correspond to the electrical signals along with the address pins of the microprocessor to the memory bus.

Address translation through the MMU

A logical address is translated into a linear address through a hardware circuit called a segmentation unit, then the linear address is translated into a physical address through a hardware circuit called paging unit. In case paging is not enabled, the linear address is treated as the physical address. Moreover, the translation process will be discussed later.

Figure 1 : Logical to physical

1.2 Models

Memory models are used as paradigms through which processors address and access memory; there are two common models:

  • Segmented memory model — it is used to break down the memory into predefined blocks known as segments. The segmented memory model is only available for the purpose of backward compatibility and is not often used.
  • Flat memory model — Unlike the segmented model, the flat memory model is a contiguous paradigm in which all the address space is treated as one block; this model is the most common in modern computers.

1.2.1 Segmented memory model

Before we look at the hardware, let’s look at the philosophy. In the early days of computing, running a program was like giving a toddler a crayon and a blank wall—they could draw anywhere.

Segmentation was introduced to bring order to this chaos. It divides memory into independent, arbitrary blocks called Segments. instead of seeing memory as one giant pool, the CPU sees it as a collection of distinct regions: a region for code, a region for data, and a region for the stack.

Each region has strictly defined boundaries: a starting location known as the Base Address, and a maximum size known as the Limit. To find the last addressable part of the segment, you simply need to add the limit to the base address as depicted in the following figure :

Figure 2 : Memory segment

The base and limit are stored in a specific 8-byte long data structure called a Segment Descriptor, these two fields are used to find the beginning and the end of the segment in question.

Here is an example from the linux kernel:

/* 8-byte segment descriptor */
// https://elixir.bootlin.com/linux/v4.9/source/arch/x86/include/asm/desc_defs.h#L22
struct desc_struct {
...
            u16 limit0;
            u16 base0;
            unsigned base1: 8, type: 4, s: 1, dpl: 2, p: 1;
            unsigned limit: 4, avl: 1, l: 1, d: 1, g: 1, base2: 8;
...
} __attribute__((packed));

Because the OS could define multiple segment descriptors, it needs a place to store them—that’s where the Descriptor Table comes into play.

Figure 3 : Selector table

Descriptor tables —Think of the descriptor tables as an array of segment descriptors, the index from the segment selector is used to determine which segment descriptor will be loaded and used, there is mainly two tables :

  • Local descriptor table (LDT) — A table that contains process segment descriptors; each descriptor is 8 bytes long, and the base of the table is stored in the LDTR register.
  • Global descriptor table (GDT) — A table that contains system segment descriptors; each descriptor is 8 bytes long, and the base of the table is stored in the GDTR register.

Now to retrieve these descriptors, the system needs to know the index and in which table, this is where we introduce the Segment selector — A 16-bit identifier index that points to a segment descriptor entry in the segment table, A segment selector consists of three parts:

Figure 4 : Segment selector
  • Index — 13-bit value used to locate the segment descriptor in the segment table.
  • Table Indicator (TI) — 1-bit value specifying the segment register to be used (GDTR / LDTR).
    • Value of 0 = GDTR , 1 = LDTR
  • Requestor privilege level (RPL) — 2-bit value specifying the privilege of the selector.

Segment & Table registers in x86

For the processor to locate the correct descriptor it uses the segment selector, these selectors must be stored somewhere the CPU could access promptly; The processor provides six segment registers and two table registers; each segment register holds a selector, and each table register holds table location. The segment registers classification is as follows:

  • CS - Code segment
  • DS - Data segment
  • SS - Stack segment
  • ES - Extra segment
  • FS/GS - General purpose segments

Additionally there is two registers that point to descriptor tables (LDT/GDT)

  • LDTR — Special register contains an address pointing to the local descriptor table (32 bits in protected mode; 64 bits in IA-32e mode) and 16-bit table limit.
  • GDTR — Special register that contains an address pointing to the base address of the global descriptor table (32 bits in protected mode; 64 bits in IA-32e mode), table limit.
Figure 5 : The hierarchy

How it works

Ok so now let’s piece all this together to understand how a linear address is resolved, of course this is a simplified diagram

Figure 6 : Linear to physical

The Segmentation Unit: From Logical to Linear

To translate a Logical Address (which is just a Segment Selector : Offset) into a Linear Address, the processor’s Segmentation Unit follows these steps:

  1. Identify the Selector: The segmentation unit reads the top 16 bits of the address representing the Segment Selector.
  2. Choose the Table: It checks the Table Indicator (TI) bit in the selector. If the bit is 0, it uses the Global Descriptor Table (GDT); if 1, it uses the Local Descriptor Table (LDT).
  3. Find the Descriptor: It multiplies the selector’s Index by 8 (each descriptor is 8 bytes) and adds the result to the table’s base address (stored in the GDTR register). This locates the Segment Descriptor.
  4. Extract the Rules: It reads the descriptor to get the segment’s Base Address (where the segment starts in memory) and Limit (the segment’s size).
  5. Check for violations : The hardware performs a sanity check. It ensures the offset is within the segment Limit and that the Requestor Privilege Level (RPL) has permission to access that segment. If either check fails, the CPU blocks the access and raises a General Protection Fault (#GP)
  6. Calculate the Address: It adds the Base Address to the Offset (Effective address) from the original instruction. The result is the Linear Address.

1.2.2 Flat memory model

A flat memory model is a linear representation of the memory (one large segment), denoting that the address space exists in a single contiguous area (e.g., 2^32-bit extending from 0x0 to 0xFFFFFFFF). To access the desired data, the flat memory model theoretically requires no additional operations on a given address as it does in segmentation; modern operating systems use the flat memory model because of its speed and simplicity.

To create a Flat Memory Model, the operating system effectively “turns off” the boundaries of segmentation. It does this by configuring every segment’s descriptor to start at the very beginning of memory (0x0) and stretch all the way to the maximum possible limit (0xFFFFFFFF). By making every segment cover the entire available space, the “walls” between them disappear, resulting in a single, continuous block of memory rather than many isolated chunks.

Figure 7 : Flat memory model

2. Modes of operation

Operating systems can interact with memory in different modes, which determine how the operating system manages and allocates memory resources. These modes can be switched as required, and serve to enable functionality, resource protection, multitasking, and to optimize memory capacity beyond its physical limits.

2.1 Real mode (Legacy)

In real mode addresses always correspond to real locations in memory. The processor operates with a 20-bit address bus and a 16-bit data bus, it calculates the physical address of a memory reference by shifting the value of a segment register to the left by four binary digits and then adding the offset address to this value. Thus, two 16-bit values (segment and offset) are combined to form a single 20-bit physical address giving 1 MB of addressable memory.

\[Real\ mode\ (20\ bit) \\ Address\ space = 2^{20} - 1\]

2.2 Protected mode (IA-32)

Protected Mode enables each process to have its own isolated address space, which prevents interference or unauthorized access to memory outside of that space. This mode is used to ensure that each process is protected from other processes running on the system. In this mode, the processors operate with a 32-bit address bus and 32-bit data bus allowing access up to 4 GB of memory.

Protected mode brings set of features to the table:

  • Virtual memory & Memory protection
  • Paging as an additional layer of memory management.

Protected mode is enabled by setting the CR0.PE which corresponds to bit 0 in the CR0 register

Protected mode could operate using either segmented or flat memory model, this could be set by setting bit number 31 in the CR0 register (CR0.PG). Protected mode is used during early boot process (BIOS/UEFI handoff) before the OS sets up the paging.

\[Protected\ mode\ (32\ bit) \\ Address\ space = 2^{32} - 1\]

2.3 Long mode (IA-32e)

Long mode (IA-32e) is used by 64-bit processors, it includes two sub-modes:

  • 64-bit mode — Supports address spaces larger than 64GB and theoretical 64-bit linear addressing (Currently hardware supports only 48 and up to 57 bits, and the rest is reserved)
  • Compatibility mode — Enables a 64-bit OS to run most existing protected mode 32-bit software without modification.
\[Long\ mode\ (64\ bit) \\ Cannonical\ Address\ space = 2^{48} -1 \\ Theoretical\ size = 2^{64} - 1\]

Switching from 64-bit to 32-bit (Compatibility mode)

In the Segment Descriptor (refer to the struct above), there is a flag called the L (l) bit (Long Mode). This flag is responsible for changing how the CPU executes the instructions :

  • L=1: The code in this segment is 64-bit and the CPU will decode the instructions as such.
  • L=0: The code in this segment is 32-bit (Compatibility Mode) and the CPU will decode the instructions as such.

The cannonical Addresses (64-bit mode)

You may wonder, what’s a cannonical address space, it’s a term used to indicate that the addresses are in their cannonical form. A canonical address is a subset of the virtual addresses, a 64-bit processor theoretically can address memory up to $2^{64}-1$ (16 Exabytes , not something that you would ever need for day to day stuff) it’s also expensive to implement, therefore modern hardware typically only implements 48 bits and possibly up to 57 bits of the address bus.

Since the registers are still 64-bits wide and to avoid discrepancies the CPU handles this through enforcing a rule that is

  • the unused upper bits must match the most significant implemented bit.

In the 48-bit implementation ( Same concept applies to the 57 limit )

  • A user space ranges from 0x0000000000000000 to 0x00007FFFFFFFFFFF
  • A kernel space ranges from 0xFFFF800000000000 to 0xFFFFFFFFFFFFFFFF

Sounds confusing right ? Well let’s look at it from a binary perspective which will make more sense.

Figure 7 : Cannonical address breakdown

The user address in this case is represented by the above diagram where bits 0-46 are represented by1 and the 47th bit is 0, in that case the upper 16-bits must match the 47th bit which is 0 therefore the correct hexadecimal representation is 0x00007FFFFFFFFFFF

So the next time you see an address starting with 0xFFFF you would know it’s a kernel space address, and if the address starts with 0x0000 you know it’s a user space address.

Conclusion

We have successfully navigated the first layer of the memory labyrinth. We’ve seen how the Segmentation Unit takes a Logical Address—a simple Selector:Offset pair—and turns it into a physical address.

In a modern Flat Memory Model, this process usually churns out a Linear Address that looks identical to the offset we started with. The segmentation “walls” have been torn down, and we are left with a pristine, continuous 4GB (or larger) address space.

But here is the catch: That Linear Address is a lie.

If you have a 32-bit Linear Address pointing to 0xB0000000 (3 GB), but your computer only has 2 GB of physical RAM installed, that address points to nothing. It is a check written against a bank account that doesn’t exist.

So, how does the OS map a linear request to a physical chip that might not even be there? How do we stop a program from overwriting another program’s memory if they both use the same Linear Address? And how do we break the 4GB barrier to access 32GB of RAM on a 32-bit bus?

The answer lies in the next piece of hardware silicon: The Paging Unit.

In the next part of this series, we will dive into the world of Page Tables, the CR3 register, and the TLB—the machinery that turns the “Linear Illusion” into “Physical Reality.”

5. References

  1. Intel 64 and IA-32 Architectures Software Developer’s Manual
  2. Understanding the linux kernel
  3. https://wiki.osdev.org/CPU_Registers_x86
  4. http://www.osdever.net/tutorials/view/the-world-of-protected-mode
  5. Protected Mode on Bona Fide OS Developer