logo
Published on

My notes of OS internals

[!NOTE] unsaved buffer of notes in Neovim caused me to lose my long notes. I had been working for 6-7 hours without saving them. This is rare since I often press :w every minute, but I was too immersed in books and writing that I forgot to save.

I don't enjoy rewriting, but I will do it again.

As we going to learn about Linux/OS internals, we will start with the booting process and how everything begins when the power button is pressed. A hardware interrupt prompts the motherboard to request a power supply. When the motherboard receives the power good signal, a hardware reset and Built-In Self Test (BIST) are performed. BIST sets EAX to 0 if tests are passed. Hardware resets have the highest priority in processor execution (Vol. 3A, Section 6.9 of the Intel Manual). A hardware reset sets processor registers to their default state and switches the processor to real mode (16-bit mode). In real mode, only a 20-bit address bus is available, providing 2^20 = 1MB of addressable space (external memory). In 16-bit mode, registers can have a maximum address of 2^16 - 1 = 65535, which equals 64 KB.

ghci> showHex (2^20 - 1) ""
"fffff"
ghci> showHex (2^16 - 1) ""
"ffff"

In real mode:

Physical Address = Segment Selector * 16 + Offset
Memory (in Hex):

|-------|-------|-------|-------|-------|-------|
0x00000  0x10000 0x20000 0x30000 0x40000 0x50000
   ↑       ↑
 0x1234    0x1234 * 16  (Segment Start)
               + 0x5678 (Offset)
                |
              0x17958 (Final Address)

Maximum CS:IP:

Physical Address = (CS << 4) + IP

ghci> showHex (0xffff `shiftL` 4 + 0xffff) ""
"10ffef"
ghci> let result = (0xffff `shiftL` 4 + 0xffff) - (2^20 - 1)
ghci> result
65520

By combining these values, we can visualize how addresses are calculated. When the processor resets, it starts from address FFFFFFF0H – which is 16 bytes below the top of the 4GB memory space. This is where the EPROM (containing the boot code) is located.

Here’s the interesting part – FFFFFFF0H is outside the usual 1MB real-mode limit. So, how does the CPU handle it?

  • After reset:
    • CS (Code Segment) = F000H
    • CS Base (hidden part) = FFFF0000H
    • EIP (Instruction Pointer) = FFF0H
    • Final address = FFFF0000H + FFF0H = FFFFFFF0H

Once the system runs the first far jump or interrupt, CS behaves normally again:

Physical Address = (CS << 4) + IP

Until the boot code finishes, do not perform far jumps, calls, or allow interrupts – otherwise, CS will recalculate, causing potential issues.

To verify this, you can refer to Intel or AMD manuals. However, let’s explore some relevant code directly. I highly recommend reading the Coreboot documentation, which contains valuable insights. I will cover only essential points here for brevity.

You can check out this section of the Coreboot source code. This shows the reset vector address – the exact point from which execution begins in the x86 architecture. This is why it is known as the reset vector.

I recommend reading the UEFI RFCs and the UEFI specification. Chapter 5 (page 111) provides critical information about partitioning, so I won’t elaborate here.

Once the BIOS/UEFI starts and initial checks are completed, the BIOS/UEFI searches for a bootable device. Typically, it scans for disk drives with a valid Master Boot Record (MBR) or GUID Partition Table (GPT). When a bootable drive is found, the BIOS reads the contents of the boot sector into memory at a fixed address and transfers execution to the code residing there.
For more information, refer to the Linux kernel documentation on x86 boot.

uefi driver compiles to efi byte code object file ,firmware contains a efi byte code interpreter this allowed uefi to support mutiple processors without being too bulky to contain instruction for each architecture so it is shipped in its own byte code format ,lz77 algorithm is used to replace repeated strings with a pointer to previous address of string and huffman coding is used for symbols.

[!IMPORTANT] Implementing the above algorithm is good excerise, rather than spending time on stupid problems on random coding websites. Details of the algorithm can be found in the Unified Extensible Firmware Interface (UEFI) Specification, Release 2.11 (page 795). This section is well-written and worth studying.