Hypervisor Software

ARMv7-a archtecture virtualization extensions add hypervisor mode to the conventional ARM modes.

hypervisor extensions provide three privileged extensions, PL0, PL1, PL2.

• PL0 is user mode, and typically used by applications

• PL1 is hyperv mode and used by the guest OS

• PL2 is used by the hypervisor to interact with harware.

The main components of hypervisor software on ARM architectures include:

• Memory Management

• Device Emulation

• Device assignment

• Exception Handling

• Interrupt Handling

• Scheduling

• Context Switching

If you're following the video series that I posted about in device-tree (or any other guide), I would highly recommend going through the ARM documentation, as well as any other educational text on hypervisors. It is tempting to just follow code, but you will miss a lot if you don't understand OS and hypervisors.

Memory Management

The hypervisor is responsible for memory management for itself and for the guest OS.

This includes virtual address translation. PL2 MMU is used in Hyp mode to translate virtual addresses used by the hypervisor to physical addresses.

The hypervisor must setup and manage translation tables for itself and guest (stage 2).

TTBR has secure and non secure version

Virtualization Translation Table Base Register (VTTBR) & Hyp Translation Table Base Register (HTTBR) - VTTBR = used to point to second stage translation tables (for guests) - translate intermediate physical mem to physical mem - hypervisor handles aborts - regarding faults the hypervisor can emulate or allocate memory to guest, etc. - unexpected faults, the hyperv can terminate the guest or report aborts - HTTBR = points to tables used for hypervisor memory mapping

For more important information on memory translation, please visit the ARM documentation for memory translation.

Briefly,

L1 translation tables have 4096 entry's, because a virtual address uses 12 bits [31:20] to map to one of those entry's - The base address of the L1 translation table is in CP15 TTB register

The L1 table points to an L2 table with 256 entries - 8 bits of the virtual address [19:12] are used to select one of the entries, which provides base address of the page. - this base is combined with remaining virtual address bits [11:0] to form physical address.

PL1&0 stage 1 - When the core is in one of PL1 (kernel) or PL0(application) mode - unlike conventional systems, the virtual is not translated to physical but Intermediate phyiscal Address (IPA). PL1&0 stage 2 - includes translation tables for each guest, these are L2 tables. - translate IPA (from stage 1) to true Physical address for accessing system memory - hypervisor saves and restores core registers used for L2/stage2 translation tables. These registers are provided by the virtualization extensions - guest is unaware of l2 tables PL2 - Translation tables the hypervisor uses for itself to manage its own virtual memory. - extensions provide registers for the hypervisor to manage its own translation tables

Device Emulation

Guest access to memory-mapped devices on the platform are subject to stage 2 (VTTBR) translation. - The VTTBR points to stage 2 translations

If multiple guests are running, device emulation is necessary - This is unecessary for this project. I am not concerned with security or running multiple guests

This is obviously because of the need to manage access to shared resources. The hypervisor can use the stage 2 translation table descriptors to prohibit access to a region to every guest.

** alternatively ** The hypervisor can 'hide' the device from guests because it is unavailable. - guests normally detect devices using id register and device tree.

This will allow the hyperv to provide dummy values for reads and ignore writes, tricking the guest into thinking the device doesn't exist.

Hypervisor must schedule guests on available cores - similar to kernel OS scheduling - Hypervisor schedules a guest to a core to execute - Guest receives full control of core and schedules tasks on core

Device Assignment

(Copied from ARM documentation) Device emulation is necessary but turns out to be expensive as all accesses to the device by the guest have to be trapped and emulated in software. The Hypervisor has the option of assigning individual devices to individual guests so that the guest can own and operate the device without requiring Hypervisor arbitration. The challenge is to hide from the guest the fact that the device is actually located at a different physical address, and generates a different interrupt ID than that which the guest is expecting. Transparent Stage 2 mappings, and interrupt virtualization can circumvent these challenges.

Exception Handling

Uses the Hyp Configuration Register (HCR) to route exceptions to hypervisor mode - Hypervisor is called using the HVC instruction - if hypervisor is in hypervisor mode - this causes HVC exception - if hypervisor is in privelidged mode - the hypervisor traps exception

Virtualization extensions allow for virtualized exceptions. The hypervisor manufactures these in response to a real exception or Device emulation needs.

CPSR A, I and F bits no longer mask physical exception when routed to hypervisor when signaled through the virtual GIC - Remember, I and F bits in the CPSR are for FIQ and IRQ exceptions. If set, the mask the respective exception suite, preventing such exceptions from interrupting CPU.

Interrupt Handling

Hypervisor Control Register has a set of 3 bits for IRQ, FIQ, and virtual exceptions - If core is in PL1&0 mode, the effect matches normal interrupt handling by the Core.

Scheduling

In a virtualized system the hypervisor schedules a guest on a core to execute. A scheduled guest assumes sole ownership of the core, and schedules its tasks on the core, as it would do on a conventional system.

Context Switching

When another guest is scheduled on a core the hypervisor, it must save state of the current guest.

The following elements of a guest context must be saved and restored:

• The general purpose registers of the core including the banked registers of all modes,

• System register contents for such things as memory management and access control.

• The pending and active states of private interrupts on the core.

• In case of guests using core private timers, the timer registers must be saved and restored so that they generate interrupts at the expected intervals.