History of Virtualization and Architectural Evolution - Oracle Solaris 10 System Virtualization Essentials - page 333

Performance and Manageability Challenges and Responses
VM systems pushed the limits of contemporary systems in hosting multiple work-
loads. Early implementations were particularly prone to delivering disappointing
performance due to poorly understood causes. Some workloads took several times
longer to run in a virtual machine than natively, even when no other users were
competing for resources. Solutions were slow to evolve, and a few problems remain
for which there are still no complete answers. Some issues discovered in early vir-
tual machine systems continue to affect today's systems, and current solutions to
these issues are influenced by the early experiences. This section describes some
of the most notable issues, the early attempts to resolve them, and solutions for
handling them on current systems.
Performance Effects of Instruction Emulation
The most obvious form of overhead in virtual machine environments is the cost
of emulating privileged operations executed by a guest's OS, as described previ-
ously. Guest operating systems need to execute these instructions correctly—for
example, disabling or enabling interrupt masks to enter or exit a critical section,
issuing I/O, marking memory pages as referenced, or changing memory maps
when dispatching different processes. These instructions must not be allowed to
run directly on the real machine because they would break the integrity of the
hypervisor and its guests. Therefore, guests run in a mode where executing these
instructions results in a hardware trap. The general (and greatly simplified) flow
of execution is as follows:
1. The guest OS in the virtual machine executes a privileged instruction that
generates a hardware trap.
2. Context-switch to the hypervisor saves the machine state and determines the
trap's cause.
3. The hypervisor emulates the instruction in software, in the context of the
guest.
4. The hypervisor does a context switch back to the guest, if it is still runnable.
A similar effect occurred with the SVC (“supervisor call”) instruction that ap-
plications used to request OS services. This instruction is similar in purpose and
implications for virtual machines to instructions used to request system calls on
other platforms. The latter include the ta software trap instruction on SPARC
and the int $0x80 and sysenter instructions on Intel. When executed in a real
machine, these instructions causes a context switch to a location pointed to by
an interrupt vector in real memory, and thus a context switch into the OS. When