1.5
ARCHITECTURE OF MICROSOFT WINDOWS OS
If you buy a CPU case with a power supply, a motherboard, a CPU, some RAM, a hard disk, a CD drive, a display card, a keyboard, and a monitor, and if you assemble everything together the right way, you get a computer. But to get the computer to do anything useful, you need software. Computer software can be roughly classified as system software and application software. System software manages the computer and related peripherals to provide support for application software, which solves real problems for users. The most fundamental system software is the operating system, which manages all of the computer's resources and provides a nice interface to the application software.
The raw machines we are working on are very primitive and awkward to program, although they are much more powerful than their predecessors. One of the operating system's main tasks is to make the raw machine easier to program through a set of well-defined system calls. The system calls are implemented by the operating system in privileged processor mode, and they define the interface between the operating system and user programs running in nonprivileged processor mode.
Microsoft Windows NT/2000 is a little bit different from traditional operating systems. The Windows NT/2000 operating system consists of two major parts: a privileged kernel mode part and a nonprivileged user mode part.
The kernel mode part of the Windows NT/2000 OS runs in the privileged processor mode that has access to all CPU instructions and the full address space. On the Intel CPU, this means running in CPU privilege level 0, which has access to 4 GB of address space, IO address space, etc.
The user mode part of Windows NT/2000 OS runs in the nonprivileged processor mode that only has access to limited CPU instructions and user address space. On the Intel CPU, the user mode code is running in CPU privilege level 3 and has access only to the lower 2 GB of per-process address space with no access to IO ports.
The kernel mode part of the Windows NT/2000 operating system provides system services and internal processes to the user mode part of the operating system. Microsoft calls this portion Executive. It's the only entry point into the operating-system kernel. For security reasons, Microsoft provides no back doors. The major components are:
Hardware Abstraction Layer (HAL): a layer of code that isolates the OS kernel mode parts from platform-specific hardware differences.
MicroKernel: low-level operating system functions, such as thread scheduling, task switching, interrupt and exception dispatching, and multiprocessor synchronization.
Device Drivers: hardware device drivers, file systems, and network support that implements user I/O function calls.
Window Management and Graphics System: implements graphical user interface functions, such as windows, controls, drawing, and printing.
Executive: basic operating-system services, such as memory management, process and thread management, security, I/O, and interprocess communication.
The user mode part of Windows NT/2000 normally consists of three components:
System Processes: special system processes, such as logon process and session manager.
Services: such as event log and schedule services.
Environmental Subsystems: provides native operating-system services to user programs through well-defined APIs. Windows NT/2000 supports the running of Win32, POSIX (Portable Operating System System Interface, an inter national standard for C language level API to the operating system), OS/2 1.2 (an operating system from IBM), DOS, and Win16 programs.
Hardware Abstraction Layer
The responsibility of the Hardware Abstraction Layer (HAL) is to provide a platform-specific support for the NT kernel, I/O manager, kernel-mode debuggers, and lowest-level device drivers. Having a HAL in the OS makes Windows NT/2000 not so dependent on a single particular hardware platform or architecture. HAL provides abstractions for device addressing, I/O architecture, interrupt management, DMA (Direct Memory Access) operations, system clocks and timers, firmware, and BIOS inter facing and configuration management.
When Windows NT/2000 is installed, the actual module for HAL is system32\hal.dll. But actually, there are different HAL modules for different architectures; only one of them gets copied to the system directory and renamed as hal.dll. Check your Windows NT/2000 installation CD, and you will find quite a few HALs, such as HALACPI.DL_, HALSP.DL_, and HALMPS.DL_. ACPI here means Advanced Configuration and Power Interface.
To check what's really provided by HAL on your system, use dumpbin hal.dll/export. You will find exported functions like HalDisableSystemInterrupt, HalMakeBeep, HalSetRealTimeClock, READ_PORT_UCHAR, WRITE_PORT_UCHAR, etc. Routines exported by HAL are documented in the Windows 2000 DDK, under Kernel-Mode Drivers, References, Part 1, Chapter 3.0: Hardware Abstraction Layer Routines.
MicroKernel
The Windows NT/2000 MicroKernel manages the main resource of the machine, the CPU. It provides support for interrupt and exception handling, thread scheduling and synchronization, multiprocessor synchronization, and timekeeping.
MicroKernel provides its services through objected-based interfaces to its client, much like the objects and handles used in the Win32 API. The main objects provided by the MicroKernel are dispatcher objects and control objects.
Dispatcher objects are used for dispatching and synchronization. They include events, mutexes, queues, semaphores, threads, and timers. Each dispatcher object has a state, which is either signaled or not signaled. MicroKernel provides routines accepting dispatcher object states as parameters (KeWaitxxx). Kernel-mode threads synchronize themselves by waiting for dispatcher objects or user mode objects that have embedded kernel mode dispatcher objects. For example, the Win32 user level event object has a corresponding MicroKernel level event object.
Control objects are used to manage kernel mode operations except dispatching and synchronization, which are covered by dispatcher objects. Control objects include asynchronous procedure call (APC), deferred procedure call (DPC), interrupt, and process.
A spin lock is a lower-level synchronization mechanism defined by the NT kernel. It's used to synchronize access to shared resources, especially in a multiprocessor en vironment. When a routine tries to acquire a spin lock, it �spins� until the lock is acquired, doing nothing else.
MicroKernel resides in NTOSKRNL.EXE on your system, which contains both the MicroKernel and the Executive. There are two versions of MicroKernel on your installation CD: NTKRNLMP.EX_for multiprocessor systems and NTOSKRNL.EX_for single-processor systems. Although the module has an .EXE extension, it's actually a DLL. Among hundreds of exported functions from NTOSKRNL.EXE, approximately 60 of them belong to the MicroKernel. All routines supported by the MicroKernel start with �Ke.� For example, KeAquireSpinLock lets you acquire a spin lock which enables you to access shared data in a multiprocessor-safe way. KeInitializeEvent initializes a kernel event structure, which can then be used in KeClearEvent, KeReset Event, or KeWaitForSingleObjects. Kernel objects are explained in Windows DDK, under Kernel-Mode Drivers, Design Guide, Part 1, Chapter 3.0: NT Objects and Support for Drivers. Kernel routines are documented under Kernel-Mode Drivers, References, Part 1, Chapter 5.0: Kernel Routines.
Device Drivers
We know that the MicroKernel manages the CPU; HAL manages things like bus, DMA, timer, firmware, and BIOS. For the computer to provide real value to users, the operating system needs to interface with lots of different kinds of devices, like the video display, mouse, keyboard, hard disk, CD ROM, network card, parallel ports, serial ports, etc. The operating system needs device drivers to talk to these hardware devices.
Most of the device drivers on Windows NT/2000 are kernel mode drivers, except Virtual Device Drivers (VDD) for MS-DOS applications and Windows 2000 user-mode printer drivers. Kernel mode device drivers are DLLs loaded into kernel address space, depending on hardware configuration and user settings.
The Windows NT/2000 operating system's interface to device drivers is layered. User applications call API functions like Win32 CreateFile, ReadFile, WriteFile, etc. The calls are translated to calls to I/O system services provided by the Windows NT/2000 Executive. The I/O manager with the Executive sets up I/O request packets (IRPs), passing them through possibly layered drivers to physical devices.
Windows NT/2000 defines four basic types of kernel mode drivers, which have different structures and functionality:
Highest-Level Driver: mostly file system drivers like the File Allocation Table (FAT) file system (inherited from DOS), NT file system (NTFS), and CD-ROM file system (CDFS) drivers, and the NT network server or redirector. File system drivers may implement a physical file system on a local hard drive; they may also implement a distributed or networked virtual file system. For example, some source-code version-control systems are implemented as a virtual file system. Highest-level drivers depend on lower-level drivers to function.
Intermediate Drivers: such as virtual disk, mirror, or device-type-specific class drivers, drivers in the network transport stack, and filter drivers. They provide either value-added features or per-class processing for devices. For example, there is a class driver for parallel-port communication. Intermediate drivers also depend on support from underlying lower-level drivers. There may be multiple intermediate drivers in a driver stack.
Lowest-Level Drivers, sometimes called device drivers: such as the PnP hardware bus driver, legacy NT device drivers, and network interface controller (NIC) driver.
Mini-Drivers: customization module for generic drivers. A mini-driver is not a full driver. It resides inside a generic driver, which becomes its wrapper, to customize it for specific hardware. For example, Microsoft defines a printer UniDriver, which is an actual printer driver. Printer vendors can develop a printer mini-driver which will be loaded by UniDriver to make it print to the printer provided by the vendor.
A device driver may not always correspond to a physical device. A device driver is an easy way for programmers to write a module that can be loaded into kernel address space. With your module in kernel address space, you can do useful things that would not be possible otherwise. With well-defined file operations in the Win32 API, your user mode application can easily communicate with your kernel mode driver. For example, www.sysinternals.com provides several very useful NT utilities which use NT kernel mode device drivers to achieve registry, file system, and I/O port monitoring. Chapter 3 of this book shows a simple kernel mode driver that reads data from kernel address space. It is used extensively in analyzing Windows graphics subsystem data structures.
While most device drivers are part of the I/O stack controlled by the Executive I/O manager and have similar structures, some device drivers are exceptions. Device drivers for the Windows NT/2000 graphics engine�for example, display driver, printer driver, video port driver�use a different structure and a direct calling sequence. Windows 2000 even allows printer drivers to run in user mode. We will say more about display drivers and printer drivers in Chapter 2.
Most of the modules loaded into kernel mode address space are device drivers. The drivers tool in Windows NT/2000 DDK can list them in a DOS box. You will find tcpip.sys for networking, mouclass.sys for mouse, kbdclass.sys for keyboard, cdrom.sys for CD-ROM, etc.
For complete discussions on Windows 2000 device drivers, read the Windows 2000 DDK, Kernel-Mode Drivers, Design Guide and References.
Window Management and Graphics System
In the earlier versions of Microsoft Windows NT, security was a major concern for OS design, so window management and graphics system ran in user address space. This led to so many performance problems that Microsoft made a major change in design by moving the windowing and graphics system from user mode to kernel mode starting with Windows NT 4.0.
Window management supports the foundation of the Windows graphic user interface by implementing window class, window, window messaging, window hooks, window property, menu, title bar, scrollbar, cursor, virtual key, clipboard, etc. It's basically the kernel counterpart of USER32.DLL, which implements what's defined in the winuser.h portion of the Win32 API.
Graphics system implements the real GDI/DirectDraw/Direct3D drawing, calling on a physical device or memory-based device. It relies on graphics device drivers like display drivers and printer drivers. Graphics system is the backbone for GDI32.DLL, which implements what's defined in the wingdi.h portion of the Win32 API. Graphics system also provides strong support for display drivers and printer drivers, by providing a fully featured rendering engine for several standard format bitmap surfaces. We will cover the details of the graphics system in Chapter 2.
Windowing and the graphics system are packed into a single huge DLL, win32k .sys, of around 1.6 MB. If you look at the exported functions from win32k.sys, you can find graphics system entry points�for example, EngBitBlt,PATHOBJ_bMove To�but not window management entry points. The reason is that window management entry points are never called by other parts of the OS kernel components, while functions in graphics system need to be called by graphic device drivers. GDI32.DLL and USER32.DLL calls WIN32K.SYS through system service calls.
Executive
Microsoft defines Windows NT/2000 Executive as the collection of kernel-mode components that form the base Windows NT/Windows 2000 operating system. Besides HAL, Micro-Kernel, and Device Drivers, Executive also includes the Executive Support, Memory Manager, Cache Manager, Process Structure, Interprocess Communication (LPC, local procedure call and RPC, remote procedure call), Object Manager, I/O Manager, Configuration Manager, and Security Reference Monitor.
Each component within the Executive provides a set of system services which can be used from user mode through interrupts, except Cache Manager and HAL. Every component also provides an entry point that can be called only by modules running in kernel address space.
Executive Support provides a set of functions callable from kernel mode, which normally starts with the prefix �Ex�. Kernel memory allocation is a main functionality provided by Executive Support. Windows NT/2000 uses two dynamically growing memory regions, called pools, to manage dynamic memory allocation in kernel mode address space. The first is nonpaged pool, which is guaranteed to be resident in physical RAM all the time. Critical routines like interrupt services can use nonpaged pool without worry about generating paging interrupts. The second pool is paged pool, which is much bigger but can be swapped out to the hard disk when physical RAM is low. For example, storage for Win32 device-dependent bitmaps is allocated from paged pool using the family of ExAllocatePoolxxx routines. Executive Support also provides an efficient fixed-size block memory allocation scheme called look-aside lists, using routines like ExAllocatePagedLockasideList. Multiple look-aside lists are allocated from the pools when the system boots up. Executive Support provides a rich set of atomic operations, such as ExInterlockedAddLargeInteger, ExInterlockedRemoveHeadList, InterlockedCompareExchange, etc. Other Executive Support functionality includes fast mutex, callback, throw exception, time translation, Universally Unique Identifier (UUID) creation, etc.
Memory Manager supports virtual memory management, balance set management, process/stack swapping during thread swapping, modified page writer, mapped page writer, system cache, page file, virtual memory to physical memory mapping, etc. Memory Manager provides routines like MmFreeContiguousMemory, MmGet PhysicalAddress, MmLockPageableCodeSection, etc. For details of Memory Manager, check Chapter 5 of Inside Windows NT, second edition.
Cache Manager provides data caching for Windows NT/2000 file-system drivers. Cache Manager routines have the prefix �Cc.� Cache Manager exports routines like CcIsThereDirtyData and CcCopyWrite. For details, check Chapter 8 of Inside Windows NT, second edition.
Process Structure routines are responsible for creating and terminating kernel mode system thread, process/thread notification, and process/thread query. For example, Memory Manager could use PsCreateSystemThread to create a kernel thread to manage dirty page writing.
Object Manager manages common behavior of objects exposed from the Executive. The Executive supports creation of directory, event, file, key, section, symbolic link, and timer objects, through routines like ZwCreateDirectorObject and ZwCreate File. Once created, Object Manager routine ObReferenceObject/Ob De refer ence Count updates reference count, ObjReferenceObjectByHandle validates object handle and returns the pointer to the object's body.
I/O Manager translates I/O requests from the user mode program or other kernel mode components into properly sequenced calls to various drivers on driver stacks. It provides a very rich set of routines. For example, IoCreateDevice initializes a device object for use by a driver, IoCallDriver sends an I/O request packet to the next-lower-level driver, and IoGetStackLimits checks the current thread's stack boundary.
Windows NT/2000 Executive also contains a small runtime library support, which is parallel to the C runtime library but much smaller. The kernel runtime library supports double-link list, Unicode conversion, bit operations, memory, large integer, registry access, time conversion, string manipulation, etc.
On Windows NT/2000, Executive and MicroKernel are packed into a single module NTOSKRNL.EXE, which exports more than 1000 entry points. Entry points exported by NTOSKRNL.EXE normal have distinguishing double-letter prefixes that indicate the components they belong to�for example, �Cc� for cache manager, �Ex� for Executive Support, �Io� for I/O manager, �Ke� for MicroKernel, �Ob� for object manager, �Ps� for process structure, �Rtl� for runtime library, �Dbg� for debugging support, etc.
System Services: Native API
The rich functionality provided by the kernel part of the Windows NT/2000 operating system is finally exposed to user mode modules through a single narrow gate, which is interrupt 0x2E on the Intel CPU. It's served by an interrupt service routine KiSystem Service, which is in NTOSKRNL.EXE but not exported. Because the service routine is running in kernel mode, the CPU switches itself to privileged execution mode auto matically, making the kernel address space addressable.
Although a single interrupt is used, more than 900 system services are provided by Windows NT/2000 through a service index number in EAX register (on the Intel CPU). NTOSKRNEL.EXE keeps a table of system services named KiServiceTable; so does WIN32K.sys in W32pServiceTable. System service tables are registered by calling KeAddSystemServiceTable. When KiSystemService services a system service call, it checks if the service index is valid and the expected parameters accessible, then dispatches to the right routine handling a particular system service.
Let's look at a few simple system services, using the Microsoft Visual C++ debugger with the help of the Windows 2000 debug symbol files. If you step into Win32 GDI and call CreateHalftonePalette in assembly code, you will see:
_NtGdiCreateHalftonePalette@4:
mov eax, 1021h
lea edx, [esp+4]
int 2Eh
ret 4
The Win32 USER function GetDC is implemented as:
_NtUserGetDC@4:
mov eax, 118bh
lea edx, [esp+4]
int 2Eh
ret 4
The Win32 KERNEL function CreateEvent is more complicated. Create EventA calls CreateEventW, which then calls NtCreateEvent from NTDLL.DLL. NtCreateEvent is implemented by:
_NtCreateEvent@20:
mov eax, 1Eh
lea edx, [esp+4]
int 2Eh
ret 14h
The Windows NT/2000 system calls are almost completely hidden from programmers. SoftICE/W from Numega provides an ntcall command that lists some kernel system calls. For more information on system calls, refer to Mark Russinovich's article �Inside the Native API,� on www.sysinternals.com. We will cover more details about GDI system calls in Chapter 2.
System Processes
The Windows NT/2000 operating system employs a few system processes in order to manage login, services, and user processes. You can find list of system processes in task manager, or you can use the Task List tool (tlist) which comes with Platform SDK.
When Windows NT/2000 is running, there are basically three hierarchies of processes. The first hierarchy contains a single system process, the system idle process, which always has process id 0. The second hierarchy holds all other system processes. It starts with a process named system, which is the parent of the session manager process (smss.exe). The session manager process is the parent of the Win32 subsystem process (csrss.exe) and the logon process (winlogon.exe). The third hierarchy starts with the program manager process (explorer.exe), which is the parent of all user processes.
Here is an annotated Windows 2000 process tree, as displayed by tlist �t command:
System Process (0) System Idle Process
System (8)
smss.exe (124) Session Manager
csrss.exe (148) Win32 Subsystem server
winlogon.exe (168) logon process
services.exe (200) service controller
svchost.exe (360)
spoolsv.exe (400)
svchost.exe (436)
mstask.exe (480) SYSTEM AGENT COM WINDOW
lsass.exe (212) local security authentication server
explorer.exe (668) Program Manager
OSA.EXE (744) Reminder
IMGICON.EXE (760)
The Process Walker tool (pwalker.exe) displays more per-process information. With Process Walker, you can find that the System Idle Process has a single thread with 0 as starting address. It may well be a mechanism for the operating system to account for idle time instead of being a real process. The System Process has a valid starting address in kernel address space and dozens of threads all with a starting address in kernel address space. So the System Process is also the home for kernel mode system threads. If you translate the thread starting addresses in the System Process to symbolic names, you will find interesting names like Phase1Initialization, ExpWorkerThread, Exp Worker Thread BalanceManager, MiDereferenceSegmentThread, MiModified Page Writer, Ke BalancedSetManager, FsRtlWorkerThread, etc. Although all the threads listed here are created by the Executive, other kernel components can create kernel threads, too. Both the Idle Process and the System Process are pure kernel mode components which do not have any modules in user mode address space.
Other system processes like session manager, logon process, etc., are user mode processes, started from a PE format execution file. For example, you can find smss.exe, csrss.exe, and winlogon.exe in the system directory.
Services
Microsoft Windows NT/2000 supports a special application type known as a service program, which is normally a console program controlled by the Service Control Manager (SCM). A service program can provide several services. Unlike normal user programs, services can be scheduled to start automatically at system boot time before user logon.
Several service programs are started when the operating system boots up. To list the service programs and services currently running on your system, use Task List (tlist .exe) with the �s option. Here is a sample list of service programs and services:
200 services.exe Svcs: AppMgmt, Browser, dmserver,
Dnscache, EventLog, LanmanServer,
LanmanWorkstation,
LmHosts, Messenger, PlugPlay,
ProtectedStorage, seclogon,
TrkWks
212 lsass.exe Svcs: PolicyAgent, SamSs
360 svchost.exe Svcs: RpcSs
400 spoolsv.exe Svcs: Spooler
436 svchost.exe Svcs: EventSystem, Netman, NtmsSvc,
RasMan, SENS, TapiSrv
480 mstask.exe Svcs: Schedule
Of particular interest to this book is the spooler service, which handles print jobs on a local machine and printing to printers over a network. We will talk more about spooler service in Chapter 2.
Environment Subsystems
When Windows NT was initially designed, not many Win32 programs had been written to run on it. So Microsoft thought it was important for Windows NT to support environments to run DOS programs, Win16 programs, OS/2 programs, POSIX (UNIX-style interfaces) programs, and Win32 programs on the same operating system. Windows NT/2000 provides different environment subsystems to run these totally different kinds of programs.
An environment subsystem is a set of processes and DLLs that exposes a subset of the operating-system services to application programs written for the specific subsystem. Each subsystem has a single subsystem process, which manages interaction with the operating system. The DLLs are mapped into application processes to allow interfacing with the subsystem process or direct interaction with the kernel portion of the operating system through OS system service routines.
The Win32 environment subsystem is gradually becoming the major subsystem supported by the Windows NT/2000 family of operating systems. Its environment subsystem process (csrss.exe) is used to handle all window management and graphic display in the user address. Win32 application programs have to issue local procedure calls to its subsystem process for window management and display, which causes performance problems. Starting from Windows NT 4.0, Microsoft moved the major portion of the Win32 environment subsystem into a kernel mode DLL, win32k.sys, together with all the graphical device drivers. Win32 subsystem DLLs should be very familiar to Windows programmers. We have KERNEL32.DLL, which handles virtual memory, I/O, heap, process, thread, and synchronization; USER32.DLL, which handles window management and message passing; GDI32.DLL, which handles graphic display and printing; ADVAPI32.DLL, which manages registry, etc. Win32 subsystem DLLs allow direct access to the OS kernel system services, and provide extra useful features not provided by OS system services. The enhanced metafile (EMF) is an example of a Win32 API feature not directly supported by win32k.sys.
The remaining two environment subsystems, OS/2 subsystem and POSIX subsystem, rely on the Win32 environment subsystem to run, although they used to be the major players in the initial design of Windows NT.
The Win32 environment subsystem now becomes a vital part of the operating system, which is always running. The OS/2 and POSIX subsystems are started only when needed by a program requiring such subsystems.
No comments:
Post a Comment