A look at Windows Virtual Memory mechanisms
Issued by http://blogs.msdn.com/slavao/archive/2005/01/29/363181.aspx
Even though the size of x86 registers is 32 bit the platform enables an OS to manipulate with 64GB of RAM using Physical Address Extensions, PAE. On Windows PAE can be turned on from boot.ini file. If PAE is not turned on, Windows will only be able to manipulate with up to 4GB of RAM even though box might have more of RAM installed.
Some user applications require more VAS than 2GB. Windows have ability to configure VAS of user application up to 3GB. This feature has significant drawback. It limits amount of VAS available to kernel down to 1GB. Increasing size of user VAS can be done by adding /3GB switch to boot.ini file. Once modification is complete system will require reboot.
Limiting kernel VAS to 1GB affects the whole box not only the application that needs larger size VAS. For example ones 3GB switch is turned on amount of RAM that the OS can support, if PAE is enabled, drops down from 64GB to16GB. 3GB switch affect all kernel components including all drivers. Having 3GB switch turned on can cause effects from drop in performance and memory allocation failures to system stalls. My suggestion is to avoid usage of 3GB switch unless you really really need it.
As I described, on x86 platforms VAS is limited to 2GB or 3GB depending on configuration in boot.ini file. On AMD64, 32 bit application can have up to 4GB. Such small VAS could be a significant drawback for high end servers manipulating with gigabytes and terabytes of data, think about SQL Server. I also said that PAE switch enables Windows to manipulate with up to 64GB of RAM. But how one process does access so much memory? In order to enable single process to manipulate with amount of RAM bigger than size of VAS, Windows provides Address Window Extension, AWE, mechanism (Remember your DOS yearsJ). Be careful many authors use term AWE memory. There is no such thing as AWE memory, one can’t go and buy AWE memory at a store J. . Since there is no AWE memory, there is no either low AWE nor high AWE memory. There is AWE mechanism that enables one to manipulate with amounts of RAM larger than VAS! The principle is simple. Using AWE API one can allocate physical memory, then using VirtualAlloc API allocate VAS regions and then using AWE mechanism bind/ubind VAS regions and physical memory. As with locked pages in order for an application to use AWE mechanism it needs to have Locked pages in memory privilege enabled.
AWE API can be used even on the boxes with RAM below of a process’s VAS. In fact AWE could be used to avoid any type of paging (Remember that when locking pages in memory using VirtualLock mechanism Windows still can page out the whole process) When using AWE mechanism, the OS can’t interfere at all. With such flexibility comes a difficulty. Misuse of AWE mechanism can stall the whole box so that only way to recover from such state could be a reboot.
Well I think it is enough info for today J From this and previous posts all I want you is to grasp the concepts. They are really important! Here is the quick summary:
- Every process has its own VAS
- VAS is often neglected by developers.
- VAS is limited resource even on 64 bit platform
- VAS is managed by Windows the same way as one would manage a heap
- VAS's smallest region is 64kb
- VAS regions is managed by corresponding VAD in kernel
- VAD describes state of VAS region - allocated, committed, uncommitted
- A VAS page bound to physical page in RAM on demand, the first time page is accessed, unless running without page file).
- Physical pages can be private or shared
- Set of all private physical pages for a process is called private bytes (PM), or virtual memory size (TM)
- Set of private physical pages in RAM is called process working set.
- Physical pages bound to specific VAS region can be locked in memory
- PAE switch enables Windows’s support for 64GB of RAM
- On x86 VAS is limited to 2GB, 3GB or 4GB
- /3GB switch increases user VAS to 3GB and decreased kernel VAS to 1GB.
- /3GB switch limits amount of physical memory enabled through PAE switch down to 16GB
- /3GB switch is not recommended
- AWE is mechanism, it is not memory.
- AWE enables single process to use memory outside of VAS limits.
- /3GB swith is not required to enable AWE mechanism
- /PAE switch is not required to enabel AWE mechansim
Some interesting gotchas:
- Allocating physical memory using AWE mechanism is slow on Windows 2000. (This is a reason why SQL Server 2000 doesn’t manage memory dynamically when AWE is enabled). The problem is fixed in Windows 2003 server so that Yukon does allocates physical memory dynamically
- Physical pages allocated through AWE mechanism are not part of process working set neither large pages (I will cover large pages sometime latter). This is exact reason why neither PerfMon or Task Manager show them