Rootkits and Rootkit Detection (Windows Forensic Analysis) Part 1


At the RSA Conference in February 2005, Mike Danseglio and Kurt Dillard, both from Microsoft, mentioned the word rootkit, and the ensuing months saw a flurry of activity as "experts" pontificated about rootkits and software companies produced tools to detect them. Even though rootkits had been around for years, originating in the UNIX world and then migrating over into the Windows realm, this issue was largely misunderstood and in some corners even ignored, in a "head buried in the sand" sort of way. The mention of rootkits at the 2005 conference resulted in a surge of interest in rootkits, and commercial rootkit detection tools were announced soon after. (There had been several freeware tools and methodologies available for some time.) As detection techniques have improved, rootkit authors have devised new ways of subverting the operating system and even the kernel in attempts to remain undetected.

The rootkit threat is significant; there is no question about that. Rootkits can hide the presence of other tools, such as keyloggers, network sniffers, and remote access backdoors, not only from the user but also from the operating system. The insidious nature of rootkits can cause issues when they are actually present as well as when they aren’t but incident responders assume that they have been, due to lack of knowledge and training. Assuming (without any hard-core data to back it up) that a rootkit has been installed on a system or infrastructure can lead an investigator or incident manager down an incorrect path with regard to reactions and decisions based on the misleading incident assessment. Considerable resources could be invested in unnecessary activities, or systems could be wiped of all data and reinstalled from clean media, all without determining the root cause, and then become reinfected soon after being put back into service.


So, just what is a rootkit? A Sophos podcast ( released on August 24, 2006, includes the statement that as a result of a poll conducted by Sophos, 37 percent of respondents did not know the definition of a rootkit. Wikipedia defines a rootkit ( as "a set of software tools intended to conceal running processes, files, or system data from the operating system." In the first part of their three-part series of articles on rootkits published on SecurityFocus, Jamie Butler, a widely regarded expert in rootkit technologies, and Sherri Sparks define a rootkit as follows (

A program or set of programs that an intruder uses to hide her presence on a computer system and to allow access to the computer system in the future. To accomplish its goal, a rootkit will alter the execution flow of the operating system or manipulate the data set that the operating system relies upon for auditing and bookkeeping.

Another way of looking at it is that a rootkit is a software program that modifies the operating system so that it is capable of hiding itself and other objects from users, administrators, and even the operating system.

Rootkits are used to hide processes, network connections, Registry keys, files, and the like from the operating system and, by extension, the administrator. The term rootkit comes from the UNIX world, where such tools were often used to gain and/or maintain "root" (akin to the Administrator on Windows) level access to a system. As similar functionality was developed in malware on Windows, the name made a similar transition along with the tools.

One of the first rootkits developed for Windows was NTRootkit, written by Greg Hoglund and released in 1999. NTRootkit consists of a driver and is still available with source code, as illustrated in Figure 7.1.

Figure 7.1 NTRootkit 0.44 Archive Showing Source Files

 NTRootkit 0.44 Archive Showing Source Files

Since then, significant research has gone into the development of rootkits and rootkit technologies. Hoglund and others have conducted classes at the BlackHat Security Conference and other conferences on how to write rootkits, and his Web site,, has become the preeminent site for Windows rootkit knowledge, development, and information sharing. Over the years, other rootkits have appeared on the scene, and development of new rootkit techniques continues unabated. There is also a great topic available on rootkits, the way they are designed, and the way they work: Rootkits: Subverting the Windows Kernel. This topic was written by Greg Hoglund and Jamie Butler and is available on

Immediately following the RSA Conference in February 2005, there was an explosion in interest in rootkits and rootkit detection, and as detection techniques became more sophisticated, so did the rootkits. Think of the trend as an ever-escalating battleground, with developments on one side spurring further developments on the other.

There are several different types of rootkits. Early versions of rootkits worked by replacing operating system utilities and applications with Trojan’ed versions so that when the Trojan’ed version of the utility was run, it was programmed not to show specific objects. For example,

Trojan’ing the netstat command would first remove the attacker’s network connections from the file listing and then display the remaining network connections as they would normally appear.

Later came the DLL injection or "user-mode" rootkits. These rootkits install in the security context of the user currently logged into the system and replace, hook, or patch various operating system calls or DLL functions. To put our netstat example in the context of a user-mode rootkit, rather than replacing the netstat command itself, a user-mode rootkit will hook Windows API function calls so that the functions themselves do not return a complete listing of all network connections. The netstat command then proceeds to display all the information it receives from the function call, not knowing that it has been given incomplete and misleading information. Hooking the listed function calls also hides the network connections from any other programs that use the same API functions. User-mode rootkits that hide files will hook the FindFirstFile() and FindNextFile() function calls so that no program that uses these function calls, including the shell (i.e., Windows Explorer), will see the files that the rootkit is hiding.

Examples of user-mode rootkits include, but are not limited to, the following:

■ AFX Rootkit 2005 is an open source rootkit written in Delphi (by Aphex) that uses DLL injection and API hooking to hide files, Registry keys, processes, and the like.

■ Hacker Defender (from, by holy_father) was perhaps the most popular and widespread rootkit available. The F-Secure site describes Hacker Defender as the most widely deployed rootkit in the world. Hacker Defender also uses port redirection so that traditional means of rootkit detection, such as remote port scans, cannot detect the backdoor implemented by the rootkit. Hacker Defender uses a configuration file, which can be found in the contents of physical memory collected from an infected system. Portions of the configuration file can be found in physical memory; it can’t recover the file as a whole. Furthermore, an examination of physical memory sees right through Hacker Defender; the examiner can see all the processes it has been hiding. The examiner has to compare the running processes found during memory analysis to the list presented by the operating system to know which ones were being hidden by the rootkit.

■ NTIllusion ( was designed to be able to infect a system, running under the lowest privileges available, subverting processes owned by the current user.

■ Vanquish is a Romanian DLL injection rootkit that can hide files, processes, Registry keys, and the like. Vanquish consists of an autoloader (.exe file) and a DLL, which in turn consists of six submodules. Vanquish requires Administrator privileges to install properly and, according to the readme file that accompanies the distribution, does not work when other rootkits are present on the system.

■ Gromozon ( is a user-mode rootkit that infects a system via a browser helper object (BHO) and uses multiple techniques to maintain persistence on the infected system (hides code in EFS files and NTFS alternate data streams, creates a service, creates a reference in the AppInit_DLLs Registry key, and the like). In addition, the rootkit removes the Debug privilege from user accounts to inhibit rootkit detection tools from functioning properly. The Symantec write-up ( on this rootkit describes it as "spaghetti" due to the various methods of persistence that the authors designed into the code.


Many rootkits are available for download online, and in many cases at

Much more insidious are the "kernel-mode" rootkits because they subvert the operating system kernel itself. Not only will kernel-mode rootkits intercept low-level API calls but also they will manipulate kernel data structures. One example of a kernel-mode rootkit is FU, developed by Jamie Butler, which uses a technique called direct kernel object manipulation (DKOM) to hide on the system. DKOM is the process of manipulating kernel-level data structures without using the Windows APIs. For example, the Windows kernel maintains a doubly linked circular list of all running processes on the system, and FU will remove requested processes from the list. The processes are still there but are not "seen" by the kernel. The scheduling quantum for the system is a thread, not a process, so the FU thread continues to run while the process is invisible to the system. FU uses a driver, named msdirectx.sys by default, to gain access and control the system. The FU program, fu.exe, terminates after it loads the driver into memory.

Kernel-mode rootkits may also subvert other kernel structures. The FUTo ( rootkit, released as the successor to the FU rootkit, is discussed at great length in volume 3 of the Uniformed Journal (, released in January 2006). FUTo extends FU’s DKOM capabilities by using assembly language code (rather than API calls) to manipulate the PspCidTable variable, which is a pointer to the handle table for process and thread client IDs. This handle table is used to keep track of all process identifiers.

Shadow Walker is a proof-of-concept kernel-mode rootkit that was discussed at the BlackHat 2005 conference. Based on the FU rootkit, Shadow Walker contains an additional driver that manipulates the memory manager to hide the existence of the rootkit files. Shadow Walker does this, in short, by ensuring that all hidden pages are in nonpaged memory and by intercepting all accesses to those pages. When the operating system requests to read those pages, the rootkit returns pages of zeros. When the operating system requests to execute those pages, it returns the malicious code. Remember the scene from the Star Wars movie in which Obi-Wan Kenobi told the StormTrooper commander, "These are not the droids you are looking for"? Yeah, just like that.

One caveat to kernel-mode rootkits is that they can also cause the system to "blue screen" if they are not properly written. Microsoft support personnel have helped many customers track down repeated BSoDs (the dreaded Blue Screen of Death), only to find that a kernel-mode rootkit was the issue. As mentioned in next topic, a crash dump or BSoD will cause a crash dump file to be written to the hard drive, and support personnel can use this file to diagnose the issue. Often, this is the way a rootkit (one that is known or perhaps a new variant) is discovered on a system.

Occasionally, the term rootkit is used in a somewhat lazy manner. For example, there is an interesting entry in the Symantec Security Response Web log from September 2006 titled "The Poor Man’s Rootkit." In that entry, the author describes a bit of malware named Trojan.Zonebac that uses a camouflage technique to "hide" its presence on the system. In short, during installation the Trojan scans the contents of the ubiquitous Run key and selects a commonly used application. It backs up the executable image for the file pointed to by the Registry value and writes itself to the file system using the name of the original file. When the system is started, the Trojan is run automatically, and it then runs the backed-up file as well, so nothing appears amiss. Furthermore, the LastWrite time of the Run key is not updated since no actual changes were made to the key.

Although this is indeed a novel and even ingenious method for hiding on a system, it is not a rootkit. In fact, hiding in plain sight by renaming the malware executable image to something innocuous is a common and effective practice. It is not uncommon for this sort of technique to be listed under "hack the admin" or "hack the examiner" rather than "hack the server."


Relying on nothing more than the name of a file to diagnose an issue can be misleading and could even cause an investigator to completely miss the true root cause of the incident. Too often, an administrator will find a suspicious file and Google the filename. He’ll then find that there is a legitimate Microsoft file by that name and so declare the incident closed. This does not apply only to administrators; I have seen malware analysts do the same thing. However, I have also seen instances in which malware was installed on a system using the name of a legitimate Microsoft file, such as alg.exe or svchost.exe. In most of the cases that I have been involved with, the administrator has found this "legitimacy" and looked no further. No one noticed that the executable images were not located in the system32 directory, for example. The point is that you cannot rely solely on the filename as a means of identifying a file and the effect it might have on a system or infrastructure.

Rootkits have also been used commercially. Not only have several rootkit authors branched out to provide custom rootkits to whomever was willing to pay their fees but also corporations have used rootkits to hide functionality. On October 31, 2005, Mark Russinovich (of SysInternals fame, now with Microsoft) announced on his blog that he’d discovered that Sony Corporation was using a rootkit in an effort to affect digital rights management and protect its property ( Among other things, Mark pointed out that not only was the use of this rootkit completely unknown to the person who had purchased the music CD and installed the software on her computer (users were not explicitly warned of the use of the rootkit, nor was it listed in the end-user license agreement) but also an attacker who did find this software installed on a system could take advantage of it and install his own tools, which would then be hidden under Sony’s umbrella. Since the discovery of this issue and the ensuing furor, Mark has moved on to be employed by Microsoft. Mark’s blog entry is archived at the Virus Bulletin site.

Notes from the Underground…

Information Sharing

In his blog entry regarding the Sony rootkit issue (at the Virus Bulletin site), Mark makes the following statement:

Until a few years ago we made the source code to Regmon available publicly, which led to the use of our hooking functions and support routines in the NTRootkit example that’s published on structure of the code in Aries indicates that it’s likely to be derived from NTRootkit code.

It’s interesting to see how different sources are used to further the development of applications, including malware. In this case, the use of the hooking functions has come full circle.

Mark and others explored the use of rootkits by corporations in their software products, and on January 10, 2006, Symantec ( released information stating that its Norton Protected Recycle Bin uses rootkit-like functionality as well.

Rootkit Detection

So now that we’ve seen a little something about what rootkits are and what they can do, how do we go about detecting the presence of a rootkit on a system? To answer this question, let’s look at two detection modes, live and postmortem. In live-detection mode, the basic scenario is that we’ve got a running system and we’re going to attempt to determine whether there is a rootkit on the system. In postmortem detection mode, we’re working with an acquired image of the system. One of the benefits of employing both detection modes is that effective rootkit detection really requires a combination of various techniques discussed in other topics of this topic, such as memory analysis, Registry and file system analysis, port scans, and the analysis of network traffic captures. All of these techniques combine to develop as complete a picture as possible to assist you in detecting rootkits.

Live Detection

Live detection of rootkits can be a tricky issue to deal with, particularly if the investigator is not knowledgeable about rootkit artifacts and what to look for on a system that might be infected with a rootkit. Often this results in a misdiagnosis and misidentification of the incident, and any further response is taken in the wrong direction.


An important caveat to running any tool that performs malware detection is that you do not want it to automatically delete files or other artifacts; all you want the tool to do is detect the presence of the malware and alert you. Deleting artifacts defeats the purpose of your analysis.

In fall 2006, Jesse Kornblum published a very interesting paper in the International Journal of Digital Evidence titled "Exploiting the Rootkit Paradox with Windows Memory Analysis" ( In this paper, Jesse identified two basic principles that all rootkits attempt to follow; that is, they want to remain hidden, and they want to run. Essentially, to remain hidden on a system, a rootkit has to minimize its footprint while still interacting with the system in some way. The system itself, specifically the operating system, needs to be able to execute the rootkit, which is trying to remain hidden and persistent across reboots. Therefore, Jesse proposes that if the operating system can find the rootkit, so can an examiner. I might add "a sufficiently knowledgeable examiner" to that statement, but I’m sure that’s what Jesse meant to say.

The predominant technique for rootkit detection on a live system is sometimes referred to as behavioral or differential (or high/low) analysis. The basic idea is that by making two different kinds of queries for the same information and looking for differences in the responses, you can detect the presence of a rootkit or of something being hidden by a rootkit. For example, one of the early rootkit detection tools was a Visual Basic script named rkdetect.vbs (which is still available at that could detect the popular Hacker Defender rootkit by running a remote query to enumerate services using sc.exe, followed by a local query (using psexec.exe and sc.exe), and then looking for anomalies or differences between the two outputs.The script would note differences in output between remote and "local" queries (again, the tools were run locally on a remote system using psexec.exe), but it also included some signature checks—that is, specific checks for specific rootkits.


Lenny Zeltser, an incident handler with the SANS Internet Storm Center (ISC), posted a diary entry ( titled "Behavior Analysis of Rootkit Malware" on July 16, 2006. In that diary entry, Lenny provides screen captures and descriptions of several rootkit detection tools (as well as links to others) being tested against some of the rootkits mentioned previously in this topic.

Over time, rootkits have evolved, using more sophisticated hiding and stealth techniques, and rootkit detection techniques have had to keep up. Differential analysis is still the best approach to detecting rootkits, but the items queried are even more granular. For example, some tools will scan the file system using commands similar to dir /s and dir /s /ah and then compare their output to the contents of the Master File Table (MFT). The idea is to perform a high-level query followed by a very low-level query (as low as possible) and note any differences in the output between the two.

Several freeware and commercial rootkit detection tools are available, yet none of them provide details of how they operate. This is done so that the rootkit authors do not have an easy means of determining how the detection tools function and can then add techniques to their rootkits to avoid detection by those tools. However, this does not deter the rootkit authors from downloading the tools and determining how they work for themselves.


RootkitRevealer (, illustrated in Figure 7.2) is a rootkit detection tool that appeared on the scene in spring 2005. (RootkitRevealer was "slashdotted" at—that is, posted to and discussed on the Web site—on February 23, 2005.) Since the initial release, it has gone through a number of changes to keep up with changes in rootkit techniques. Whenever rootkits are developed that include new techniques to avoid detection, tools such as RootkitRevealer quickly incorporate counter-antidetection techniques.

The author’s description of RootkitRevealer specifically states that although the tool is designed to and can detect rootkits that hide files and Registry keys, it does not detect kernel-mode rootkits (such as FU) that modify kernel objects.


When running any tool, you need to be aware of how it works and what it does; this applies to rootkit detection tools as well as any other tool. I worked an engagement once where the customer’s incident response was badly managed and uncoordinated. While some administrators were instructed to do specific tasks, several took it upon themselves to run destructive antivirus scans on systems as well as run RootkitRevealer. To perform its scans, RootkitRevealer installed itself as a service, and the executable image had a random name, although the image file itself had random padding (so that the hash of the file was never the same). This was done as an antirootkit detection technique. At one point, an administrator called me to tell me that he had discovered a "massively infected" system that had eight strange services running, and RootkitRevealer had not detected them as rootkits. Well, first of all, if the administrator could "see" the services listed, they probably weren’t hidden by a rootkit. Second, all the executable image files had the same icon. Third, all the executable image files were RootkitRevealer. Due to a lack of coordination and knowledge of the tools being used, incident-response activities resulted in what appeared at first glance to be a massive infection.

Figure 7.2 RootkitRevealer GUI

RootkitRevealer GUI


GMER ( is a freeware GUI-based rootkit detection application that attempts to detect:

■ Hidden processes, files, services, Registry keys, and drivers

■ Drivers hooking the system service descriptor table (SSDT), interrupt descriptor table (IDT), or IO request packet (IRP) calls

GMER is also capable of showing NTFS alternate data streams, as illustrated in Figure 7.3.

Figure 7.3 GMER GUI


Also available from the GMER Web site is a small CLI application called catchme that is capable of detecting user-mode rootkits such as Gromozon, Hacker Defender, AFX, and Vanquish. The GMER site has the rootkit detection tool available for download as well as several videos of rootkits being detected and log files from scans where rootkits were detected. GMER also detects Master Boot Record (MBR) rootkits and allows the inclusion of your preferred antivirus application in the GMER user interface.

Next post:

Previous post: