Introduction
In last topic, we discussed collecting volatile data from a live, running Windows system. From the order of volatility listed in RFC 3227, we saw that one of the first items of volatile data that should be collected during live-response activities is the contents of physical memory, commonly referred to as RAM. Although the specifics of collecting particular parts of volatile memory, such as network connections or running processes, have been known for some time and discussed pretty extensively, the issue of collecting, parsing, and analyzing the entire contents of physical memory is a relatively new endeavor, even today. This field of research has really opened up in the past several years, beginning in summer 2005, at least from a public perspective.
The most important question that needs to be answered at this point is "Why?" Why would you want to collect the contents of RAM? How is doing this useful, how is it important, and what would you miss if you didn’t collect and analyze the contents of RAM? Until now, some investigators have collected the contents of RAM in the hope of finding something they wouldn’t find on the hard drive during a postmortem analysis— specifically, passwords. Programs will prompt the user for a password, and if the dialog box has disappeared from view, the most likely place to find that password is in memory. Malware analysts will look to memory in dealing with encrypted or obfuscated malware, because when the malware is launched, it will be decrypted in memory. More and more, malware is obfuscated in such a way that static, offline analysis is extremely difficult at best. However, if the malware were allowed to execute, it would exist in memory in a decrypted state, making it easier to analyze what the malware does. Finally, rootkits will hide processes, files, Registry keys, and even network connections from view by the tools we usually use to enumerate these items, but by analyzing the contents of RAM we can find what’s been hidden. We can also find information about processes that have since exited.
In 2008, Greg Hoglund (perhaps best known for writing the first viable rootkit for Windows systems and rootkit.com, but also as the CEO of HBGary, Inc.) wrote a short paper titled "The Value of Physical Memory Analysis for Incident Response" (www.hbgary.com/resources.html). In that paper, Greg describes what can be extracted from the contents of physical memory, and perhaps most importantly, the value of that information with respect to addressing issues in incident response and computer forensic analysis.
A Brief History
In the past, the "analysis" of physical memory dumps has consisted of running strings or grep against the "image" file, looking for passwords, Internet Protocol (IP) addresses, e-mail addresses, or other strings that could give the analyst an investigative lead. The drawback of this method of "analysis" is that it is difficult to tie the information you find to a distinct process. Was the IP address that was discovered part of the case, or was it actually used by some other process? How about that word that looks like a password? Is it the password an attacker uses to access a Trojan on the system, or is it part of an instant messaging (IM) conversation?
Being able to perform some kind of analysis of a dump of physical memory has been high on the wish lists of many within the forensic community for some time. Others (such as myself) have recognized the need for easily accessible tools and frameworks for retrieving physical memory dumps and analyzing their contents.
In summer 2005, the Digital Forensic Research Workshop (DFRWS; www.dfrws.org) issued a "memory analysis challenge" in order "to motivate discourse, research, and tool development" in this area. Anyone was invited to download the two files containing dumps of physical memory (the dumps were obtained using a modified copy of dd.exe available on the Helix 1.6 distribution) and answer questions based on the scenario provided at the Web site. Chris Betz and the duo of George M. Garner, Jr., and Robert-Jan Mora were selected as the joint winners of the challenge, providing excellent write-ups illustrating their methodologies and displaying the results of the tools they developed. Unfortunately, these tools were not made publicly available.
In the year following the challenge, others continued this research or conducted their own, following their own avenues. Andreas Schuster (http://computer.forensikblog.de/en/topics/windows/memory_analysis) began releasing portions of his research on the English version of his blog, together with the format of the EProcess and EThread structures from various versions of Windows, including Windows 2000 and XP. Joe Stewart posted a Perl script called pmodump.pl as part of the Truman Project (www.secureworks.com/research/tools/truman.html), which allows you to extract the memory used by a process from a dump of memory (important for malware analysis). Mariusz Burdach released information regarding memory analysis (initially for Linux systems but then later specifically for Windows systems) to include a presentation at the BlackHat Federal 2006 conference. Jesse Kornblum has offered several insights in the area of memory analysis to include determining the original operating system from the contents of the memory dump. During summer 2006, Tim Vidas, (http://nucia.unomaha.edu/tvidas/), a senior research fellow at Nebraska University, released procloc.pl, a Perl script to locate processes in RAM dumps as well as crash dumps.
Since then, the field of study with respect to collecting and analyzing memory dumps has grown by leaps and bounds, and in many instances the key figures have risen to the top. Perhaps the most notable individual in the field of memory analysis is Aaron Walters, co-creator of the FATKit (http://4tphi.net/fatkit/) and the Volatility Framework (https://www. volatilesystems.com/default/volatility). Although tools have been released to allow for collecting the contents of physical memory from Windows XP and Vista systems (addressed in detail in this topic), Aaron and his co-developer, Nick L. Petroni Jr., have focused primarily on providing a framework for analysis of memory dumps. Aaron and Nick have been assisted by Brendan Dolan-Gavitt (you can find Brendan’s blog at http://moyix.blog-spot.com/, and his graduate research page is www.cc.gatech.edu/~brendan/) and others who have made significant contributions to the area of memory analysis and specifically to the Volatility Framework. Matthieu Suiche (www.msuiche.net/) has contributed a program for dumping the contents of physical memory, and has also published information and tools for parsing Windows hibernation files (which have been incorporated into the Volatility Framework). Andreas Schuster (http://computer.forensikblog.de/en/) has continued his contributions to parsing of Windows memory dumps, including the release of a PTFinder (discussed later in this topic) for Vista in November 2008, which is included as part of the PTFinder collection of Perl scripts.
In addition to free, open source tools for parsing Windows memory dumps, the security company Mandiant (the company Web site is www.mandiant.com, and the company blog is http://blog.mandiant.com/) released its Memoryze memory collection and parsing tool, along with the Audit Viewer tool for better presentation of the results of Memoryze. In addition, the folks at HBGary (www.hbgary.com/) have released their own memory analysis application called Responder, which comes in Professional and Field editions. The HBGary Web site includes a wealth of information about the tools, including videos demonstrating how to use them.
Collecting Process Memory
During an investigation, you might be interested in only particular processes rather than a list of all processes, and you’d like more than just the contents of process memory available in a RAM dump file. For example, you might have quickly identified processes of interest that required no additional extensive investigation. There are ways to collect all the memory used by a process—not just what is in physical memory, but also what is in virtual memory or the page file.
To do this, a couple of tools are available. One is pmdump.exe (www.ntsecurity.nu/ toolbox/pmdump/), written by Arne Vidstrom and available from NTSecurity.nu. However, as the NTSecurity.nu Web site states, pmdump.exe allows you to dump the contents of process memory without stopping the process. As we discussed earlier, this allows the process to continue and the contents of memory to change while being written to a file, thereby creating a "smear" of process memory. Also, pmdump.exe does not create an output file that can be analyzed with the Microsoft Debugging Tools.
Tobias Klein has come up with another method for dumping the contents of process memory in the form of a free (albeit not open source) tool called Process Dumper (available in both Linux and Windows versions from www.trapkit.de/research/forensic/pd/index.html). Process Dumper (pd.exe) dumps the entire process space along with additional metadata and the process environment to the console (STDOUT) so that the output can be redirected to a file or a socket.A review of the documentation that Tobias makes available for pd.exe provides no indication that the process is debugged, halted, or frozen prior to the dumping process. Tobias also provides the Memory Parser graphical user interface (GUI) utility for parsing the metadata and memory contents collected by the Process Dumper. These tools appear to be an extension of Tobias’s work toward extracting RSA private keys and certificates from process memory (www.trapkit.de/research/sslkeyfinder/index.html).
Tip::
Jeff Bryner wrote pdgmail (you can find the pdgmail tool at www.jeffbryner.com/code/pdgmail), a Python-based tool that uses the output of Tobias’s Process Dumper program to search for Gmail artifacts (e.g., contacts, last access records, account names, etc.). Jeff posted to the SANS Forensic blog (http://sansforensics.wordpress.com) regarding pdgmail, stating that he had not tested it on the Windows platform. Jeff subsequently released pdymail for extracting Yahoo! mail remnants (http://jeffbryner.com/code/pdymail) from process memory, as well. In this section of the topic, we discuss dumping the contents of process memory, whereas later in the topic we discuss how an analyst can dump the contents of process memory for a full dump of physical memory. You can use either method to retrieve data for use with pdgmail or pdymail.
Another tool that is available and recommended by a number of sources is userdump.exe, available from Microsoft. Userdump.exe will allow you to dump any process on the fly, without attaching a debugger and without terminating the process once the dump has been completed. Also, the dump file generated by userdump.exe can be read by the Microsoft Debugging Tools. However, userdump.exe requires that a driver be installed for it to work, and depending on the situation, this might not be something you’d want to do.
Based on conversations with Robert Hensing, formerly of the Microsoft PSS Security team, the preferred method of dumping memory used by a process is to use the adplus.vbs script that ships with the debugging tools, as this methodology attaches a debugger to the process and suspends the process to dump it. Adplus stands for Autodumplus and was originally written by Robert (the documentation for the script states that versions 1 through 5 were written by Robert, and as of Version 6, Israel Burman has taken over). The help file (debugger.chm) for the MS Debugging Tools contains a great deal of information about the script as well as the cdb.exe debugging tool, which it uses to dump the processes you designate. The MS Debugging Tools do not require that an additional driver be installed, and they can be run from a CD. This means the tools (adplus.vbs and cdb.exe, as well as supporting dynamic link libraries, or DLLs) can be written to a CD (adplus.vbs uses the Windows scripting host Version 5.6, also known as cscript.exe, which comes installed on most systems) and used to dump processes to a shared drive or to a USB-connected storage device. Once the dumps have been completed, you can use the freely available MS Debugging Tools to analyze the dump files. In addition, you can use other tools, such as BinText, to extract ASCII, Unicode, and resource strings from the dump file. Also, after dumping the process, you can use still other tools (such as those discussed in next topic) to collect additional information about the process from the running system, which may provide a quicker view into the details of the process itself. Handle.exe (which is available from http://technet.microsoft.com/en-us/sysinternals/bb896655.aspx and requires that you have Administrator rights on the system when running it) will provide you with a list of handles (to files, directories, etc.) that have been opened by the process, and listdlls.exe (Version 2.25 at the time of this writing, available from http://technet.microsoft.com/en-us/sysinternals/bb896656.aspx) will show you the full path to and the version numbers of the various modules loaded by a process.
Extensive help is available for using adplus.vbs, not only in the MS Debugging Tools help file but also in Microsoft Knowledge Base article 286350 (http://support.microsoft.com/kb/286350). You can use adplus.vbs to hang the process while it is being dumped (i.e., halt it, dump it, and then resume the process) or to crash the process (halt the process, dump it, and then terminate). To run adplus.vbs in hang mode against a process, you would use the following command line:
This command will create a series of files within the debug directory within a subdirectory prefaced with the name Hang_mode_ that includes the date and time of the dump. (You can change the location where the output is written using the —o switch.) You will see an adplus.vbs report file, the dump file for the process (multiple processes can be designated using multiple —p entries), a process list (generated by default using tlist.exe; you can turn this off using the -noTList switch), and a text file showing all the loaded modules (DLLs) used by the process.
Although all the information collected about processes using adplus.vbs can be extremely useful during an investigation, you can use this tool only on processes that are visible via the application program interface (API). If a process is not visible (say, if it’s hidden by a rootkit), you cannot use these tools to collect information about the process.
This command results in a 4012.dmp file that is 118,300,672 bytes in size.
Dumping Physical Memory
So, how do you go about collecting the full contents of physical memory, rather than just dumping a process? Several methods are available, each with strengths and weaknesses. The goal of this topic is to provide an understanding of the various options available as well as the technical aspects associated with each option. This way, as a first responder or investigator, you’ll make educated choices regarding which option is most suitable, taking the business needs of the client (or victim) into account along with infrastructure concerns.
DD
DD (http://en.wikipedia.org/wiki/Dd_(Unix)) is the short name given to a powerful tool from the UNIX world which has a variety of uses, not the least of which is to copy files or even entire hard drives. DD has long been considered a standard for producing forensic images, and most major forensic imaging/acquisition tools as well as analysis tools support the dd format. GMG Systems Inc. produced a modified version of dd that runs on Windows systems and can be used to dump the contents of physical memory from Windows 2000 and XP systems. This version of dd was part of the Forensic Acquisition Utilities, but is no longer available. This utility was able to collect the contents of physical memory by accessing the \Device\PhysicalMemory object from user mode. The following command line could be used to capture the contents of RAM in the file ram.img on a network share or a USB thumb drive attached to the system:
In the preceding command line, the block size (bs value) is set to 4 kilobytes (KB) or 4096 bytes, as this is the default size for pages in memory. Therefore, the command tells dd.exe to grab one page at a time. This version of dd.exe also allows compression and the generation of cryptographic hashes for the content. Due to the volatile nature of RAM (i.e., it continues to change throughout the process of dumping it), however, it is not advisable to hash it until it is written from the disk, simply because there is no advantage in doing so. If the user images memory twice, even with little delay, the contents of RAM and thus the subsequent hashes will be different. In this case, it is only worthwhile to employ a cryptographic hash to address the integrity of the collected memory dump.
Warning::
The version of dd.exe from George M. Garner, Jr., discussed earlier, is no longer available or supported, but there may be responders out there who still have a copy of the tool on a CD or hard drive someplace (I do!). Discussion of and reference to the tool is provided here for completeness, as well as to recognize George’s contributions to the field of memory acquisition and analysis.
Nigilant32
Other tools use a process similar to dd.exe to capture the contents of RAM. Nigilant32 (www.agilerm.net/publications_4.html), from Matt Shannon of Agile Risk Management, uses a graphical interface to allow the responder to acquire the contents of physical memory. Figure 3.1 illustrates a portion of the Nigilant32 GUI, showing the option for imaging physical memory.
Figure 3.1 Portion of the Nigilant32 GUI
Nigilant32 has the advantage of a GUI, which may be a friendlier tool for some responders to use. You can deploy Nigilant32 on a CD or USB thumb drive along with other tools, and use it to quickly collect the contents of physical memory (primarily from Windows XP systems). As with dd.exe and other tools, Nigilant32 requires local access to the system from which the responder wishes to dump memory. You also can deploy Nigilant32 prior to an actual incident occurring, and responders can either run the tool locally and dump physical memory to a thumb drive or network share, or access the system remotely (via applications such as VNC or Remote Desktop Client) and perform the memory dump.
ProDiscover
ProDiscover IR (Version 4.8 was used in writing this topic, and Version 5.0 was released in summer 2008) also allows the investigator to collect the contents of physical memory (as well as system BIOS) via a remote server applet that can be distributed to a system via removable storage media (CD, thumb drive, etc.) or via the network. Figure 3.2 illustrates the user interface for this capability.
Figure 3.2 Excerpt of the Capture Image Dialog Box from ProDiscover IR
KnTDD
The problem with using tools such as dd.exe or Nigilant32, however, is that as of Windows 2003 SP1, access to the DevicePhysicalMemory object has been restricted from user mode (http://technet.microsoft.com/en-us/library/cc787565.aspx). That is, only kernel drivers are allowed to access this object. As such, tools such as dd.exe, Nigilant32, and ProDiscover (as previously mentioned, the Incident Response edition of ProDiscover includes a servlet that will allow an analyst to access physical memory on a remote Windows XP system) will not allow you to collect the contents of physical memory from Windows 2003 SP1 and later systems, including Vista. To address this issue, George M. Garner, Jr. (a.k.a. GMG Systems) produced a new utility called KnTDD, which is part of the KnTTools set of utilities. According to the licensing for KnTTools, the utilities are available for private sale to law enforcement personnel and bona fide security professionals. KnTDD includes the following capabilities:
■ Able to acquire the contents of physical memory using multiple methods, including via the \Device\PhysicalMemory object
■ Runs on Microsoft operating systems from Windows 2000 through Vista, including AMD64 versions of the operating systems
■ Able to convert a raw memory "image" to Microsoft crash dump format so that the resultant data can be analyzed using the Microsoft Debugging Tools
■ Able to acquire to a local removable (USB, FireWire) storage device as well as via the network using Transmission Control Protocol/Internet Protocol (TCP/IP)
■ Designed specifically for forensic use, with audit logging and cryptographic integrity checks
The KnTTools Enterprise Edition includes the following capabilities:
■ Bulk encryption of output using X.509 certifications, AES-256 (default), and downgrading to 3DES on Windows 2000
■ Memory acquisition using a KnTDDSvc service
■ A remote deployment module that is able to deploy the KnTDDSvc service by either "pushing" it to a remote admin share or "pulling" it from a Web server over Secure Sockets Layer (SSL), with cryptographic verification of the binaries before they are executed
One of the aspects of using dd.exe, and tools like it, that you need to keep in mind is Locard’s Exchange Principle. To use these tools to collect the contents of RAM, they must be loaded into RAM as a running process. This means memory space is consumed and other processes may have pages written out to the page file.
Another aspect of these tools to keep in mind is that they do not freeze the state of the system, as occurs when a crash dump is generated. This means that while the tool is reading through the contents of RAM, as the thirtieth "page" is being read the eleventh page could change as the process using that page continues to run. The amount of time it ultimately takes to complete the dump depends on factors such as processor speed and rates of bus and disk I/O. The question then becomes, are these changes that occur in the limited amount of time enough to affect the results of your analysis?
Under most incident response conditions, tools such as dd.exe might be the best method for retrieving the contents of physical memory, particularly from Windows 2000 and XP systems. Such tools do not require that the system be taken down, nor do they restrict how and to where the contents of physical memory are written (e.g., using netcat, you can write the contents of RAM out over a socket to another system rather than to the local hard drive). Tools have been developed and made freely available (discussed later in this topic) to parse the contents of these RAM dumps to extract information about processes, network connections, and the like. Further, development of the KnTTools and other similar tools allows for continued support of this methodology beyond Windows 2003 SP1.
Note
The primary issue with using a methodology such as the Forensic Acquisition Utilities or KnTTools is that the system is still running when the contents of physical memory are retrieved. This means that not only are memory pages consumed simply by using the utilities (i.e., executable images are read and loaded into memory), but as the tool enumerates through memory, pages that have already been read can change. That is, the state of the system and its memory are not frozen in time, as would be the case with acquiring a forensic image of a hard drive via traditional "forensic" methodologies. Interacting with a live system, however, may be the only manner with which certain information can be retrieved. Keeping Locard’s Exchange Principle in mind, responders must thoroughly and clearly document their actions when performing live response. Issues such as the need for live response, as well as the "footprint" of live-response tools, have been (and will likely continue to be) the subject of discussion for some time.
Win32dd
Matthieu Suiche released his own tool for dumping the contents of physical memory, called win32dd (http://win32dd.msuiche.net/). Win32dd is described as a "free kernel land and 100% open-source tool"; this means that like mdd.exe, win32dd.exe is free, but unlike ManTech’s tool, win32dd.exe is open source. Win32dd.exe has some additional features, including the ability to create a WinDbg-compatible "crash dump" (similar to a Windows crash dump) which you can then analyze using the Microsoft Debugging Tools (www.microsoft.com/whdc/ DevTools/Debugging/default.mspx). As with mdd.exe and other CLI tools, win32dd.exe can be included in batch files.
Version 1.2.1.20090106 of win32dd.exe was available at the time this topic was being written. You can view the syntax information available for win32dd.exe using the following command:
As you can see, win32dd.exe has a number of options available for dumping the contents of physical memory. To create a raw, dd-style memory dump file, you can use the following command line:
Including the command-line options (such as -l 0) in the name of the output file serves as a modicum of additional documentation for the analyst, to identify the command-line switches used. As with other CLI tools, including such a command in a batch file is simple and straightforward (besides being self-documenting).
Memoryze
The consulting company Mandiant released its own memory collection and analysis tool, called Memoryze (www.mandiant.com/software/memoryze.htm). Memoryze finds its origins in the Mandiant Intelligent Response (MIR) product, and has been made freely available from the Mandiant Web site, along with examples of how to run the tool.
This will create a memory dump file in a directory structure within the current directory; running the command as is on my system created the directory structure Audits\WINTERMUTE\20090103003442\, and within that final directory it created several XML files and a memory image file. Running the batch file with the -output switch will let you configure the location for the dump file.
Also, when run on a live system, Memoryze reportedly "makes use of the page file(s)," incorporating memory contents from the page file into the collection process. This allows for more complete collection of data, as memory contents that have been swapped out to the page file are now available during the analysis process. In addition, Memoryze Version 1.3.0 (announced February 9, 2009) is capable of dumping memory that is accessible via F-Response, which we’ll discuss later in this topic.
