In this blog post I will introduce a new Volatility Linux
plugin, tmpfs, and discuss its uses
and implementation. The purpose of this plugin, which can currently be found here, is to reconstruct any tmpfs
filesystem contained within a Linux memory capture and fully recover it to
disk.
What is tmpfs?
Tmpfs is a Linux filesystem whose
contents reside only in memory. This means that files and directories inside of
tmpfs mounts are never written to the local disk and that once a tmpfs mount is
unmounted, the entire filesystem is “gone”.
Why is tmpfs interesting?
Tmpfs is interesting from a forensics perspective for a few
reasons. The first is that, in a traditional forensics scenario, the
investigator expects that he can shut a computer off, images its disk(s), and
get back the filesystem at the time of when the computer was running. With
tmpfs, this is obviously not true…
The second reason that tmpfs is interesting is because of
how it is used by Linux distributions as well as by attackers and certain
malware samples. On many Linux
distributions, the /tmp directory is
mounted as tmpfs so that the directory will be cleared on reboot. This helps
speedup the boot process as the init code does not need to unlink every file
and directory under /tmp, but it also means that the traditional (disk only) preservation
process will miss any files stored in /tmp.
This directory often contains scratch data and install files for many
applications as well as for rootkits/malware that try to be agnostic to the
distribution they are being installed on.
Linux also uses tmpfs to
implement shared memory through /dev/shm.
While recovery of this directory may help in recovering IPC data, its main
purpose related to forensics & IR is that it is often used as a scratch
directory by attackers to download files, compile programs, and to store the
output of commands and malware hooks.
Since the filesystem goes away on reboot, attackers know they can
frustrate and/or defeat traditional forensics investigations using this method.
The third reason that tmpfs is of interest is that Linux
live CDs use tmpfs as the underlying store for all changes made to the
filesystem after the machine boots, since obviously it cannot write back to the
CD. If you are interested in reading about memory analysis of live CDs, I did a
presentation and white paper on it at Blackhat DC 2011 ( slides | paper ) so I will not
recap it all here. The paper also
discusses much of the kernel internals described at the end of this blog post.
Finally, while still being researched, it has been
discovered that many Android applications use a tmpfs instance created by the
Android runtime to store “interesting” runtime data and state. The tmpfs plugin has already been tested on a
number of phones using Volatility’s Android support and the data was verified
to be fully and correctly recovered on all devices. For the latest details on this, please
contact Joe Sylve (@jtsylve).
How do you use the plugin?
The plugin requires two steps to work. First, you run it
with the ‘-L’ option to list the tmpfs filesystems within the image:
python vol.py --profile=Linuxthisx86 -f /root/lime-tmpfs linux_tmpfs –L
The output of this invocation on my test VM looks like this:
Volatile Systems Volatility Framework 2.1_rc31 -> /lib/init/rw2 -> /dev/shm
Now, let us say I want to recover the contents of /dev/shm,
so I then run the plugin with the –S option of ‘2’ and an output directory:
python vol.py --profile=Linuxthisx86 -f /root/lime-tmpfs linux_tmpfs -S 2 -o outputdir/
When this command is finished, the entire filesystem of
“/dev/shm”, including all files, directories, and metadata (atime, mtime,
permissions), will be replicated to the “outputdir” directory. In a real forensics investigation, “outputdir”
should be on an external drive or a separate mount point that can be
remounted read-only after the filesystem is recovered.
Limitations
When recovering the filesystem from memory to a disk there
are a few limitations that investigators should be aware of:
1) While the script does preserve the modified and accessed
times of the recovered files and directories, it has no method in which to
preserve the create times. This is because the Python os.utime function only supports modifying the accessed and modified
times. In order to change the create times on a standard Linux filesystem, the
script would need to interact with something along the lines of debugfs,
which can be error-prone and only supports the Ext family of filesystems. Similar burdens are faced on Windows and Mac
filesystems as well.
The result of this limitation is that the create time of all
entries in the filesystem will actually be the time that the Volatility plugin
created them. If enough interest is shown in the plugin and this limitation, I
can easily extend the plugin to support writing out a mactime file with all the
correct information.
2) The owner/group information is not preserved for
recovered files. This may be added to future versions of the script, at least
if the output filesystem is Linux based.
3) There is currently no support for recovering deleted
files within tmpfs stores.
How does the plugin work?
Now that we have explained the purpose of the plugin, it is
time to dig into some Linux kernel internal in order to explain the plugin's
operation. If you are not a programmer or not interested in kernel internals,
you can freely skip to the end of the post.
For the plugin to work it needs to be able to the following:
1) Find and list all tmpfs mount points in the memory image
2) Recursively enumerate and reconstruct the entire
filesystem for every tmpfs instance
3) Record the needed metadata of each file and directory and
update the recovered files on disk to match it
4) For every file found, recover the file contents and write
them to disk
1) Finding all tmpfs mount points
The tmpfs mount points are found by leveraging the
capabilities of the mount plugin (volatility/plugins/linux/mount.py).
The mount plugin works by walking the mount_hashtable
hash table, which stores a vfsmount structure for every mount
point. This is the same operation the kernel does at runtime to populate the /proc/mounts file. The tmpfs
plugin then filters the vfsmount
structures to only those representing tmpfs mounts. This is all that is needed to implement the
‘-L’ option of the plugin.
2) Recurse the filesystem
Each vfsmount
structure contains a pointer to the root directory entry of the filesystem, of
type dentry, in its mnt_root member. dentry structures are generic structures used to represent files,
directories, pipes, sockets, and all other file types within the kernel. They
have no 1:1 mapping on disk and there can be multiple dentry structures per inode.
Once the root dentry structure
is obtained, we can then recursively traverse the entire filesystem by walking
each dentry’s d_u.d_child member. For each entry in the child list, we check if
it’s a directory or a file. If it’s a directory, the plugin simply creates the
directory in the output directory’s hierarchy and then recursively processes
it. If it’s a file, then we need to recover its contents from memory and create
the file in the output hierarchy. For
both directories and files, we need to update the metadata of the newly created
files to match those in memory. This process effectively recovers the entire
filesystem to disk in the exact order and layout as in memory.
3) Recovering metadata
Recovering metadata is a simple process because all the
information is directly stored within the inode
structure of the file or directory, which is stored in the d_inode member of the dentry structure. This line of code from the plugin accesses
all the metadata from the inode that
is currently recovered:
(perms, size, atime, mtime) = (inode.i_mode, inode.i_size, inode.i_atime, inode.i_mtime)
4) Recovering
file contents
In
order to recover a particular file’s contents, we need to be able to locate the
structures that track its memory pages within the kernel’s page cache and then
find the actual physical pages that store the contents.
The
page cache is a performance enhancing
mechanism used by the kernel to cache physical pages in RAM so that they do not
have to be read from the raw device or network device on which they are stored each
time they are accessed. For example, on an active SSH server, a number of files
(passwd, shadow, sshd_config, /bin/bash, etc) are going to be constantly read
as users log in and off the system. If the kernel had to fully read in these
files from disk to memory each time a user logged in, performance would be
unacceptably bad. To alleviate this,
pages that are read in from disk are then put in the page cache, so the next
time an applications needs to be executed or reads a file, the kernel can simply
return the page from the cache without having to touch the disk. An aside: If
you have ever dd’ed a disk image in Linux and watched ‘top’, you may have noticed
that your entire RAM contents was being used inside the kernel. Much of that
was the kernel reading ahead on the disk being imaged and filling the page
cache with the next blocks to be processed by dd.
Since
the filesystem type we want to recover (tmpfs) is stored only in memory, it is
implemented so that all its pages are always stored within the page cache. This
lets the filesystem work as all others do, and the kernel will never look for a
disk or other source to read from because the files will always be up-to-date
within the cache.
In order to locate the
pages of a particular file, we need to use its index into the page cache. This
is stored within the dentry.d_inode.i_mapping
member which is of type address_space. address_space
structures are used to track sets of sparse physical pages that together
form a contiguous data unit (in this case, a file within a tmpfs mount). To access the pages of a file inside its address_space, we need to walk the radix
tree of struct pages stored within
the address_space’s page_tree member. A struct page tracks a physical page and lets the kernel determine
the physical address of the page’s contents.
To
find the correct index in the tree for a page, we simply divide the offset of the
page into the file by page size (4k in this case). We do this in a loop for the
entire file in order to recover each page sequentially. I will save the reader the pain of having to
understand the kernel’s radix tree, but the interested reader can look at the radix_tree_lookup_slot function in the
tmpfs plugin. The page cache and its storage format is also very nicely explained in Understanding the Linux Kernel 3rd Edition. Once we are able to
walk the page_tree of a particular
file and recover each index (a struct
page), we then need to determine the physical address of the page it
tracks.
Finding
the physical address is performed by indexing the struct page into the mem_map array.
This array holds a struct page for every physical page of memory. Indexing into mem_map will give us the page number of the physical page, which we can then shift
left by PAGE_SHIFT (12) in order to get the page’s physical offset. This process roughly
corresponds to the page_to_pfn macro
within the kernel.
Once
we have the physical offset, all we have to do is read it using Volatility’s
API and then concatenate each page’s contents together into the file. At this
point we will have recovered the entire filesystem along with its contents and
metadata.
Conclusion
This blog post has showcased a new and exciting Volatility
plugin. Being able to fully recover tmpfs filesystems out of memory greatly
adds to the depth of Volatility’s Linux support for both traditional Linux
installs as well as Android devices. By
incorporating this plugin into your forensics and incident response processes,
you will recover information that until now has large been ignored or missing.
Please check back frequently as I will be show casing more
plugins leading up to the release of Volatility 2.2 at the OMFW Conference. If you are interested in the latest research in memory forensics, I highly
suggest you register for and attend OMFW as many of the best memory forensics
researchers will be presenting and attending.
If you have any questions or comments about this post, please Email me, catch me on Twitter (@attrc), or reply in the comments section.
No comments:
Post a Comment