Disk Image Forensics Part II - PowerPoint Presentation

Disk Image Forensics Part II “When all else fails, we carve”

Overview In class problem 1 File Carving Deleted Files & Deleted File Recovery Using Sleuthkit to Recover Deleted Files

In Class Problem 1 During our investigation of Evil Corp’s host machines we came across an interesting flash drive but upon initial investigation it appears to be empty. We have imaged the USB for you. Available as In Class Problem 1 echo "f38257ae5124049acd83285aed8908b2aba838b1900b89ff697d5ab69cd8ee48 intern_task1.lzma" > temp; sha256sum -c temp intern_task1.lzma 2>/dev/null | grep OK; rm temp Verify the integrity with the second command. See if you can recover any information that could be valuable to our investigation. It is probably just a rabbit hole but you are just interns so you’ve got time. *Note: the secret, if there is one, is in the form of a CTF flag. You have 15 minutes to get the flag.

Carving Carving - Extraction of meaningful file content from otherwise unstructured streams of data. In many ways “it is a science and art unto itself.” (DFOST) Basic File Carving Process Search a data stream for file headers and magic values Determining or guessing the end of the file Saving this substream of data into a carved file Major Issue: False positives

Carving Tools Foremost Scalpel Binwalk Hachoir Framework tcpxtract(extract file from network traffic) Bulk Extractor (Extract specific kinds of data from a image file) Radare2 (Originally built for forensics but mostly used for RE now), FS specific tools, Type specific tools

File Carving Foremost Developed by Air Force Office of Special Investigations The program uses a configuration file to specify headers and footers to search for. Intended to be run on disk images, foremost can search through most any kind of data without worrying about the format. Last update May 2009 Maximum carved file size is 4GB Configuration stored in /etc/foremost.conf

File Carving Scalpel Originally based off of foremost but has since had considerable updates Enhancements if you download and install v2.0 https://github.com/sleuthkit/scalpel *Difficult to compile Claims it can handle files up to exabytes large Multithreaded, multiprocess capabilities can be run on a GPU Last updated 2014 Configuration stored in /etc/scalpel/scalpel.conf

File Carving Binwalk Fast easy to use tool for analyzing, reverse engineering, and extracting firmware images Uses libmagic #the same library the file command uses on linux Why might this be a disadvantage to a forensic examiner?

File Carving Binwalk Installation git clone https://github.com/ReFirmLabs/binwalk.git cd binwalk sudo pip install -U pylzma crypto sudo python setup.py install binwalk -h | grep ^Binwalk | cut -d ' ' -f2 //to check the version

File Carving Binwalk Useful options -ez extract and carve files (Easy right?) --dd {type:ext[:cmd]} Ex. binwalk --dd {zip archive:zip:unzip %e} #this will carve and unzip found zip files -Me #performs recursive extraction

File Carving Hachoir Framework Python binary file manipulation framework Several Sample Tools come with the framework hachoir-metadata #get metadata from binary files hachoir-strip #modify a file to remove metadata hachoir-grep #search for strings in a binary and is charset independent (Supposedly) Slow and might use a lot of memory hachoir-urwid #display the content of a binary file in text mode Installation sudo pip3 install -U urwid hachoir3

File Carving Disadvantages?

Deleted File and Deleted File Recovery Deleted Files (most recoverable)- files that have been unlinked, the filename entry is no longer presented when a user views a directory, and the filename, metadata structure, and data units are marked as “free”. However, the connection between these layers are still intact when forensic techniques are applied to the file system. Recovery consists of recording the relevant file name and metadata structures and then extracting the data units. Orphaned Files - similar to deleted files except the link between the file name and metadata structure is no longer accurate. Recovery of the data is still possible but there is no direct correlation from the file name to recovered data.

Deleted File and Deleted File Recovery Unallocated Files - files have had their once-allocated filename entry and associated metadata structure become unlinked and/or reused. In this cast the only means of recovery is carving the not-yet-reused data from the unallocated space of the volume Overwritten Files - have had one or more of their data units reallocated to another file. Full recovery is no longer possible, but partial recovery may depend on the extent of overwriting. Files with file names and/or metadata structures intact that have had some or all data units overwritten are sometimes referred to as Deleted/Overwritten or Deleted/Reallocated

Deleted File and Deleted File Recovery File Slack - the minimum space that can be allocated on a volume is a single block. If this is 4KB then a ASCII text file containing a single byte, the letter ‘a’, will consume an entire block. On a disk with 512 byte sectors this will consume 8 sectors. Where did the other bytes that do not contain the letter ‘a’ come from? … It Depends on the File System and OS but generally: The block to be used is marked as allocated and assigned to the file’s metadata structure The ‘a’ followed by 511 null bytes are place in the first sector The other seven sectors in this case are considered allocated but contain the data that was originally stored in that location. This space is called slack space.

Deleted File and Deleted File Recovery Historic Note on RAM Slack - While all modern operating systems pad the written sector with null bytes, this was not always the case. MS-DOS and older DOS-based versions of Microsoft Windows would pad the rest of the sector out with whatever contents of memory happened to be next to data being written. These data, between the end of allocated data and the beginning of previously allocated data, became known as RAM slack. Given this, RAM slack could potentially contain data that were never written to the disk, such as cryptographic keys or passphrases.

Sleuthkit Naming Convention Reminder “mm-”: tools that operate on volumes (aka “media management”) “fs-”: tools that operate on file system structures “blk-”: tools that operate at the data unit (or “block”) layer “i-”: tools that operate at the metadata (or “inode”) layer “f-”: tools that operate at the file name layer “j-”: tools that operate against file system journals “img-”: tools that operate against image files “-stat”: displays general information about the queried item “-ls”: lists the contents of the queried layer “-cat”: dumps/extracts the content of the queried layer

Sleuthkit File Recovery File carving is your last option for recovering lost files Carving an entire disk is time consuming You will get a lot of false positives if you do not narrow your search Following hypothesis will narrow your search Using sleuthkit can narrow down your search

Sleuthkit File Recovery Using sigfind, fsstat, and parted to recover a partition. If we have existing deleted partitions it would be helpful to be able to recover the partitions sigfind -o119-t fat disk.img -o #the sector offset you want to begin searching at -t #sigfind provides “templates” to search for. Many file system templates can be search for this way. fsstat -o 119 disk.img #get file system information -o #offset to look for file system metadata sudo parted disk.img --script mkpart primary fat32 119 -- -1s sudo mount -o loop,offset=$((119*512)),gid=$USER,uid=$USER disk.img ~/mnt To see which loop device mount used you could do sudo losetup --list

Sleuthkit File Recovery Using fls, icat, tsk_recover Once we have determined where the file system resides we can use these tools to recover data fls -p -l -r -o119 disk.img -l #list extended information such as creation time/modified time -r #recursively go through directories and list contents -o #offset of the beginning of the file system (in sectors) -d #display deleted entries only -p #display the full path of each entry

Sleuthkit File Recovery Using fls, icat, tsk_recover Once we have determined where the file system resides we can use these tools to recover data icat -o119 disk.img 7 #dump the contents of inode 7 -o offset of filesystem (in sectors) tsk_recover -o 119 disk.img ~/recovered Recover all deleted files from disk.img at offset 119 save them to ~/output directory -a #recover allocated files only -e #recover all file allocated and unallocated

Sleuthkit File Recovery Using Timeline Analysis to narrow down search fls -o 63 -f fat -m / -r images/disk.dd > body.txt fls -o 3233664 -f fat -m /var/ -r images/disk.dd >> body.txt -m mnt #output in body file format with mnt prepended to the names mactime -b body.txt [DATE_RANGE] https://wiki.sleuthkit.org/index.php?title=Mactime_output https://wiki.sleuthkit.org/index.php?title=Mactime https://wiki.sleuthkit.org/index.php?title=Timelines

Sleuthkit File Recovery Other Methods Narrow down search using string offset strings -t d -e s disk.img -t d#Print the byte offset as decimal -e s #look for 7-bit byte encoded strings (ASCII,etc.) fsstat -t fat -o 119 disk.img #determine fragment size *NOTE at this point we know the string is located in the filesystem at sector offset 119 (offset/fragment_size) - 119 gives us the fragment (a.k.a data unit, cluster)

Sleuthkit File Recovery Other Methods Narrow down search using string offset ifind -o 119 -a disk.img -d $((offset/fragment_size - 119)) #gives us an inode Side note: -n file_name #you can look for the metadata structure using the filename istat -o 119 disk.img <inode_num> ffind -o 119 -a disk.img <inode_num> tsk_recover -o 119 disk.img <inode_num> https://wiki.sleuthkit.org/index.php?title=FS_Analysis

Questions?

Relevant Links and Interesting Stuff A Survey On Data Carving in Digital Forensics CTF Forensics Intro. https://github.com/trailofbits/ctf/tree/master/forensics srch_strings has a double free corruption bug and crashes. Feel free to look into fixing this. Here’s an issue on SIFT workstation github https://github.com/sans-dfir/sift/issues/178 http://sift.readthedocs.io/en/latest/cheatsheet/index.html#recovering-dat a

Relevant Links and Interesting Stuff Homework hint: Your partitions will not be correct if you blindly run mkfs.* on /dev/loop[0-n] devices. You will want to run it on /dev/loop[0-n]p[0-n]. Or specify the CORRECT parameters to mkfs (it’s easier to do it the first way). sudo fdisk -l disk.img #will show you where the image is set as a loopback device and it’s partitions To see the partitions in dev permanently edit / etc /default/grub and change the following line GRUB_CMDLINE_LINUX=" loop.max_part =63" sudo update-grub Restart the machine To see the partitions in dev temporarily if lsmod | grep - wq loop; then rmmod loop; fi modprobe loop max_part =31