Thursday, September 11, 2008

Digital Evidence Formats

Folks, this is a little off our normal topic of "Intrusions and Malware Analysis," but I think it's entirely relevant.

The Plea
Over the last 6 years that I've been involved with computer forensics/computer security I regularly reflect on the lack of open standards for handling digital evidence. Can we please, as a community, make a grassroots decision to standardize our imaging on a common format?

The Problem
I, personally, have experienced problems with proprietary image formats that are password protected or have incompatible format versions for the analysis software that I am using. When these problems arise it often costs a ridiculous amount of time and potentially money to "fix" the problem. The problem is proprietary evidence container formats. There, I've said it.

Today
Currently, if I had to choose, I would say that "dd", or raw bitstream, would be the best format to standardize on. Split, complete, whatever. Raw format allows for the most flexibility in performing forensics examinations. Every forensics suite (that I know of) supports the format, there are never issues or concerns regarding format or software version, we're able to split the image to support different file systems if need be... In my opinion, it's the best option available to us.

Tomorrow
Now, you may be sitting there thinking, "But raw doesn't support compression" or "But raw doesn't allow me to password protect the evidence." You're right on both counts. I don't think that Raw format is the end-all fix to this problem, but let's start with the raw format and see how we can meet various requirements of evidence containers while staying in the realm of open standards.

Compression
If there is one thing that Open Source/Open Standards software has given us it's compression algorithms. One might argue that there are too many compression formats on open platforms and I would agree. We need to identify a compression algorithm that would bring reasonable compression and ease of split file output for our increasingly large image files.

Protection
The selected compression standard must also allow for password protection. Some evidence may not be able to be processed with a chain of custody at every point and should be protected from illicit viewing and potential tampering.

Other Details
I would like to think of the file format as more of a wrapper than the actual content itself. In addition to handling some level of compression and protection, I'd like to be able to insert a 'hand receipt' into the package. Generate a hashlog of the drive and insert into the overall package. Think about it more like a portable filesystem than a file itself:

\ImageFileName.fez
- PackageContents.xml (Contains MD5 of image, information about origin, etc.)
- Hashlog.txt
- ImageFile.raw.dd
This is very similar to how the Open Office Document format works, it makes sense and could make working across company/organization boundaries much more efficient.

Questions
Did I miss any major features that you would want to see? Let me know in the comments!

2 comments:

Anonymous said...

The biggest problem with open formats is getting them adopted by the commercial forensic tool providers. There is already a open forensic format, the advanced forensic format ( http://www.afflib.org/ ), that has the features your looking for; however, it has been available for some time now and has not been accepted well. Of course there is a fuse file system driver that could convert the aff image (virtually) to a raw image, but commercial tools exist mainly in Windows, FUSE in Unix. Again, the problem isn't technical, but the politics of business.

Unknown said...

@anonymous:

Well stated. I'll look into afflib more. Thank you for the info.

I agree that the problem is not technical, the problem is "politics of business."

While I did discuss some of the technical aspects of what I would want to see in a container spec, my intention was to encourage folks to start asking for open standards in commercial software and ask that developers of open systems keep open standards in mind.

I'll continue to look into the afflib standard and consider a couple of helpful tools that may make it easier to use afflib in the future.

Thanks again for the comments!