{"id":8024,"date":"2022-03-11T23:09:51","date_gmt":"2022-03-11T23:09:51","guid":{"rendered":"https:\/\/www.hexacorn.com\/blog\/?p=8024"},"modified":"2022-03-12T23:13:52","modified_gmt":"2022-03-12T23:13:52","slug":"good-file-what-is-it-good-for-part-2","status":"publish","type":"post","link":"https:\/\/www.hexacorn.com\/blog\/2022\/03\/11\/good-file-what-is-it-good-for-part-2\/","title":{"rendered":"Good file… (What is it good for) Part 2"},"content":{"rendered":"\n

This series<\/a> talks about ‘good’ files. That is, files (samples) produced by reputable vendors, often signed, and hopefully not compromised by stolen certificates, vulnerabilities, supply-chain attacks or bothered by other err… minor inconveniences :-).<\/p>\n\n\n\n

Say you have amassed a bunch of ‘good files and declare your first ‘goodware’ collection as ‘ready for processing’. <\/p>\n\n\n\n

What do you do with it?<\/p>\n\n\n\n

The easiest way to start processing this sampleset is by applying advanced file typing first. We want to know what files we collected, and mind you, I mean not only a distinction between your random media file (jpeg, gif) and PE\/ELF\/MACH-O, but also binary vs. scripting, 32- vs 64-bit, EXE vs DLL, user mode vs. kernelmode, standalone exe vs. .NET, signed vs unsigned, standalone vs installer, installer\/packer\/protector types: autoit, pyinstaller, perl2exe, legacy PE file protection layers (mpress, pecompact, themida, etc.), MSI, and gazillion of existing installer types that typically store the installation information as a compressed\/encrypted appended data behind the generic executable installer stub (Nullsoft, InnoSetup, Wise, etc.). <\/p>\n\n\n\n

To perform this task I use a combination of DiE<\/a> and my own spaghetti-code script that I have been improving over last 17 years (sorry, not for sharing, it’s absolutely disgusting!).<\/p>\n\n\n\n

Once we know what we are dealing with we can try to unpack stuff.<\/p>\n\n\n\n

Why unpack you may ask?<\/p>\n\n\n\n

Because what you download from vendors’ sites is often installers that internally store many additional files, many of which are OF INTEREST. Yup, additional embedded installers, standalone EXE\/DLL\/OCX\/SYS, redistributables, etc.<\/p>\n\n\n\n

How to do it?<\/p>\n\n\n\n

The 7-zip<\/a> is a natural candidate, but we need to be careful. I suspect NSRL analysts use it extensively<\/a> and they unpack everything that has ‘a binary pulse’ recursively until 7z returns error. As a result, they get tones of executable file sections’ metadata sneaking in to their final hash set, and… in the end no one wins. <\/p>\n\n\n\n

The other natural candidate is Universal Extractor<\/a> (and the updated versions of it f.ex. Universal Extractor 2<\/a>). You can’t win this battle either, because it’s too complex and you lose control of what goes unpacked and what ends up in your final metadata set, same as with an overzealous use of 7z.<\/p>\n\n\n\n

We should definitely use these tools though, just need to apply some moderation. <\/p>\n\n\n\n

For instance, we can disable extraction of PE files internals by 7-zip by using its stx<\/a> command line argument:<\/p>\n\n\n\n

7z x -stxPE foo.exe<\/pre>\n\n\n\n
With that, you could run this recursively on all files (including traversing unpacked directories) and get a decent list of internal files, but without unpacking PE files! <\/p>\n\n\n\n
For Universal Extractor-based tools their best use is … analysis of their code. You will find info on both the syntax and required toolkit information that helps to unpack less common archive formats. It’s the best way to study their handling of particular file formats\/installers and how to unpack them so that we can then cherry-pick these that work for us. Again, we don’t want all, as you will end up unpacking lots of useless information e.g. .MSI files and generating a lot of poor quality metadata. Be choosy.<\/p>\n\n\n\n
For InnoSetup there is InnoUnp<\/a>.<\/p>\n\n\n\n
For Nullsoft, there is a 7z version 15.05 that extracts Nullsoft installer files very neatly, including the [NSIS].nsi<\/em> file that is a decently reproduced Nullsoft Installation Script!<\/p>\n\n\n\n
For AutoIT executables you can use ClamAV as pointed out<\/a> by @SmugYeti<\/a>.<\/p>\n\n\n\n
Yup, you have to analyze your advanced file typing results first and then … divide and conquer.<\/p>\n\n\n\n
So…<\/p>\n\n\n\n
Imagine you have file typed all the samples, you know how to unpack them, what’s next?<\/p>\n\n\n\n
I think there are two ways to go about the next step – it starts with a script picking up a single file from your repository, and:<\/p>\n\n\n\n