You are browsing the archive for File Formats ZOO.

Appended data — goodware

September 7, 2019 in Batch Analysis, Clustering, File Formats ZOO

When you take a look at large corpora of appended data — the data that is a part of many PE files, but is not loaded as a part of PE image loading into memory (when a program starts) — patterns emerge.

For malware, this usually means an abuse of a popular installer.

For goodware, it’s a business as usual.

Using the state machine script I discussed in my other post today, I extracted 4 top hexadecimal values from the appended data of many goodware installers.

There are no surprises there — many of appended data blobs are typically in a format utilized by popular and ‘genuine’ installer packages (stub+appended data):

 181472 00 00 00 00 
 131876 4D 53 43 46 - CAB file
  36369 2E 66 69 6C - .file
  36359 7A 6C 62 1A - Inno Setup
  31960 13 00 00 00 
  27981 3B 21 40 49 - 7z SFX
  24883 50 4B 03 04 - Zip
  21721 40 55 41 46 - AMI Flash Utility
  13896 01 00 00 00 
   9489 A3 61 4A 6A 
   9470 5C 73 65 6C -  \self\bin\x86\msvcp60.pdb. 
   8021 52 61 72 21 - Rar!
   7077 0E 00 00 00 
   6855 5F 45 4E 5F - _EN_CODE.BIN

There is an appended that is a CAB, ZIP, RAR file, as well as some proprietary appended data file formats as well.

How can we utilize it from a detection perspective?

Some of them that are not popular among malware samples could become exclusions.

Outliers are a perfect test bed for any PE parser testing. Yes… Does your parser parse every PE file structures properly? While analyzing data for this blog post I have spotted many badly parsed PE files. This is quite a slap in my face. My parser has grown organically over many years and I was quite confident that it ‘handles’ many outliers. I know now that I have to improve it. A humble lesson for any sample collector really…

Finally, knowing what types of installers are being used by a goodware, you can use it as a hint on how to craft your red team tools not to stand out. It may sound silly, but if ‘next gen’/AI/ML algos really exist and they train on a crazily large corpora of samples… chances are that they will learn to ignore many of these popular file setups…

Enter Sandbox part 25: How to get into argument

June 11, 2019 in File Formats ZOO, Malware Analysis, Sandboxing

When you begin your programming career one of the first lessons focuses on reading command line arguments. It is very trivial, but when you start coding more and in new languages you will quickly discover that it’s actually less than trivial and a bit of a mess.

Programming languages use many different ways to access the command line arguments, e.g.:

  • argv
  • wargv
  • args
  • $argv
  • @ARGV
  • arg
  • sys.argv
  • ParamStr
  • Command$
  • WScript.Arguments
  • etc.

I can’t count how many times I googled proper name/syntax for these over the years – ad hoc programming in different languages makes it quite difficult to remember. Also, some programming languages start indexing of arguments from 0, some from 1.

A way to access these parameters also differs. Sometimes you have it available as a string, an array, sometimes you need to call a function to retrieve specific items for you, and in some cases you need to write your own parser or tokenizer.

And finally, some frameworks require certain (standard) approach to passing arguments so that a (standard) parsing routine can extract them properly. Then there are quirks – paths with spaces, extra spaces, ANSI, Unicode characters, and you have two buffers available for parsing – a path to actual executable, and its command line. And the first is not always a full path, or is a path expressed in a different way than expected.

It gets even more complicated when you start reversing. This time it’s not only programming languages per se, but also the binaries they produce and these differ depending on architecture, OS, compiler’s flavor, version, optimization settings. It is all very messy.

Grepping a repo of import function names I came up with this short list of APIs & external, or internal symbols/variables:

  • CommandArgs
  • CommandLineToArgvW
  • GetCommandLineA
  • GetCommandLineW
  • g_shell_parse_argv
  • osl_getCommandArg
  • osl_getCommandArgCount
  • rb_argv
  • StringToArgv
  • _acmdln
  • _wcmdln
  • __argc
  • __argv
  • __p__acmdln
  • __p__wcmdln
  • __p___argc
  • __p___argv
  • __p___wargv
  • __wargv

Why would we need these?

Many programs require command line arguments to run. Sandboxes that can’t recognize these will fail to produce an accurate report. Not only some malware is using this trick on purpose, there are also tones of good programs that end up in sandbox repositories and never get properly analyzed (e.g. compiled work from students of IT, or native OS binaries)

Sandboxes that recognize programming frameworks & the way they parse command line arguments are in a better position to analyze such samples. This is because there is at least a theoretical possibility of heuristic determination if a sample require command arguments, or, if it accepts any. At the very least, they should hint that in their reports.

There are some command line arguments that are universal and can be guessed e.g. /? or /h. Others require a lot of reversing since program’s logic is often hidden under many layers of code and nested calls.

What kind of heuristics we can come up with?

For instance, if an API called immediately after GetCommandLine is ExitProcess then the chances are this program requires command line arguments.

If we can determine location and internal layout of WinMain or main functions and then also of an argc variable (using e.g. signatures, hooking, or emulation, or by monitoring stack), we can attempt to trace the access to this variable. When access is detected we can try to analyze code that is using the variable’s value. If our sample exits almost immediately after this comparison the program most likely is requiring command line arguments.

Other possibilities could involve:

  • monitoring of dedicated parsing routines, e.g. getopt function, but also many inline functions that are embedded in popular frameworks
  • string detection for popular arguments, e.g. /s, -embedding
  • string detection for help information, e.g.: usage:
  • detection of installer type, version (they usually accept some command line arguments that are predefined)
  • fuzzy comparison against known files (if we know sample X required command line arguments, chances are that a similar file will too)
  • ‘reverse proof’ of no CLI requirement
    • if it calls GUI functions then less likely to wait for arguments (but may still accept them)
    • if it is an installer, then we typically know how to handle it (e.g. using clickers)
    • if it is a driver – no command line arguments
    • if it is a DLL, most likely no command line processing (BUT some of the exported functions do rely on command line arguments!)
  • etc.

Overall this is a non-trivial task and there are very poor chances of offering a generic solution here, but it is a good idea to at least flag the file for manual analysis. Either in-house or in a report for client.