Saturday.

Thursdays, Fridays, Saturdays are the days when the malware is compiled the most often.

It kinda makes sense*.

Who would like to work Sundays and Mondays?**

*remember what they say about statistics (data based on 2.5M samples)
**obviously, the APT guys

Administrator.

Many malware samples contain debug strings that include paths often directly pointing to a location where the source code is stored and so it happens that often it’s also a location under the USERPROFILE. For the fun of it, I extracted the strings from a large batch of samples and came up with the following statistics (showing top 50):

   3893 Administrator
   2963 JUANJO
   1121 ryanch
    928 Boy
    617 UserXP
    612 user
    519 1337
    502 User
    465 Admin
    435 root
    422 bld4act
    418 Owner
    347 nosferatus
    305 Administrateur
    300 M4x
    296 ismael
    277 goga
    277 Kyle
    255 Mirko
    247 1134
    244 kdglkrkjdfhslej
    241 FEDERIKO
    234 t0fx
    231 rstephens
    219 DarkCoderSc
    218 gcc
    205 icyheart
    200 Dave
    197 michael
    197 Roshan
    197 James
    195 Ben
    182 John
    178 admin
    173 Dev
    161 box1
    157 nonadmin
    153 FELIPE
    152 Familie
    151 Timothy
    137 Dhivin
    133 Vortex
    131 Robert
    130 dabdoub
    129 USER
    127 dr zinou
    125 packar
    122 David
    116 nathu
    116 Daniel

It’s obviously biased.

Other interesting names include:

• tom age five
• GANGSTA
• Krusty the Clown
• ^_^
• ItchyFingerz
• irishboy
• romantic
• lol
• brad pitt
• Love Bebe
• LorD^^$$steal3R
• Cyber-Warrior Ender
• auchan
• F-B-I
• Valued Sony Customer
• SexyReplay
• Microsoft
• Poo
• Trojan
• P@wn3d
• Emperor Zhou Tai Nu

There are over 7000 account names on the list. If you want the full list, please contact me offline.

Part 1, Part 2, Part 3

In the last three posts I talked about batch analysis, clustering and applying these techniques to APT sampleset.

Batch processing is a step necessary for retrieving ‘clusterable’ data from samples in an automated fashion.

Clustering is a way of putting these samples into buckets, potentially grouping them into some families.

I want to see if w/o using any assumption/knowledge (retrieved from the white paper or other blogs) it is possible to cluster these samples in a reliable way. It is an interesting experiment and I am curious if I will ever get closer to already known clusters. Quite frankly, I don’t know yet. We shall see.

The clustering I have done so far was focused on dynamic analysis and a little bit on the source code analysis. In this post I will exploit code analysis further – this time focusing on disassembled .asm files generated as usual by the IDA Pro.

The resulting assembly code is quite nice for parsing as each line contains only one line of code – this allows to group the code into blocks on function boundaries and for each call to API or to another subroutine (including calls via registers), we can extract a simplified code of the program procedures e.g.

sub_401000    proc near        ; CODE XREF: _main+20Ap
[...]

lea    ecx, [esp+310h+szLongPath]
push    104h        ; nSize
push    ecx        ; lpFilename
push    0        ; hModule
call    ds:GetModuleFileNameA

lea    edx, [esp+310h+szLongPath]
push    104h        ; cchBuffer
lea    eax, [esp+314h+szLongPath]
push    edx        ; lpszShortPath
push    eax        ; lpszLongPath
call    ds:GetShortPathNameA

lea    ecx, [esp+310h+Parameters]
push    offset String2    ; "/c del "
push    ecx        ; lpString1
call    ds:lstrcpyA

mov    esi, ds:lstrcatA
lea    edx, [esp+310h+szLongPath]
lea    eax, [esp+310h+Parameters]
push    edx        ; lpString2
push    eax        ; lpString1
call    esi ; lstrcatA

lea    ecx, [esp+310h+Parameters]
push    offset s->>>nul    ; " >>NUL"
push    ecx        ; lpString1
call    esi ; lstrcatA

mov    esi, ds:ShellExecuteA
push    0        ; nShowCmd
push    offset Directory ; lpDirectory
lea    edx, [esp+318h+File]
push    offset Parameters ; "/c    del wuauclt.exe"
push    edx        ; lpFile
push    offset Operation ; "open"
push    0        ; hwnd
call    esi ; ShellExecuteA

push    0        ; nShowCmd
push    offset Directory ; lpDirectory
lea    eax, [esp+318h+File]
push    offset s->CDelSvchost_exe ; "/c    del svchost.exe"
push    eax        ; lpFile
push    offset Operation ; "open"
push    0        ; hwnd
call    esi ; ShellExecuteA

[...]
retnsub_401000    endp

becomes

GetModuleFileNameA

GetShortPathNameA

lstrcpyA

lstrcatA

lstrcatA

ShellExecuteA

ShellExecuteA

ShellExecuteA

and can be written as a single line of code

GetModuleFileNameA|GetShortPathNameA|lstrcpyA|lstrcatA|lstrcatA|ShellExecuteA|ShellExecuteA|ShellExecuteA

Applying such methodology on procedure boundaries and to each disassembled program I eventually came up with a shortened and flattened source code of each sample. I then built a histogram of the most common sequences of such code blocks across all the source code from all files and got the following stats:

   5514 |sub
   2507 |sub|sub
   1332 |sub|sub|sub
    860 |sub|sub|sub|sub
    558 |__security_check_cookie(x)
    479 |__security_check_cookie(x)|__security_check_cookie(x)
    475 |sub|sub|sub|sub|sub
    392 |sub|sub|sub|sub|sub|sub
    353 |operator delete(void *)
    276 |sub|operator delete(void *)
    269 |sub|sub|sub|sub|sub|sub|sub
    235 |sub|sub|sub|sub|sub|sub|sub|sub
    185 |sub|sub|sub|sub|sub|sub|sub|sub|sub
    168 |sub|sub|sub|sub|sub|sub|sub|sub|sub|sub
    165 |__alloca_probe|sub|sub
    137 |eax
    132 |sub|sub|ecx
    132 |__alloca_probe|sub
    130 |_atexit
    123 |sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub
    110 |_chkstk|sub|sub
    108 |strlen|operator delete(void *)|operator new(uint)|strcpy
    106 |nullsub
    106 |__alloca_probe
    101 |_chkstk|sub
     97 |eax|sub
     92 |__alloca_probe|sub|sub|sub|sub
     91 |__alloca_probe|sub|sub|sub
     88 |_chkstk|sub|sub|sub
     88 |__alloca_probe|sub|sub|sub|sub|sub|sub
     85 |__alloca_probe|sub|sub|sub|sub|sub
     80 |exception const &)
     75 |__alloca_probe|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub
     73 |strlen
     73 |_chkstk|sub|sub|sub|sub|sub
     72 |sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub
     71 |__alloca_probe|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub
     71 |_Tidy(bool,uint)
     69 |sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub
     68 |sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub
     68 |_chkstk|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub
     68 |_chkstk|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub
     68 |_chkstk|sub|sub|sub|sub|sub|sub|sub|sub
     68 |InternetCloseHandle|InternetCloseHandle|InternetCloseHandle
     67 |sub|eax
     63 |_chkstk|sub|sub|sub|sub|sub|sub
     62 |__alloca_probe|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub
     62 |__alloca_probe|sub|sub|sub|sub|sub|sub|sub|sub
     61 |free
     60 |sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub
     60 |allocator<char>>(char const *)|_atexit
     59 |sub|_CxxThrowException(x,x)
     56 |_CxxThrowException
     56 |InternetReadFile
     55 |_chkstk|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub
     55 |_chkstk
     55 |SetUnhandledExceptionFilter
     52 |operator new(uint)|exception(char const * const &)|_CxxThrowException(x,x)
     52 |operator delete(void *)|_CxxThrowException(x,x)
     52 |_flsall
     51 |_chkstk|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub
     51 |__alloca_probe|sub|sub|sub|sub|sub|sub|sub|sub|sub
     50 |_chkstk|sub|sub|sub|sub
     49 |j_free
     48 |_chkstk|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub
     47 |sub|sub|_CxxThrowException(x,x)
     47 |__alloca_probe|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub
     45 |sub|sub|sub|sub|eax
     44 |strchr|strchr
     44 |malloc|sub|sub|free
     43 |dword ptr [ecx+8]
     42 |__alloca_probe|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub
     40 |sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub
     40 |sub|_Split(void)|_wmemmove|sub|_Eos(uint)|_Split(void)|_Tidy(bool)|sub
     40 |operator delete(void *)|operator delete(void *)
     40 |_chkstk|sub|sub|sub|sub|sub|sub|sub|sub|sub
     40 |__alloca_probe|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub
     40 |ReadFile|_memcpy_0
     39 |sub|_CxxThrowException
     39 |GetModuleFileNameA|GetShortPathNameA|GetEnvironmentVariableA|lstrcpyA|lstrcatA|lstrcatA|GetCurrentProcess|SetPriorityClass|GetCurrentThread|SetThreadPriority|ShellExecuteExA|SetPriorityClass|SetProcessPriorityBoost|SHChangeNotify|GetCurrentProcess|SetPriorityClass|GetCurrentThread|SetThreadPriority
     38 |sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub
     38 |_chkstk|sub|sub|sub|sub|sub|sub|sub
     37 |GetCurrentProcess|OpenProcessToken|LookupPrivilegeValueA|AdjustTokenPrivileges|CloseHandle|GetLastError
     36 |sub|sub|dword ptr [eax]|sub|sub|sub
     36 |sub|ecx
     36 |dword ptr [ecx+4]
     36 |_memset|sub|__security_check_cookie(x)
     35 |sub|sub|__security_check_cookie
     35 |sub|operator delete(void *)|operator delete(void *)|operator delete(void *)|operator delete(void *)
     35 |__invalid_parameter_noinfo
     34 |operator new(uint)
     34 |_free
     34 |_LocaleUpdate(localeinfo_struct *)|___strgtold12_l|sub|__security_check_cookie(x)
     33 |sub|sub|eax|sub
     33 |sub|operator delete(void *)|operator delete(void *)
     33 |_chkstk|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub
     33 |__errno|__invalid_parameter
     32 |operator delete(void *)|operator new(uint)
     32 |memset
     31 |operator new(uint)|sub
     31 |_chkstk|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub
     30 |eax|sub|sub|sub|sub
     30 |_chkstk|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub
     30 |__EH_prolog|_Tidy(bool)|_strlen|sub|sub|_CxxThrowException(x,x)
     30 |SetServiceStatus
     28 |_chkstk|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub
     27 |sub|_Split(void)|_memcpy|sub|_Eos(uint)|_Split(void)|_Tidy(bool)|sub
     27 |strlen|sub
     27 |memcpy
     27 |_strcmpi|memset|memset|CreateToolhelp32Snapshot|Process32First|sprintf|strcat|Process32Next|CloseHandle|_strcmpi|OpenSCManagerA|EnumServicesStatusExA|operator new(uint)|CloseServiceHandle|strcat|EnumServicesStatusExA|sprintf|strcat|operator delete(void *)|CloseServiceHandle|_strcmpi|GetLogicalDrives|sprintf|strcat|sprintf|strcat|lstrcatA|GetDriveTypeA|strcat|GetVolumeInformationA|strcat|strcat|sprintf|strcat
     27 |_strcmpi|atoi|OpenProcess|TerminateProcess|CloseHandle|strcat|_strcmpi|OpenSCManagerA|OpenServiceA|GetLastError|strcat|CloseServiceHandle|ControlService|GetLastError|strcat|CloseServiceHandle|CloseServiceHandle
     27 |__alloca_probe|sub|sub|sub|sub|sub|sub|sub
     27 |GetProcAddress
     27 |GetExitCodeProcess|PeekNamedPipe|Sleep|ReadFile|CloseHandle|CloseHandle|memset|strcpy|strlen
     26 |sub|sub|sub|sub|_memcpy_s
     26 |sub|eax|sub|eax|sub
     26 |sub|_Tidy(bool)|_Tidy(bool)|sub
     26 |strstr|strchr|operator new(uint)|strchr|strchr|strchr|strchr|strchr|strchr|strchr|strchr|strchr|operator delete(void *)
     26 |strlen|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub
     26 |sprintf|HttpAddRequestHeadersA|HttpSendRequestA|GetLastError|InternetQueryOptionA|InternetSetOptionA|sprintf
     26 |__ld12cvt
     26 |___strgtold12|sub
     26 |__EH_prolog3|sub|sub|_CxxThrowException(x,x)
     26 |InternetOpenA|InternetSetOptionA|InternetSetOptionA|InternetSetOptionA|InternetConnectA|HttpOpenRequestA|strlen|HttpAddRequestHeadersA
     26 |$+5
     25 |rand
     25 |malloc|CreatePipe|CreatePipe|CloseHandle|CloseHandle|CloseHandle|CloseHandle|free|sub|CloseHandle|CloseHandle
     25 |_chkstk|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub
     25 |__invalid_parameter_noinfo|__invalid_parameter_noinfo
     25 |__alloca_probe|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub
     25 |URLDownloadToFileA|strcat
     24 |sub|sub|sub|sub|sub|GetProcAddress|sub|sub|sub
     24 |sub|edx|sub
     24 |sub|_Split(void)|_wmemmove|sub|_Eos(uint)|_Split(void)|sub|sub
     24 |shutdown|closesocket
     24 |send
     24 |fopen|fseek|fread|fseek|ftell|fseek|fread|fclose|fclose|fread|fclose|sub
     24 |edx
     24 |dword ptr [eax+40h]
     24 |_beginthreadex|CloseHandle
     24 |__alloca_probe|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub
     24 |GetModuleHandleA|GetProcAddress
     23 |unknown_libname_1
     23 |sub|sub|sub|sub|operator delete(void *)
     23 |sub|OpenProcess|TerminateProcess|Sleep|CloseHandle|sub
     23 |strlen|CreateFileA|strlen|operator new(uint)|memset|WriteConsoleInputA|operator delete(void *)|CloseHandle
     23 |strcat|sub|WaitForSingleObject|strcat|strcat|strlen|sub
     23 |j_free|j_free
     23 |j_free|_CxxThrowException
     23 |LoadStringA|sub
     23 |CloseHandle
     22 |~type_info(void)|operator delete(void *)
     22 |sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub
     22 |sub|operator new(unsigned __int64)|exception(char const * const &)|_CxxThrowException|sub|sub|j_free
     22 |operator new(uint)|operator new(uint)|sub
     22 |operator new(uint)|operator delete(void *)
     22 |operator delete(void *)|operator delete(void *)|operator delete(void *)
     22 |exception(char const * const &)
     22 |eax|sub|sub|sub
     22 |GetCurrentProcess|GetCurrentProcess|DuplicateHandle|CreateProcessA|CloseHandle
     22 |CompareStringA
     22 |$+5|sub|sub
     21 |sub|_wcslen|sub|sub|sub|sub
     21 |sprintf|sprintf|sub
     21 |malloc|recv|sub|sub|_strnicmp|WriteFile|recv|free|ExitThread|SetEvent|free|ExitThread
     21 |malloc|PeekNamedPipe|ReadFile|sub|sub|_itoa|send|sub|Sleep|PeekNamedPipe|free|ExitThread
     21 |_strcmpi|memset|CreateProcessA|strcat|CloseHandle|_strcmpi|OpenSCManagerA|strcat|OpenServiceA|GetLastError|strcat|CloseServiceHandle|StartServiceA|GetLastError|strcat|CloseServiceHandle|CloseHandle
     21 |__get_sse2_info
     21 |__alloca_probe|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub
     21 |__alloca_probe|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub
     21 |__alloca_probe|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub
     21 |__alloca_probe|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub|sub
     21 |GetCurrentProcess|OpenProcess|GetLastError|sprintf|strcat|OpenProcessToken|memset|sprintf|CreateProcessAsUserA|strcat|CloseHandle|CloseHandle|GetLastError|sprintf|strcat|CloseHandle|GetLastError|sprintf|strcat|CloseHandle
     21 |CreateEventA|CreateEventA|sub|WaitForSingleObject|CloseHandle
     21 |$+5|sub

Using these shortened procedures for cluster generations gives some promising results e.g.:

sub
DeleteFileW
DeleteFileA

1328eaceb140a3863951d18661b097af.asm
31e5e58dbdfad05175613e795298ebb5.asm
6f9992c486195edcf0bf2f6ee6c3ec74.asm
c99fa835350aa9e2427ce69323b061a9.asm
e476e4a24f8b4ff4c8a0b260aa35fc9f.asm
ea1b44094ae4d8e2b63a1771a3e61fd5.asm
fc1937c1aa536b3744ebdfb1716fd54d.asm

LoadLibraryA
GetProcAddress
GetProcAddress
GetProcAddress

3f8682ab074a097ebbaadbf26dfff560.asm
4b19a2a6d40a5825e868c6ef25ae445e.asm
54d5d171a482278cc8eacf08d9175fd7.asm
56de2854ef64d869b5df7af5e4effe3e.asm
75dad1ccabae8adeb5bae899d0c630f8.asm
8462a62f13f92c34e4b89a7d13a185ad.asm

htons
socket
connect
closesocket

468ff2c12cffc7e5b2fe0ee6bb3b239e.asm
727a6800991eead454e53e8af164a99c.asm
bd8b082b7711bc980252f988bb0ca936.asm
db05df0498b59b42a8e493cf3c10c578.asm
e1b6940985a23e5639450f8391820655.asm

ecx
eax
dword ptr [esi+10h]
sub
ecx
eax
sub
sub
sub
sub
sub
sub
sub
sub

12f25ce81596aeb19e75cc7ef08f3a38.asm
268eef019bf65b2987e945afaf29643f.asm
468ff2c12cffc7e5b2fe0ee6bb3b239e.asm
4c6bddcca2695d6202df38708e14fc7e.asm
5a728cb9ce56763dccb32b5298d0f050.asm
727a6800991eead454e53e8af164a99c.asm
8e8622c393d7e832d39e620ead5d3b49.asm
bd8b082b7711bc980252f988bb0ca936.asm
c6a4bb1a4e4f69ec71855d70d6960859.asm
db05df0498b59b42a8e493cf3c10c578.asm
e1b6940985a23e5639450f8391820655.asm
ef8e0fb20e7228c7492ccdc59d87c690.asm

LoadLibraryA
GetProcAddress
sub
sub
strstr
strchr
GetSystemDirectoryA
time
srand
malloc
sub
sub
strncmp
Sleep
sub
Sleep
sub
Sleep
CreatePipe
CreatePipe
GetStartupInfoA
CreateProcessA
GetLastError
_snprintf
sub
CreateProcessA
CreateThread
CreateThread
WaitForMultipleObjects
GetExitCodeThread
TerminateThread
GetExitCodeThread
TerminateThread
GetExitCodeProcess
TerminateProcess
sub
sub
GetLastError
_snprintf
sub
CloseHandle
CloseHandle
CloseHandle
CloseHandle
sub
sub
Sleep
PeekNamedPipe
ReadFile
sub

0dd3677594632ce270bcf8af94819caf.asm
270d42f292105951ee81e4085ea45054.asm
523f56515221161579ee6090c962e5b1.asm

Notably, the disassembled code – after some selective processing and normalization – can be treated in a same way as student source code submissions for their assessments at uni and… be checked for plagiarism. The most common technique used for this purpose relies on measuring the  cosine similarity. I am currently playing with it and will write more about my findings in another post.

Thanks for reading!

Part 1, Part 2, Part 3

In my last post, I presented results of the batch analysis and clustering attempts of the APT1 sampleset. Today, I will continue on the topic of clustering, this time looking at the problem from a different angle. Again, results are not mind blowing, but it’s an experiment and it’s not about the destination, but about the journey

Typical programs, even if recompiled or rebuilt with different configuration often preserve their internal structure. It is partially a result of ‘if it works, don’t touch’ approach, programmer’s laziness, his/her coding habits, a side effect of the ‘business logic’ implemented in the program and many other factors. One can compare a patched file and immediately spot the changes; or, pick up a a characteristic sequence of code or data and search for similar sequences in the whole sampleset. The fact that it is possible to compare programs on a binary level is well known and pretty much every 0day hunter has at some stage used or is currently using this technique in their bughunting adventures (using tools like BinNavi).

In a context of sample clustering, I think we don’t necessarily need to go as far as in-depth binary code comparison – there are a lot of shortcuts we can take here. The easiest is to pick up these code sequences that refer to strings.  And to narrow down the scope for this post, we only look at string comparisons. They are used for parsing of command line arguments, RAT/bot commands, data sent over the protocols and so on and so forth.

There are a few ways compilers implement string comparisons on a machine code level – based on a quick research and already done clustering on the APT1 sampleset, I know that plenty of string comparisons are done using just 5 functions: _strnicmp, memcmp, strcmp, strncmp, strstr.

In order to look at the code sequences of this sort we need to find a better way of extracting strings from the samples, because a typical static analysis tool extracts strings from the file in a ‘dumb’ way and doesn’t provide the necessary programming/algorithmic context. Using a tool like PESectionExtractor doesn’t help either as the context it provides is related only to a physical location of the string, and doesn’t tell us how the string is being used by the program. What we need is a tool that can disassemble the code properly or even better – decompile it. And the obvious choice here is Ida Pro with its Hex-Rays plugin.

The resulting files are a bit too large for the direct copy and paste in this post, so I am providing direct links to text files below:

•  _strnicmp_cluster.txt
• memcmp_cluster.txt
• strcmp_cluster.txt
• strncmp_cluster.txt
• strstr_cluster.txt

I have recently been toying around with clustering of various malicious sample sets – running files through a sandbox and static analysis tools, and then applying various normalization and histograms to the output. The results are not mind-blowing, but encouraging. They help in grouping various malware families into separate buckets, improve log parsing routines, and in some cases can be also leveraged to quickly discover hidden properties of the malware e.g. encryption keys, User Agents, HTTP verbs, etc. etc. – these may be then  used for more in-depth analysis of proxy logs, etc.

Here is a short list of ‘clusterable’ attributes just in case you want to design your own clustering solution and are looking for a quick cheat list; it is certainly far from being complete, but may give you some pointers:

STATIC

• File Name
• File Extension
• File Size
• File Type
  • This will have a lot of ‘subtypes’ – for MZ files see details here and here
  • For executable – sequence of bytes at the entry point, and at the real entry point (for main, wmain, DLLMain, as well as for VB, Delphi code)
  • For PE file – for each of these: their names where applicable, sizes, flags, entropy, strings:
    • sections (for list of known sections see here)
    • import tables
    • export tables
  • For PE file -
    • PE type
    • Image base
    • Compilation/debug time stamps
    • Resources – number, topology
    • Debug strings
• File Entropy
• Compiler (PEiD, etc.)
• Packer, protector
• File hashes (MD5, SHA1, CTPH, …)
• Extracted strings
• Presence and characteristics of appended  data (e.g. installers)
• Sequences of code
  • Disassembled code
  • Decompiled code
  • Selected code (e.g. map of calls)
• Detection by various AVs
• Multimedia properties (e.g. width, height, EXIF data, etc.)

DYNAMIC

• Accessed IPs
• Accessed URLs
• GET and POST Queries
• User Agents
• Ports used
• Created/accessed Mutexes/mutants
• Created/accessed Atoms
• Created/accessed Window names
• Created/accessed Window classes
• Created/accessed Windows topology
• Windows’ visibility
• Windows’ Unicodeness
• Windows’ topology
• Windows’ titles
• Windows’ classes
• Crypto used + built-in or API-based
• Popular strings used (e.g. copyright banners as seen here)
• Execution paths (code, sequences, code blocks, API sequences)
• Use of location-independent code
• Use of escalation of privileges tricks
• Use and type of code injection
• Use of kernel drivers (including system DLLs)
• Use of stolen certificates
• Use of anti-* techniques
• Use of 0days
• Use of timestomping
• Use of dynamically vbuilt strings (run-time)
• Use of code to adjust privileges)
• Use of keylogging techniques (and what type: hook, API hook, etc.)
• Use of external tools (e.g. cmd.exe, reg.exe, net.exe)
• Use of autoruns.inf
• Use of DKOM
• Use of code directly accessing physical drives
• Use of code directly accessing physical memory
• Use of code directly accessing BIOS
• Use of hypervisor
• MBR – code modification
• MBR – partition table modification
• Passwords used for encryption and to access (e.g. FTP/SMTP/IRC)
• Dropped file locations, names
• Searched path locations, registry names
• Targeted applications (e.g. browser, mail, IM and P2P clients, etc.)
• Added/modified registry entries
• APIs executed and their arguments
  • Type of APIs (kernel32 win32 APIs or ntdll Zw/NT APIs)
  • Delays used in waiting functions
  • APIs/techniques used for memory allocation (heap, virtual*, stack-based, etc.)
  • APIs/techniques used for self-deletion
  • APIs/techniques used for running other .exes
  • APIs/techniques used for network (winsock or wininet/also Rtl functions from ntdll)
  • APIs/techniques used for network enumeration (Net*, WNet*, Domain*)
  • Process enumeration APis
  • …

Let me interrupt you here…

Okay, okay, I get i!!! It is a never ending list!!!

Another round of stats – this time the top 100+ most ‘popular’ PE i386 file formats used by malware from over 1.2M samples.

Legend:

• MZ PE i386 = PE 32 bit
• DLL = DLL
• Corrupted or Tricky = for some reason parser failed (usually some PE file tricks)
• APPDATA xxxxxxxx = appended data followed by first 1-4 characters
• SIG = contains directory entry pointing to signature (often it’s a random garbage though, not stolen certificates)
• DEB = contains debugging information
• COM = COM library
• .NET = .NET PE
• and lots of names related to various installers

 (44.17%)    560067    MZ PE i386
  (6.59%)     83554    MZ PE i386 DLL
  (6.16%)     78149    MZ PE i386 Corrupted Tricky
  (4.84%)     61379    MZ PE i386 DEB
  (3.51%)     44529    MZ PE i386 APPDATA 00000000
  (2.99%)     37871    MZ PE i386 SIG
  (2.81%)     35644    MZ PE i386 Tricky
  (2.01%)     25462    MZ PE i386 DLL COM
  (1.30%)     16478    MZ PE i386 NullSoft 2.46-1 SIG
  (1.28%)     16253    MZ PE i386 DLL DEB
  (1.28%)     16220    MZ PE i386 .NET
  (1.04%)     13128    MZ PE i386 SYS
  (0.98%)     12459    MZ PE i386 Tricky SIG
  (0.92%)     11614    MZ PE i386 NullSoft Unknown
  (0.82%)     10393    MZ PE i386 InnoSetup
  (0.78%)      9831    MZ PE i386  AutoIt or AutoHotKey
  (0.77%)      9709    MZ PE i386 Corrupted Tricky DEB
  (0.65%)      8273    MZ PE i386 .NET APPDATA 00000000
  (0.65%)      8217    MZ PE i386 DEB SIG
  (0.64%)      8166    MZ PE i386 NullSoft 2.46
  (0.61%)      7757    MZ PE i386 DLL APPDATA 00000000
  (0.54%)      6881    MZ PE i386 .NET DEB
  (0.48%)      6131    MZ PE i386 Zip Sfx
  (0.48%)      6054    MZ PE i386 Tricky DEB
  (0.47%)      5938    MZ PE i386 Rar SFX
  (0.46%)      5891    MZ PE i386 NullSoft 2.45
  (0.46%)      5836    MZ PE i386 APPDATA B80E0000
  (0.44%)      5631    MZ PE i386 DLL Corrupted Tricky
  (0.42%)      5318    MZ PE i386 Appended MZ
  (0.42%)      5312    MZ PE i386 APPDATA 01000000
  (0.42%)      5279    MZ PE i386 InstallAware
  (0.41%)      5232    MZ PE i386 Tricky DEB SIG
  (0.40%)      5074    MZ PE i386 NullSoft 2.27
  (0.37%)      4733    MZ PE i386 Trymedia
  (0.36%)      4549    MZ PE i386 APPDATA 00000000 DEB
  (0.36%)      4546    MZ PE i386 APPDATA 3C706172
  (0.34%)      4336    MZ PE i386 SYS DEB
  (0.33%)      4161    MZ PE i386 APPDATA A5B79A82
  (0.29%)      3690    MZ PE i386 NullSoft 2.46 SIG
  (0.23%)      2973    MZ PE i386 Trymedia SIG
  (0.23%)      2925    MZ PE i386 APPDATA 88110000
  (0.23%)      2918    MZ PE i386 .file
  (0.22%)      2799    MZ PE i386 Rar SFX DEB
  (0.22%)      2728    MZ PE i386 APPDATA B00E0000
  (0.19%)      2440    MZ PE i386 .NET Tricky
  (0.19%)      2422    MZ PE i386 DLL Tricky
  (0.19%)      2405    MZ PE i386 APPDATA 31353835
  (0.18%)      2255    MZ PE i386 DLL COM APPDATA 00000000
  (0.18%)      2234    MZ PE i386 APPDATA 56566245
  (0.17%)      2206    MZ PE i386 NullSoft 2.46-5 SIG
  (0.16%)      2078    MZ PE i386 APPDATA 08080000
  (0.16%)      2036    MZ PE i386 DLL COM DEB
  (0.16%)      1990    MZ PE i386 .NET DLL DEB
  (0.14%)      1750    MZ PE i386 APPDATA 001F0023
  (0.14%)      1750    MZ PE i386 APPDATA 5B424547 SIG
  (0.13%)      1706    MZ PE i386 DLL SIG
  (0.13%)      1678    MZ PE i386 NullSoft 2.24
  (0.13%)      1633    MZ PE i386 NullSoft 2.44
  (0.13%)      1597    MZ PE i386 DLL APPDATA 928F8C89
  (0.13%)      1585    MZ PE i386 Wise
  (0.12%)      1582    MZ PE i386 DEB
  (0.12%)      1576    MZ PE i386 DLL APPDATA 861DC8F1
  (0.12%)      1545    MZ PE i386 APPDATA 73676567
  (0.12%)      1537    MZ PE i386 APPDATA 50415443
  (0.12%)      1517    MZ PE i386 APPDATA 5A425245
  (0.11%)      1458    MZ PE i386 APPDATA 60170000 DEB
  (0.11%)      1417    MZ PE i386 DLL Corrupted Tricky DEB
  (0.11%)      1374    MZ PE i386 APPDATA 68480000
  (0.11%)      1367    MZ PE i386 NullSoft 25-Apr-2011.cvs
  (0.11%)      1359    MZ PE i386 APPDATA 3C62696E
  (0.10%)      1288    MZ PE i386 APPDATA 88190000
  (0.10%)      1272    MZ PE i386 APPDATA 980E0000
  (0.10%)      1219    MZ PE i386 APPDATA 6BD6EB2C
  (0.10%)      1213    MZ PE i386 InnoSetup SIG
  (0.09%)      1176    MZ PE i386 InstallShield DEB
  (0.09%)      1174    MZ PE i386 APPDATA 680C0000
  (0.09%)      1159    MZ PE i386 CAB SFX (shifted)
  (0.09%)      1137    MZ PE i386 SYS DLL DEB
  (0.09%)      1122    MZ PE i386 APPDATA 90909090
  (0.09%)      1102    MZ PE i386 APPDATA 00A80000 DEB
  (0.09%)      1091    MZ PE i386 APPDATA 05000000
  (0.09%)      1087    MZ PE i386 .NET DLL
  (0.09%)      1082    MZ PE i386 APPDATA 22A72792
  (0.08%)      1048    MZ PE i386 .NET Corrupted Tricky
  (0.08%)      1043    MZ PE i386 APPDATA C26402DF
  (0.08%)       990    MZ PE i386 Rar SFX (shifted) DEB
  (0.07%)       947    MZ PE i386 APPDATA 3C232440
  (0.07%)       903    MZ PE i386 DLL COM Appended MZ
  (0.07%)       896    MZ PE i386 NullSoft 2.14
  (0.07%)       892    MZ PE i386 Rar SFX (shifted)
  (0.07%)       885    MZ PE i386 APPDATA 0D0A0D0A
  (0.07%)       880    MZ PE i386 SYS DLL
  (0.07%)       877    MZ PE i386 NullSoft 01-Jun-2011.cvs SIG
  (0.07%)       874    MZ PE i386 SmartInstallMaker v.5.02
  (0.06%)       808    MZ PE i386 DLL COM SIG
  (0.06%)       807    MZ PE i386 NullSoft 2.37
  (0.06%)       802    MZ PE i386 ADAEBOOK
  (0.06%)       789    MZ PE i386 APPDATA 78766D00
  (0.06%)       764    MZ PE i386 DLL COM
  (0.06%)       737    MZ PE i386 Install Creator
  (0.06%)       719    MZ PE i386 APPDATA 2A2A2A2A
  (0.06%)       715    MZ PE i386 WebCompiler
  (0.06%)       707    MZ PE i386 APPDATA 00
  (0.05%)       693    MZ PE i386 APPDATA 08001700
  (0.05%)       669    MZ PE i386 APPDATA 00000000 SIG
  (0.05%)       665    MZ PE i386 NullSoft 2.24 SIG
  (0.05%)       656    MZ PE i386 APPDATA 31353836
  (0.05%)       651    MZ PE i386 DLL APPDATA 45474645 DEB
  (0.05%)       628    MZ PE i386 DLL DEB SIG
  (0.05%)       622    MZ PE i386 APPDATA 43434343
  (0.05%)       617    MZ PE i386 APPDATA 34120000

Over last few months I have been publishing various stats pulled out of malware collection that I am batch analyzing. The purpose of analysis is not only just getting interesting numbers and utilizing it as a nice filler for the blog – all this data is being retrieved with a purpose of enhancing HexDive and for my other projects.Until now, I have been presenting data from a superset of all malicious PE files in a collection.  It crossed my mind recently that it would be interesting to focus on a subset of PE files as well and for starters I picked up kernel drivers.

Getting all strings and then cherrypicking up system functions out of the samples is relatively quick as there is not so many of them – the result of top 100 most popular APIs sorted by number of occurrence is presented below:

18431    RtlInitUnicodeString
16625    IofCompleteRequest
16214    ExAllocatePoolWithTag
14783    ZwClose
12899    MmGetSystemRoutineAddress
12002    ZwOpenKey
11911    ObfDereferenceObject
11719    IoCreateDevice
11430    IoGetCurrentProcess
11411    ExFreePool
11395    IoDeleteDevice
11198    RtlAnsiStringToUnicodeString
10969    ZwCreateFile
10895    wcslen
10848    strncmp
10672    strncpy
10585    wcscpy
10195    IoCreateSymbolicLink
10141    swprintf
9957    wcscat
9899    PsCreateSystemThread
9495    MmIsAddressValid
9466    ZwSetValueKey
9112    PsLookupProcessByProcessId
9106    ObReferenceObjectByHandle
8971    PsGetVersion
8630    ZwCreateKey
8600    RtlCopyUnicodeString
8334    KeDelayExecutionThread
7925    RtlCompareUnicodeString
7886    wcsncpy
7861    ZwQueryValueKey
7525    KeTickCount
7135    KeQuerySystemTime
7052    IoRegisterDriverReinitialization
6674    PsSetCreateProcessNotifyRoutine
5968    ExFreePoolWithTag
5671    ZwEnumerateKey
5427    ZwQuerySystemInformation
5414    ZwSetInformationFile
5249    ZwDeleteKey
5072    wcsstr
5017    KeWaitForSingleObject
4922    ZwCreateSection
4855    ZwMapViewOfSection
4757    IoDeleteSymbolicLink
4747    PsTerminateSystemThread
4708    wcschr
4605    wcsrchr
4540    KeServiceDescriptorTable
4226    KeQueryTimeIncrement
4218    ZwUnmapViewOfSection
4070    IoDeviceObjectType
3941    ZwReadFile
3740    KeInitializeEvent
3706    KeInitializeTimer
3562    ObQueryNameString
3538    ZwWriteFile
3522    KeSetEvent
3495    DbgPrint
3470    KeGetCurrentIrql
3381    KeBugCheckEx
3313    ZwQueryInformationFile
3286    ZwOpenFile
3232    IoFreeMdl
3171    RtlInitAnsiString
3043    memcpy
3037    IofCallDriver
2897    memset
2892    RtlFreeUnicodeString
2870    IoAllocateMdl
2629    MmProbeAndLockPages
2461    MmUnlockPages
2349    RtlUnicodeStringToAnsiString
2340    ZwAllocateVirtualMemory
2291    IoFreeIrp
2265    MmMapLockedPagesSpecifyCache
2144    KeGetCurrentThread
2134    KfReleaseSpinLock
2090    RtlFreeAnsiString
2031    KeStackAttachProcess
2025    KfRaiseIrql
2022    KfLowerIrql
1997    IoAllocateIrp
1997    ExAllocatePool
1994    RtlCompareMemory
1967    ExGetPreviousMode
1930    RtlTimeToTimeFields
1918    sprintf
1896    KeUnstackDetachProcess
1884    KfAcquireSpinLock
1870    ZwOpenProcess
1808    PsGetCurrentProcessId
1795    KeReleaseMutex
1747    RtlAppendUnicodeToString
1746    KeInitializeSpinLock
1740    IoCreateFile
1729    ProbeForRead
1727    KeClearEvent
1713    RtlUnwind

update2

updated section list/fixed bugs – thanks to Nicolas Brulez and Tomislav Pericin (ap0x)

update

added one more list – List of popular section names

old post

I continue to batch analyze my malware collection and the latest list I generated contains:

• The most popular PE file section names
• The packer/protector section names/keywords – I tried to build a separate list of known section names/keywords that belong to known packers/protectors

You can find the lists below – please let me know if you find any mistakes (especially in packer sections’ names/attribution); Thanks!

The most popular PE file section names (top 100)

 658574 .rsrc   
 590338 .text   
 545976 .data   
 442607 .rdata  
 298316 .reloc  
 194273         
 178386 .idata  
 111369 .tls    
 109676 CODE    
 105309 DATA    
 100668 BSS     
  40293 UPX0    
  37838 UPX1    
  35164 .adata  
  35020 .bss    
  31336 .edata  
  28137 .ndata  
  15890 .itext  
  15451 .aspack
  12818 INIT    
   9665 UPX2    
   9376 .Upack  
   7727 PS      
   6786 .CRT    
   6628 .vmp0   
   6602 .nsp1   
   6590 .nsp0   
   6560 .code   
   6542 .sdata  
   6423 .nsp2   
   6270 .pdata  
   5710 tldksods
   5462 .       
   5395 Themida
   4313 .vmp1   
   4054 .MaskPE
   3926 PAGE    
   3721 .text-co
   3721 .data-co
   3314 rdata   
   3249 BitArts
   3035 .didata
   2886 idata   
   2881 .packed
   2803   @   @
   2707 .textbss
   2299 .text1  
   2257 .data1  
   2150 .petite
   2079 .texc   
   1926 Shared  
   1793 pebundle
   1714   u     
   1557 MEW F   
   1536 .UPX0   
   1513     t   
   1450 .data2  
   1434 text    
   1346 .RLPack
   1331 .vmp2   
   1300 .ex_cod
   1286 sdt     
   1280 mdata   
   1267 cdata   
   1263 sdata   
   1240 .pklstb
   1238 .MPRESS1
   1235 .MPRESS2
   1204 .UPX1   
   1201 .rdata p
   1191 .brdata
   1183 .udata  
   1131 .crt    
   1114 .sxdata
   1091 htomaota
   1083 .perplex
   1076 PAGEWMI
   1057 edata   
   1044 .delete
   1038 .relo2  
   1031 pec1    
   1015 .mackt  
   1009 PAGEDRV
    981 .svkp   
    980 .avp    
    969 .ByDwing
    967 .DATA   
    963 .debug  
    943 0 ext   
    899 .xdata  
    876 .ccg    
    865 .data ri
    857 .wqvwbj
    857 .kewyo  
    857 .axlgt  
    852 .spack  
    849     ta  
    839 .exc    
    824 .avc    
    807 PAGESYS

The packer/protector section names/keywords

• .aspack – Aspack packer
• .adata – Aspack packer/Armadillo packer
• ASPack – Aspack packer
• .ASPack – ASPAck Protector
• .ccg – CCG Packer (Chinese Packer)
• BitArts – Crunch 2.0 Packer
• DAStub – DAStub Dragon Armor protector
• !EPack – Epack packer
• FSG! – FSG packer (not a section name, but a good identifier)
• kkrunchy – kkrunchy Packer
• .mackt – ImpRec-created section
• .MaskPE – MaskPE Packer
• MEW – MEW packer
• .MPRESS1 – Mpress Packer
• .MPRESS2 – Mpress Packer
• .neolite – Neolite Packer
• .neolit – Neolite Packer
• .nsp1 – NsPack packer
• .nsp0 – NsPack packer
• .nsp2 – NsPack packer
• nsp1 – NsPack packer
• nsp0 – NsPack packer
• nsp2 – NsPack packer
• .packed – - RLPack Packer (first section)
• pebundle – PEBundle Packer
• PEBundle – PEBundle Packer
• PEC2TO – PECompact packer
• PECompact2 – PECompact packer (not a section name, but a good identifier)
• PEC2 – PECompact packer
• pec1 – PECompact packer
• pec2 – PECompact packer
• PEC2MO – PECompact packer
• PELOCKnt – PELock Protector
• .perplex – Perplex PE-Protector
• PESHiELD – PEShield Packer
• .petite – Petite Packer
• ProCrypt – ProCrypt Packer
• .RLPack – RLPack Packer (second section)
• RCryptor – RPCrypt Packer
• .RPCrypt – RPCrypt Packer
• .sforce3 – StarForce Protection
• .spack – Simple Pack (by bagie)
• .svkp – SVKP packer
• Themida – Themida Packer
• .Themida – Themida Packer
• .packed – Unknown Packer
• .Upack – Upack packer
• .ByDwing – Upack Packer
• UPX0 – UPX packer
• UPX1 – UPX packer
• UPX2 – UPX packer
• UPX! – UPX packer
• .UPX0 – UPX Packer
• .UPX1 – UPX Packer
• .UPX2 – UPX Packer
• .vmp0 – VMProtect packer
• .vmp1 – VMProtect packer
• .vmp2 – VMProtect packer
• VProtect – Vprotect Packer
• WinLicen – WinLicense (Themida) Protector
• .WWPACK – WWPACK Packer
• .yP – Y0da Protector
• .y0da – Y0da Protector

List of popular section names

• .arch – Alpha-architecture section
• .bss – Uninitialized Data Section
• .BSS – Uninitialized Data Section
• .code – Code Section
• .cormeta – CLR Metadata Section
• .CRT – Initialized Data Section  (C RunTime)
• .data – Data Section
• .DATA – Data Section
• .data1 – Data Section
• .debug – Debug info Section
• .debug$F – Debug info Section
• .debug$P – Debug info Section
• .debug$S – Debug info Section
• .debug$T – Debug info Section
• .didata – Delay Import Section
• .edata – Export Data Section
• .fasm – FASM flat Section
• .flat – FASM flat Section
• .idata – Initialized Data Section  (Borland)
• .idlsym – IDL Attributes
• .itext – Code Section  (Borland)
• .ndata – Nullsoft Installer section
• .pdata – Exception Handling Functions Section (PDATA records)
• .rdata – Read-only Data Section  (Borland)
• .reloc – Relocations Section
• .rodata – Read-only Data Section
• .rsrc – Resource section
• .sbss – GP-relative Uninitialized Data Section
• .sdata – GP-relative Initialized Data Section
• .srdata – GP-relative Read-only Data Section
• .sxdata – Registered Exception Handlers Section
• .text – Code Section
• .text1 – Code Section
• .textbss – Section used by incremental linking
• .tls – Thread Local Storage Section
• .tls$ – Thread Local Storage Section
• .udata – Uninitialized Data Section
• .vsdata – GP-relative Initialized Data
• .xdata – Exception Information Section
• BSS – Uninitialized Data Section  (Borland)
• CODE – Code Section (Borland)
• DATA – Data Section (Borland)
• edata – Export Data Section
• idata – Initialized Data Section  (C RunTime)
• INIT – INIT section (drivers)
• PAGE – PAGE section (drivers)
• rdata – Read-only Data Section
• sdata – Initialized Data Section