A lot of people are sharing their Yara creation (look for #100DaysofYARA tag on Twitter), so I thought I will share a bit too.
This is a very unusual way of using Yara and I hope you will find it interesting.
When we think of Yara rules we usually have very specific cluster of strings in mind – formed by be it an API, a debug string, a snippet of code, etc. What if instead we used yara to scan files for much large sets of strings? While it may sound counterintuitive, Yara is really very well prepared to do “carpet bombing” string scans on target files. It’s actually super fast and efficient.
Let’s have a look at an example.
Imagine that you want to find all English words inside a file. I choose “English” because it’s easy to demo, but you could use any other language really. The traditional approach would rely on running the “strings” tool over the target file and then manually combing through the results, cherry-picking words that “look” English. For other languages you may need a localized version of “strings” tool (e.g. my old tool hstrings could help), but the principle is the same. In some cases you could also apply knowledge of file structure so that could extract some of the strings ‘natively’ (e.g. from resources in PE file).
We can also approach it from a different angle. We will build a list of all English words and then search for them in the file. All at once. There are obvious caveats – we can never sure we have a list of all English words e.g. gobbledygook or ragamuffin may not be on the list, and short words will certainly be causing a lot of False Positives, but it’s just a POC of an idea.
So, we find a random English words list. We write a small script to extract all 6+ character long strings and exclude strings starting with digits and we then convert it into a set of Yara rules. Yara accepts up to 10K strings per rule so we have to split the dictionary into multiple rules.
my $cnt=0;
my $n=0;
while (<>)
{
s/[\r\n]+//g;
next if length($_)<6;
next if /^[0-9]/;
s/\"/\\"/g;
if ($n==0)
{
print "
rule ".sprintf("eng_%04d", $cnt)."
{
strings:
";
}
print "\$ = \"$_\" ascii wide nocase\n";
$n++;
if ($n>9999)
{
$cnt++;
$n=0;
print "
condition:
any of them
}
";
}
}
print "condition:
any of them
}
";
The resulting rules can be saved into eng.yar file and then compiled with yarac to eng.yac:
yarac eng.yar eng.yac
We will get a lot of warnings about the rule slowing down the scanning, but who cares 🙂
warning: rule "eng_0000" in eng.yar(10008): rule is slowing down scanning warning: rule "eng_0001" in eng.yar(20016): rule is slowing down scanning warning: rule "eng_0002" in eng.yar(30024): rule is slowing down scanning warning: rule "eng_0003" in eng.yar(40032): rule is slowing down scanning warning: rule "eng_0004" in eng.yar(50040): rule is slowing down scanning ...
Note, the resulting file is gigantic – ~600MB in size. You can reduce is by mingling with “ascii wide nocase” sets (if you exclude them, the file will be only ~70MB).
We can now use the rules on e.g. Notepad:
yara -s -C eng.yac c:\windows\notepad.exe
-s – will extract strings
-C – will tell yara the rules are compiled
The results will look like this:
eng_0000 c:\windows\notepad.exe 0x280b5:$: Accelerator 0x2822a:$: Accelerator 0x2822a:$: Accelerators 0x26f00:$: Accept 0x2b9bd:$: Access 0x2862e:$: Acquire 0x286f6:$: Acquire 0x2862e:$: AcquireS 0x286f6:$: AcquireS 0x28df9:$: Activation 0x27faf:$: Active 0x28e04:$: actory 0x286d9:$: Address 0x289c5:$: alLock 0x28b68:$: alLock eng_0001 c:\windows\notepad.exe 0x25050:$: A\x00p\x00p\x00l\x00i\x00c\x00a\x00t\x00i\x00o\x00n\x00 0x25260:$: A\x00p\x00p\x00l\x00i\x00c\x00a\x00t\x00i\x00o\x00n\x00 0x2ba0f:$: application 0x2baf4:$: application eng_0002 c:\windows\notepad.exe 0x2b75e:$: Archit 0x2b88f:$: Archit 0x2b75e:$: Architect 0x2b88f:$: Architect 0x2b75e:$: Architecture 0x2b88f:$: Architecture 0x227ba:$: A\x00r\x00o\x00u\x00n\x00d\x00 0x2b6c8:$: assembl 0x2b713:$: assembl 0x2b7e5:$: Assembl 0x2b7f9:$: assembl [...]