Perl and Python Scripting Templates…

One of the most important (basic) technical skills in cybersecurity are:

  • Knowing Excel (or Google sheets)
  • Knowing basic programming/scripting (bash, cmd, powershell, vbs, vba, autoit, python, perl, etc.)
  • Knowing and staying up to date with tools

I covered item #1 a few times.

I did cover #2 to some extent as well, but I’d like to expand on it today.

And #3 is your kinda FOMO at work – there are way too many projects/tools available today to know-them-all, but the more you know of more of them, the easier your job will become. As in, for almost every single cyber/hacking/reversing idea you can think of, there is someone, somewhere who has not only already thought of it before, but also implemented some cool tool, PoC, etc. I will go as far as to saying… tool and ideas foraging is one of the most important cyber skills today. Taking shortcuts, effectively using what is already out there is the ‘street-savvy’ cyber skill equivalent A.D. 2023.

Now, using tools is cool, but sometimes, and often really… we still need to do some work ourselves. This is why today I will focus on the #2… Just… a bit more optimized.

I can’t count how many times over last 2 decades I was in a need to write a simple script that would take a directory or a filename as an input, and then would do some quick processing of the files found inside that given directory (recursively), or on that specific given file, and then would spit out the results.

After doing the same repetitive work of coding the same routines over and over again I finally decided that I need some sort of a template. And I have developed one that I now use for quick&dirty processing of ‘many files of some kind’ on regular basis, and where the basic logic of enumerating the directory, checking the file extensions, their size, etc is already built-in. And anytime I re-use it, I simply mod the logic of that template to my needs, f.ex. use the right file-reading routine (f.ex. read as a single binary blob, or line-by-line), use appropriate character-encoding (ANSI, UTF-8, UTF-16, etc.), and then do some data processing (extract lines of interest, decrypt some data, etc.), and finally – spit out the results to the console.

I must admit that I used perl template for this sort of quick&dirty, case-by-case bulk file parsing solutions for many years. It actually worked like a charm, and I have used improved variants of the main template on web logs, executables, quarantine files, clusters of unknown files that needed classifying , etc. but eventually, with the whole world turning into Python over last decade, I developed a template for it as well.

Here they are:

If you find it useful, if you think I should add more code to any of these, please let me know. Thanks!

DeXRAY, DFIR, and the art of ambulance chasing…

Pretty much all of my DeXRAY posts ever published been focusing on new versions of this tool being released. Today I will talk about the ‘making of the sausages’ part of this process, aka how DeXRAY came to be.

If you have been working in a DFIR space for more than a decade you probably already know that any type of high-fidelity evidence found on an endpoint is gold, and Quarantine folders/files are one of the best in this category… These are locations where security software stores intercepted/blocked/quarantined files. Before the strong adoption of NextGen, EDR the AV products used to catch many malware files ‘just in time’, then encrypt their content, often move them to a ‘special safe location’, and delete them from their original location. And yes, these encrypted files (most of the time) can be decrypted by DeXRAY….

Despite this informative intro you may still ask… why do we even need talk about Quarantine files and folders today?

First of all, I believe not every DFIR analyst is aware of these file system locations. And as the time goes by, probably less and less of them, as well. It’s a knowledge of the past, after all. Moreso, in a world of ever-changing landscape that is affecting not only the actual threats, but also security solutions, it’s not uncommon for the following events to occur:

  • multiple security solutions installed on the same host, plus
    • installation of one, doesn’t imply the older one was (fully) uninstalled! that is, there may be remnants of the old security solution still present on the system, not only the program binaries, configuration, but also quarantined files!
  • different polices used by these security solutions may cause interesting interference (f.ex. exclusion policies for directories/files in one may suppress some detections, but still trigger other detections in another solution)
  • some DFIR analysts can actually miss an opportunity to discover these existing quarantined files, because they simply don’t know about them!

So, if you want to improve your chances of detecting something interesting on the endpoint you investigate, this post is for you.

And yes, we are ambulance chasing, but for a good reason! Discovering that someone else (meaning: some other software) had discovered something before us is actually NOT A BAD THING. I would go as far as to say that while discovering and analysing quarantined files is being a bit of cheating, it may actually cut down a lot of analysis time in some cases. And in the DFIR world, time is really of essence.

The ambulance chasing rule #1 is that when you process your evidence, make sure you pay attention to these low-hanging fruits and nuggets…

Before I go into gore details, let me digress to deliver a personal rant: analysing paths where security software stores its quarantined files is not easy in 2010s/2020s. It requires a lot of patience, plus some more. The security solutions of ‘today’ migrated away from the golden era of 90s/2000s. Big time. Yup, while in the past you would download the software and just install it, today you can’t install anything w/o creating an online account at least, and/or (pre- or) paying for a subscription, even if just for a test period (credit card authorizations). So, if you want to try yourself – you have been warned: I went through hell of doing it for many security solutions and do not recommend. For realz, you are going to be exposed to a lot of b/s and ‘I really don’t wanna do it’ equilibristics. Plus, some solutions use consoles that are no longer present on the client side (endpoint) either, and have been moved to the server-side, so you will actually need these b/s online accounts — yes, the temp emails, phone numbers won’t cut it. Let me be blunt and say it’s actually quite an experience to install many of the security software packages of today w/o getting seriously pissed off… Now, imagine you are that damsel in distress, you know nothing about security, but you suspect you got hit by some malware/hacking attacks and want to purchase a security solution to help you with your problem. I am feeling very very sorry for you in 2023… Anyway… this is the end of the rant 🙂

The good news is that from a forensic investigators’ perspective, these solutions have already been (pre)installed on the systems you analyze. As such, we just need to find these quarantined folders/files!

Here are the rules:

  • If part of the directory / folder refers to ‘/.*?Quarantine/’ — check it!
  • If part of the directory / folder refers to ‘/chest/’ — check it!
  • If part of the directory / folder refers to ‘/QB/’ — check it!
  • If part of the directory / folder refers to ‘/Infected/’ — check it!
  • If part of the directory / folder refers to ‘/Backup/’ — check it!
  • If part of the directory / folder refers to ‘/$360Section/’ — check it!
  • If part of the directory / folder refers to ‘/fq/’ — check it!
  • If part of the directory / folder refers to ‘/qv/’ — check it!
  • If part of the directory / folder refers to ‘/Jail/’ — check it!
  • If part of the directory / folder refers to ‘/Safestore/’ — check it!
  • if the file extension is one of these
    • ‘.v3b’, ‘.eqf’, ‘.qua’, ‘.qv’, ‘.bdq’, ‘.q’, ‘.cmc’, ‘.vir’, ‘.ifc’, ‘.nqf’, ‘.tmp’ (with a header ‘KSS’), ‘.klq’, ‘.qnt’, ‘.bin’ (with a file name being a hash), ‘.lqf’, ‘.quar’, ‘.data’, ‘.bup’, ‘.mal’, ‘.exv’, ‘.dlv’, ‘.virus’, ‘.infected’, ‘.malware’, ‘.suspicious’, ‘.sdb’, ‘.qbd’, ‘.qbi’, ‘.idx’, ‘.qtn’, ‘.vbn’, ‘quarantine.db’ — check it !!!

I’d love to say – you see? it’s that simple. Yet, I know it is not. Still… happy ambulance chasing!

There you have it. It was that easy.