Mitre Att&ck – from JSON to CSV

I love JSON-formatted data so much that… anytime I see something valuable stored in this format I really can’t resist the temptation of converting it to CSV so that I can actually browse it and/or visually understand/analyze some of it 🙂

I know, I am old-school 🙂

Mitre Att&ck JSON file is a maverick on its own. Updated on regular basis, it contains so much cyberjuice that it is almost a crime not to convert it to CSV, at least some of it 😀

How do we go about it?

This is one of the ways… we use the following script with the input being latest version of the Mitre Att&ck JSON file. The file’s comments include the actual link….

Run it, and see if you like its output… Note that it uses lots of Boolean (0 or 1) output values in many of its columns – this is by design – these may help you to filter the data in Excel or Google Sheets as per your need…

And YES, I know there is jq, I know there are JSON readers/viewers/beautifiers, and I recently learned of pyattck, too. Still, when it comes to data analysis, I really like to keep my options open but also keep them close and control them a bit…

Perl and Python Scripting Templates…

One of the most important (basic) technical skills in cybersecurity are:

  • Knowing Excel (or Google sheets)
  • Knowing basic programming/scripting (bash, cmd, powershell, vbs, vba, autoit, python, perl, etc.)
  • Knowing and staying up to date with tools

I covered item #1 a few times.

I did cover #2 to some extent as well, but I’d like to expand on it today.

And #3 is your kinda FOMO at work – there are way too many projects/tools available today to know-them-all, but the more you know of more of them, the easier your job will become. As in, for almost every single cyber/hacking/reversing idea you can think of, there is someone, somewhere who has not only already thought of it before, but also implemented some cool tool, PoC, etc. I will go as far as to saying… tool and ideas foraging is one of the most important cyber skills today. Taking shortcuts, effectively using what is already out there is the ‘street-savvy’ cyber skill equivalent A.D. 2023.

Now, using tools is cool, but sometimes, and often really… we still need to do some work ourselves. This is why today I will focus on the #2… Just… a bit more optimized.

I can’t count how many times over last 2 decades I was in a need to write a simple script that would take a directory or a filename as an input, and then would do some quick processing of the files found inside that given directory (recursively), or on that specific given file, and then would spit out the results.

After doing the same repetitive work of coding the same routines over and over again I finally decided that I need some sort of a template. And I have developed one that I now use for quick&dirty processing of ‘many files of some kind’ on regular basis, and where the basic logic of enumerating the directory, checking the file extensions, their size, etc is already built-in. And anytime I re-use it, I simply mod the logic of that template to my needs, f.ex. use the right file-reading routine (f.ex. read as a single binary blob, or line-by-line), use appropriate character-encoding (ANSI, UTF-8, UTF-16, etc.), and then do some data processing (extract lines of interest, decrypt some data, etc.), and finally – spit out the results to the console.

I must admit that I used perl template for this sort of quick&dirty, case-by-case bulk file parsing solutions for many years. It actually worked like a charm, and I have used improved variants of the main template on web logs, executables, quarantine files, clusters of unknown files that needed classifying , etc. but eventually, with the whole world turning into Python over last decade, I developed a template for it as well.

Here they are:

If you find it useful, if you think I should add more code to any of these, please let me know. Thanks!