You are browsing the archive for Uncategorized.

I present you Splunk panel data

May 27, 2020 in Uncategorized

I never liked dashboards so it took me a long time to get convinced that there is a lot of value in them. There are at least three reasons why I didn’t like them, and I think it’s important to highlight them here as it is as well may be that these reasons are why you are not the biggest fan of them either.

My reasons are:

  • too much data (too much data)
  • data is often presented in a way that is not actionable (too many fields, presentation layer doesn’t help analysts)
  • UI over-engineering (pew pew maps, large font, etc. lots of stuff stealing precious space)

As I progressed with my Splunk experience and under a good influence of other expert splunkers, today I look at dashboards with a much friendly eye…

My approach changed so much that I reserve alerting to high-fidelity stuff only; something that is important to justify triggering an alert, even email sent. For the other stuff, one that is typical threat hunting business as usual I just use panels.

When you plan an actionable panel one thing that stands out immediately is the fact that data we present on these mini-canvas is really hard to squeeze in there in a first place. Moreso, it has to fight for space with many other panels and as a result, badly designed panels make badly designed dashboards and in the end, no one wants to look at them.

Let’s look at how we can declutter it all.

Data reduction

I have covered LFO and normalization in my other post. Normalize, remove repetitions, and keep the least possible number of outliers.

Too many fields

Many splunkers are obsessed with presenting every single field on their panels. I am wondering why. The role of a threat hunting dashboard is to help analysts with eyeballing large amount of data, so it is not even triage stage. We are just looking for anything that stands out, a needle that will trigger the actual triage. Less data on the screen doesn’t make this activity any less actionable. We often don’t need to show time, full paths, all hosts where the file was found, its file size, or file attributes, and all the other gore details that are present in logs. You also don’t need a field with a Mitre Att&ck tactic & technique name either. Less is more. Depending on your panel, it may as well be just a file name, a domain name, or a data transfer volumes. Avoid enriching data, unless you have space for that.

Very long fields

This typically applies to urls, very long process names, and incredibly long command line arguments, often observed to be used by java.exe, chrome.exe processes.

The trick number one is to completely exclude these that are very long; this is risky though as malware could use the very same trick.

Another approach is to analyze command line arguments and normalize them. So, yes normalization is very helpful not only for LFO, but also for actual presentation.

You can replace command line arguments with place holders f.ex.:

foo.exe -pid 0x1234 -session 0x798789da9 arg1

could become:

foo.exe -pid <hex> -session <hex> arg1

You can also go a step forward and remove all known placeholders. That is, first normalize, then remove individual tuples of known argument names and their placeholder values. For example shown above it would leave us with a much shorter version to look at:

foo.exe arg1

That result really saved us a lot of space and… didn’t make the data any worse. In fact, we saved not only space, but also time needed to eyeball this chunk of information.

And yes, you could go even further and use the replace function and remove _all_ arguments no matter what name as long as they conform to certain command line regex pattern e.g..


You could apply similar logic to url variables (separated by ‘&’ and in a form of variable=value).

Yet another take is to shorten the actual content. First we calculate length of reported data. If it is shorter that we care about and fits nicely on a panel, we just do nothing. If it is too long though, there are at least two way to deal with it:

  • use substr to truncate the string (we will only see its prefix)
    • longstring becomes longstr…
  • use substr twice and get smaller chunks from beginning and end of the string (we will see its prefix and suffix)
    • longstring becomes lon…ing

And finally… for certain data types, you can shorten them by:

  • extracting only important features e.g. extension from a file name, file name from a full path, domain from an url, a subset of variables from url, etc.
  • normalizing longer path chunks into made-up, shorter placeholders e.g. c:\windows\system32 –> %sys32%
  • removing superfluous suffixes e.g. TLD, SLD, or even the whole name of domain if it is repetitive

Too many rows of data

We don’t always need to show all rows e.g. multiple host names, account names, command lines, etc. A very handy snippet shown below saves us a lot of space, and also informs us that there is more data:

| eval field=if(mvcount(field)<4,field,mvappend(mvindex(field,0,3),"…"))

It extracts first 3 rows from a multivalue data and truncates the rest.

Consistency of UI metaphore

Data enrichment is easy to spot. It’s just there.

Data reduction/depletion doesn’t manifest itself in any way, until we tell the analysts that it happened. In both cases above you may notice that any data that is truncated is always substituted with ‘…’. This bit acts as a hint for analysts that in order to see the whole data set they need to remove the limitation and run the query in a separate Splunk window.

Windows 10 is ‘mine’…, Part 1

May 5, 2020 in Malware Analysis, Reversing, Tips & Tricks, Uncategorized

I don’t like Windows 10, but it likes… the progress…

So… now that win7 is ded, and winxp doesn’t work that well for malware analysis (and it’s 32-bit only), I finally (a few months back really) put myself together to build my perfect test guest 64-bit Windows 10 OS… and while doing so I came across a lot of quirks, took some screenshots, and I thought I will jot down some notes here in case you face similar issues…

Note, I am a big fan of VMWare, so the info below is primarily focused on VMWare Workstation… VirtualBox experience should not be too far off though… I hope….

Here are the steps I took to make my Windows 10 Guest OS perfect (to be clear, I followed many of these steps on my host Windows 10 as well):

  • Install to SSD
    • I bought my first SSD circa 2011 and never looked back; this an incredible performance booster and you need it for your frequently used VM guests!!!
    • Assume your SSD will go kaput on you at any time around 3-5 years down the line, so make regular backups
  • Install the Windows 10 OS; whether you go from a clean ISO, or upgrade your old Win7/Win8 it doesn’t matter
  • Go through the wizard…
  • Choose your OS version and continue until you install the whole thing
  • Note: do not use Microsoft accounts if asked, only the local one!!!
  • Decline all the privacy/spying Options:
  • Go on…
  • After 1-2 restarts you should have a clean OS installed
  • It’s time to install VM Tools:
  • You may need to run the VMTools setup64.exe manually from a mounted DVD:
  • Restart

At this stage you have the OS installed and VM Tools are running – SAVE THE VM SNAPSHOT NOW. If anything goes wrong, you can revert to it.

The VM tools allows you to change screen resolution and copy & paste between the host and the guest system, as well as access the network shares.


But it’s just the beginning…

  • Download and run O&O ShutUp10. Choose all options aka ‘Apply all settings’. Yup, make it all green:
  • You will need to restart the system after applying the changes
  • Now…
  • OS is installed, the basic nuisance is gone, but it’s not over yet.
  • Download Total Commander 64-bit version (TC) from
    • Run TC as Admin
    • Now you can do anything you like on the system and have a better Program/File Manager than Explorer will ever be
    • Hope you have a Total Commander license, it’s worth it!
  • Now download psexec
    • Run psexec -s -i cmd.exe from your elevated cmd.exe (admin)
    • Now you have a terminal under SYSTEM account
    • Launch Regedit.exe
    • Go to Computer\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services
    • Yeaah…
    • Time to kill annoying services:
      • You have two options: be a good boy, or a bad boy; choose the latter….
      • aka….
        • Walk through all of them; I know it’s painful, but…
        • If you don’t like the particular service, just mod its ‘Start’ entry to become ‘4’ which is an equivalent of ‘Disabled’
        • I know for some options you can run gpedit.msc and select ‘disable service’ options under admin templates, but well… these will in the end run services. Since you just want to kill the nuisance, kill it at source i.e. this is right under Services key… Be brutal… Windows 10 is a telemetry and nuisance virus and you need to make it work like kinda enhanced Windows 7.
        • In particular, disable Windows Update, Windows Defender, MS Store, Security Health services and Search indexing
        • Be aware that disabling all this you will lose updating capability –> snapshots are your friends !!!
  • Time to customize your UI
    • I personally prefer good ALT-TAB with icons, so I add
      • HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\AltTabSettings=1
    • I kill Cortana bar (remove from view), manually
  • I lock the Taskbar
  • I make all notification icons to be always visible
  • I choose ‘Never Combine taskbar tools’
  • And then install tools…

Part 2 to follow…