Sysmon doing lines, part 4

Two days ago Mark released a new version of sysmon – version 8.0. It adds new features and addresses the issue I highlighted in my previous post.

This post is not about new version of sysmon though.

It’s more about its inner workings that I looked at a while ago.

Sysmon has two primary components – sysmon driver and sysmon service process (we will skip the architectural differences between sysmon’s x86 vs. x64 versions). The first one (driver) intercepts the events, the second (service process) writes them to the Event Log in a loop, using the DeviceIoControl to talk to the driver.

If you ever patched binary in memory or on disk, you know where it is going…

The core functionality that actually logs the stuff to the Windows Event Logs is called from the inside of the sysmon.exe service process. It’s nothing unusual, but obviously it’s also a potential weakness.

Since it’s a process – it can be patched. I tried it on a file level, and the results of a single byte patch are shown below – not a single event is being logged:

Assuming the attacker detects sysmon, and can acquire the required rights to modify it – patching in memory or on disk is pretty trivial and can basically disable the functionality of the tool…

Obviously, this applies to _any_ tool and any process or component using ReportEvent or EventWrite APIs, so want to reiterate how the availability of tools bias security researchers… Sysmon is used here more as a scapegoat to demonstrate the old-school rootkit technique than a targeted crusade against this fantastic tool.

How to fix it?

Perhaps sysmon could do some sanity checks of its integrity on a driver level? – It’s much harder to modify the driver and load its patched version than an .exe (especially on 64-bit Windows). Perhaps driver could also do occasional checks if the number of Events written to Event Logs is as expected/increasing? And if the sysmon is a dependency, any admin or EDR tool using it should probably verify the integrity of the file prior to launching it? Including automation? I believe there is no generic solution here at the moment, but basic self-checks could simply rely on verifying the signatures of the file (for file patch). For in-memory patches, this is much harder as you need a dedicated code that compares images on disk vs. images  in-memory. You may be surprised, but some EDR solutions actually attempt to do that – it’s often much better than then ‘see-it-all’ approach that generates lots of noise.

Going back to rootkits – I used this term on purpose. Applying a single-byte patch will give you zero logging. It’s silly and easily detectable. A more complex patch could simply rootkit the bad guys activities out of the event log (just need to hook either the sysmon itself, or Event reporting APIs). As usual… whoever gets there first… wins.

Sysmon doing lines, part 3

Update

This issue was fixed by Mark Russinovich on 2018-07-06; that was pretty quick!

Old

Sysmon is an easy target, because it’s easily downloadable and everyone can poke around in its code or toy around with the system and see what sysmon logs. It’s obviously not fair – if other EDR code was that easily available I am pretty sure we would see a cascade of ‘funny stuff’ in these products as well.

Anyway…

In my older post I presented a simple technique that may fool parsers and their state machines into ‘thinking’ they are parsing correct records while in fact they are processing data some malicious software meticulously crafted for them. This is not necessarily sysmon’s problem, but who would read that old post if there was no clickbait value in the title, right?

Back to sysmon and poking around… once you start looking at it you can quickly discover that it can be run in a so-called debug mode – all we have to do is provide an undocumented command line switch ‘-t’ when we install it. When I first discovered it I got really excited, only to immediately get a bucket of cold water thrown by the Twitter post by @mattifestation who figured it out in… Jan 2018.

It’s a really cool feature.

When you run ‘sysmon -t -i’ the program will start throwing a lot messages to the console and some of them will eventually trigger your interest. Especially if you ‘help’ them a bit to appear 🙂

So… what we see in this error message is a very crucial information.

The sysmon had to truncate a very long command line which I have provided to a test process. It was so long that it had to be trimmed.

A-ha… but how long?

Well, it turns out sysmon doesn’t like command line longer than 0x2000 characters – i.e. this a number of wide characters it can swallow, before trimming down the rest.

Now this 0x2000 (Wide characters) is actually 16384 bytes only.

I was curious where the 0x2000 came from, because after reading various versions of MSDN pages about CreateProcess I know very well that the lpCommandLine argument can be much longer; as per the MSDN:

The maximum length of this string is 32,768 characters,
including the Unicode terminating null character.

So… this is an interesting discrepancy.

I have a hypothesis (and I am totally guessing it) that the sysmon author used the arbitrary limit imposed on cmd.exe command line arguments.

Such discrepancy is a nice gift and we can of course abuse it.

Since we can’t pass the command line arguments that are longer than 0x2000 characters to cmd.exe let’s try to use powershell instead.

If you run ‘powershell <0x2000 spaces> calc’ you will spawn Windows Calculator.

What will you see in the logs?

This:

And if you export it to TXT or XML you will get this:

So… using long command line arguments provided to executables that can work with such madness (e.g. powershell) can help to evade sysmon logs…

If you want to test it, grab this .exe.