Sysmon doing lines, part 3

Update

This issue was fixed by Mark Russinovich on 2018-07-06; that was pretty quick!

Old

Sysmon is an easy target, because it’s easily downloadable and everyone can poke around in its code or toy around with the system and see what sysmon logs. It’s obviously not fair – if other EDR code was that easily available I am pretty sure we would see a cascade of ‘funny stuff’ in these products as well.

Anyway…

In my older post I presented a simple technique that may fool parsers and their state machines into ‘thinking’ they are parsing correct records while in fact they are processing data some malicious software meticulously crafted for them. This is not necessarily sysmon’s problem, but who would read that old post if there was no clickbait value in the title, right?

Back to sysmon and poking around… once you start looking at it you can quickly discover that it can be run in a so-called debug mode – all we have to do is provide an undocumented command line switch ‘-t’ when we install it. When I first discovered it I got really excited, only to immediately get a bucket of cold water thrown by the Twitter post by @mattifestation who figured it out in… Jan 2018.

It’s a really cool feature.

When you run ‘sysmon -t -i’ the program will start throwing a lot messages to the console and some of them will eventually trigger your interest. Especially if you ‘help’ them a bit to appear 🙂

So… what we see in this error message is a very crucial information.

The sysmon had to truncate a very long command line which I have provided to a test process. It was so long that it had to be trimmed.

A-ha… but how long?

Well, it turns out sysmon doesn’t like command line longer than 0x2000 characters – i.e. this a number of wide characters it can swallow, before trimming down the rest.

Now this 0x2000 (Wide characters) is actually 16384 bytes only.

I was curious where the 0x2000 came from, because after reading various versions of MSDN pages about CreateProcess I know very well that the lpCommandLine argument can be much longer; as per the MSDN:

The maximum length of this string is 32,768 characters,
including the Unicode terminating null character.

So… this is an interesting discrepancy.

I have a hypothesis (and I am totally guessing it) that the sysmon author used the arbitrary limit imposed on cmd.exe command line arguments.

Such discrepancy is a nice gift and we can of course abuse it.

Since we can’t pass the command line arguments that are longer than 0x2000 characters to cmd.exe let’s try to use powershell instead.

If you run ‘powershell <0x2000 spaces> calc’ you will spawn Windows Calculator.

What will you see in the logs?

This:

And if you export it to TXT or XML you will get this:

So… using long command line arguments provided to executables that can work with such madness (e.g. powershell) can help to evade sysmon logs…

If you want to test it, grab this .exe.

Logman, the Windows volverine

Update

Coincidentally, @HECFBlog @nicoleibrahim released a tool ETL Parser that helps to at last extract raw buffers and as a result it produces a CSV file with data that can be analyzed row by row. This is the closest so far that I have seen for a tool to be able to help analyze the output of api trace log. Have a look at the screenshot:

Old Post

Almost everyone is excited about .etl files that are produced as a result of Event Tracing for Windows.

Almost, because not me.

This feature is so rich in … features. Except it’s only for MS internal consumption. At least, it seems to be the case in some very really interesting… well… cases…

It all started when I looked at the logman tool command line arguments; one that immediately caught my attention was this one:

logman create api

Aha… A basic API Monitor!

But not only that. It allows to run API Monitoring across different systems!!!

Oh… this could be a nice covert channel to transfer data between systems. Write one piece that constantly calls a monitored API and feeds it with the data to transfer, and then – on the other end – collect this data on the other system using logman tool. You could definitely implement it in a duplex set-up too.

Simple.

So I thought.

First, I immediately tested the ‘create api’ bit it and it worked like a charm:

logman create api foo -f bincirc -exe c:\windows\notepad.exe -o c:\test\notepad.etl

logman start foo

Now you have to simply run Notepad.exe (c:\windows\notepad.exe) and the .etl log will be created&updated as long as notepad.exe runs; the example content of the binary .etl file is as follows:

You can clearly see the APIs being recorded when Notepad is running:

  • RegisterClassW with the Unicode class ‘OleMainThreadWndClass’
  • GetAppCompatFlags2
  • IsProcessDPIAware
  • GdiReleaseDC
  • etc.

The only problem remaining:
– how to convert this binary blob into something more readable?

After googling around, I discovered that the tool basically embraces the api tracng functionality offered by the IApiTracingDataCollector interface. When you read into the docs of this interface you will soon realize that logman’s command line argument ‘-f’ that is supposed to determine the file type for the output file can be only… binary. So any attempts to specify TSV, or CSV will fail 🙁

Okay.

Since I couldn’t bite it from this end, I started looking at Windows Performance Analyzer (WPA) and other tracing and trace conversion tools.

No matter which one I used, I could only get a basic flow of the events, but w/o these interesting gore details of each API call:

It’s all nice and eye candy, but the problem is already visible:
– if you don’t know how to parse, you simply don’t.

As a last resort I tried ‘tracerpt’:

tracerpt <log> -o <output> -of CSV

.. and it provided a similar enigmatic output:

Some more googling around and now I know I need to get a .PDB file for the api trace provider, or some other schema files that will allow WPA to parse it properly…

Sigh…

The provider’s GUID is 7535aef9-4a3b-482b-91eb-25db0299995d.

It is nowhere to be found.

Eventually I gave up & I asked on Twitter.

Unfortunately, Alex Ionescu, and Zac Brown are both pessimistic 🙁 Still, I am grateful to them for chipping in and providing the useful feedback – if they can’t do it, I guess we explored all the ‘normal’ possibilities.

But then I am thinking… so… if it is an internal schema, then there is only one way out.

Brute-Force attack.

I may cover it in the next post.

From the blue team’s perspective, if you want to detect logman’s activity, in particular api tracing, you may be interested in this process tree:

The example command line arguments from my VM test run are as follows:

  • C:\Windows\system32\svchost.exe -k netsvcs
    • taskeng.exe {A7B77AA7-00E9-45C4-92C5-31C3868DB30D} S-1-5-18:NT AUTHORITY\System:Service:
      • C:\Windows\system32\rundll32.exe C:\Windows\system32\pla.dll,PlaHost “foo” “0x934_0x91c_0x2a045dff88a”

One thing is also worth mentioning – many logman command line option talk about running some task when some event happens; at first I thought it may be a new persistence mechanism, until I realized the tasks names required by these commands are task names registered using a Task Scheduler. So… yet another fail on the documentation, as it is very cryptic and talks about these various traces in a very superficial way 🙁