June 24, 2016 in Hexacorn
The insects that are similar to a common housefly, except a bit nastier. Cuz they offer a pain of a retrospection 🙂
I dedicate this post mainly to younger readers who dream of opening their own company as I will share some lessons learned from the experience of having one. Who knows, maybe it will be helpful to someone.
- First of all, Hexacorn is a real (‘registered’) company.
- Secondly, my wife and I run it as a part-time hobby/job.
- Thirdly, it is profitable. Not in Richard Branson way, but still… it paid for some nice holidays 🙂 and the Hex-Rays licenses 🙂
The company needs to be a real deal. If you want your own, start it as a part-time job (unless you are young, and/or rich and into startups)!
Back in 2009 I wanted to open my own company in UK. We lived there back then and I dreamed of an opportunity to do some work from home using some free cycles I had.
My wife and I kept thinking of a name and eventually she came up with ‘hexacorn’ – a neologism or a portmanteau really. The idea being that it’s related to ‘hexa’decimal, and – obviously – uni’corn’s: representing or symbolizing (probably in a certainly pretentious way) the indescribable property of grace, beauty, awe associated with activities related to digging into hexadecimal code and data… be it reversing, programming, forensics, data coding, decoding, data compressing, decompressing, patterns of any sort, entropy, conversions, transformations, visualizations, hacks of any sort, file formats, logs, sandboxing, statistics, packets, bytes, nibbles, bits… basically, anything that makes us all in the DFIR/RCE world tick 🙂
It’s sometimes called a bug for solving puzzles, investigations or analysis, but fundamentally… a wish to break things apart for the sake of it and pretend that we are actually building something :).
Just kidding – the real pleasure is the moment you understand ‘what happened on this system’ or ‘how this piece of (malicious, but not only) software works’.
It’s a moment of an enlightenment. And very software focused in my case.
Back to the company. We bought the domain, talked about it for a while, and… we forgot about it cuz I was thinking of changing the job 🙂
I eventually did change it, and in the early 2011 we moved to Hong Kong. After few months the idea to open the company came back, the domain was already there, so the only thing left was to actually register the company.
Luckily, it’s not my first company I was the owner of (not boasting here, it’s just easier if you have a basic idea how it works and what you need to actually start), so I more or less knew what to expect, but it turned out that Hong Kong is very supportive when it comes to opening the company. It’s a simple, very straightforward process, and not that expensive. If you need any advice, please ping me offline.
Once the company was registered, we started waiting for the first customers to appear.
We kept waiting.
After a few months, our mood sunk and we started to give up.
Eventually I engaged in some sort of ‘marketing’ activities that relied on hard selling my services to a couple of old friends and strangers I could find on Linkedin, and random folk on the intertubes.
Still, no results.
This company thing just didn’t want to fly… 🙁
Registering company != making money. When you are focused on binary, it’s hard for you to be a salesman as well! 🙂
In the mean time, my wife got a job as a B2B and she started making our first company money. This continued until we left Hong Kong this year, and it was great to see the constant flow of money coming from that single account for a couple of years.
If you run a company, it’s good to have an account that is long lasting.
The concept of an account may be scary, but the reality is that you need to make money. A client paying every month (regularly!) is not only ‘nice to have’. It is a must.
On my side, after giving up a bit I got my first ‘hexacorn’ job from a friend who I believe is one of the best in the DFIR industry (he is just keeping a very low profile) and who happens to have his own company – he needed some urgent malware analysis…
I delivered, then raised my first invoice.
The best first customers are most likely people you already know. They may not know you are now a very important businessman. Until you tell them.
This relationship continues till today and I am always looking into that guy’s masochistic leadership when it comes to comparison ‘who works harder’ :). He beats me. Every single time :-O.
And no, after my first invoice I didn’t buy Rolls-Royce, or a Jet yet :), but it was very encouraging…
Imagine… someone needed my help & was eager to pay for it!!
A few months passed, I got a few more gigs from other friends as well… and started making some little money.
Eventually, we closed the fiscal year with a small profit!
It felt great!
Once you start making some money, it feels really satisfying
I must mention here that the cost of running a company on a side of a regular job is not low.
I spent a lot of time working at night, and often staying really late to get stuff done quickly, but at the same time at the cost of being a real zombie next day in the office. Being the otaku type didn’t help and I most likely estranged most of the people I worked with…
Next, when you close the fiscal year… there are costs associated with it. When you register a company you need to do things according to the law, open a business account at the bank, pay for the address /virtual office/, hire accountant, do audit every year, pay annual fee for registration in the Company Registry, and it simply costs… then there is a corporate tax as well, plus if you pay yourself some salary, you need to fill-in a lot of forms for revenue, insurance, pay personal tax, pay for pension, adjust personal tax, etc., then visit various offices every once in a while, and also collect forwarded snail mail, file invoices, vouchers, put them in a chronological order, potentially scan them, and basically… spend a significant number of hours doing the boring admin work!
Yup. It does take hours and days! Do not underestimate the time it takes…
So, if you start working independently you need to calculate your consulting fees taking the above into account. When you work for the employer, all these little ‘big’ things are sorted by ‘the other guys’ (other departments). Here, you represent a small company and you have to do it yourself… Or, you face fines, and potentially go to the court 🙂
Not fun if you only want to do binary!!!
Working on your own, even not full-time is a hell of a challenge… time + formalities != fun
The money aspect is important, but the enjoyment of actually doing work as your own boss is quite intoxicating. The time that I didn’t work I invested heavily in developing this blog. Nearly 5 years after I started it (Oct 2011), it is 250+ posts, most of them trying to talk about stuff that (I believe) has never been published before, not necessarily very in-depth, but covering lots of ideas from my [censored] years of toying around with ‘binary’, and data that is often not shared publicly – probably because of their perceived value, or there was no research (at least public) in this space at all (f.ex. list of popular mutexes, atoms, events, etc.).
As I say, it’s nothing mind blowing, but the fact I was able to produce so much means actually quite a lot to me. Just imagine, thinking of something, then developing it, is really different from just talking about it. It takes a lot of time, and I can’t even count how many hours the development of this blog took so far. Believe or not, for some posts, even if appearing rough and just providing relatively short/stripped down data dumps it takes _years_ to gather this data for the release! Let alone manual editing to exclude the noise of any sort. For example, being a small shop I can’t afford to run a cluster of sandboxes, so I run it all on a single machine! It’s running for 6 years in a row as we speak… And how many things I didn’t publish, because research hours went to /dev/null. Or, because I made a stupid assumption and researched the topic from a wrong angle. And this is not an act of self-appreciation or depreciation here, it’s a hard reality that “thinking” research, and actually converting it into “doing” research, and then writing about it is a very very time-consuming activity. This is why more and more research-hungry guys can only do research for vendors. They need a cozy atmosphere where their research is embraced, encouraged, and actually paid for + they have much bigger resources at hand. Big Data FTW.
When you do it yourself, it just takes away the ‘free time’ you have+costs significant money to maintain (I work with ~60 TB of data scattered amongst a large number of external drives that I had to pay for from my own pocket). Again, not to brag, more to inform that when you are into it – do it for real + actually spend the money. If you are doing it as a part time, it’s even easier, because you already have the full-time salary anyway…
Back to the time-consuming efforts… this is (I believe) why many blogs of individual researchers don’t last. After few, few dozen posts, people lose their steam, burn out, other things happen in life and they move on. There is also a ‘writing block’, and ‘ideas block’ which hits every once in a while.
So far I didn’t hit the wall yet, but I am very self-conscious how volatile my knowledge is, and that the overall know-how in this field is extremely volatile indeed. Only 6-7 years ago no one even thought of having EDR at their disposal, open source timeline and memory analysis tools for all Windows flavors, OS/X, Linux, mobile devices, there was no widely-used Splunk, Elastic Search, IOC standards, threat intel, threat hunting, not even basic information sharing… let’s be honest… even the sense of real community was not there. Today posting IOCs is a norm for many security vendors, and many guys ‘know’ each other from Twitter, various ‘open’, ‘closed’ groups, working in the same company or sector, and of course cons and blogs. It’s actually an awesome community to be a part of. I truly enjoy it.
Also, important thing to mention is that a tremendous number of _real_ developers joined the industry – they produce extraordinary results – many tools from not so recent past are no longer used as they have been replaced with real frameworks (volatility, plaso, so many $MFT and Registry and parsing tools (did someone say ‘Eric Zimmerman‘?) for a huge number of artifacts!). It slowly gets to the point where an individual doing research in this field has not enough power to act as a researcher. Seeing vendor reports today one can sense and see that apart from gigantic automation, data mining, collaborative research behind the scenes, they employ a number of the brightest minds in the industry. The DFIR/RCE area slowly becomes commoditized.
I think it is a good thing. Perhaps a bit scary work-wise, but it does make a huge difference to see the campaign analysis reports today and these produced just a few years back. Or, being able to ‘see’ the endpoint via a console, and see its historical transactions (f.ex. programs ran in last XYZ hours) allowing us to quickly contain an incident. Windows Event logs, Carbon Black logs, historical data… Something that was not present before, or took days to uncover just a few years back….
Back to the company history…
Over last few years – mainly through social media – I got to know many folks in this field. There are two personal lessons I took from this online presence: I am not that special, and this thing called DFIR/RCE gets extremely specialized pretty fast and it doesn’t stop to accelerate – a quick exchange with peers often leads to immediate answers, or some new interesting discoveries. We are more and more pressurized to skim over things by the pace of this field! And … to cooperate!
It does make me feel uncomfortable at times… How come, I am not the only one thinking of this particular scenario? That, my friend, is ego. Leave it outside the community…
So, we should not be afraid to ask and share. You and people you ask will appreciate it!
If you intend to do research you can’t do it on your own anymore. Sharing is extremely important.
Right, I was about to talk about the company’s history.
One day [don’t ask when, I don’t remember], things changed. I suddenly started getting many various requests from old friends, ex-coworkers, and also complete strangers to help them with software development, reverse engineering, forensic analysis…
Happy Days #2!
For the period of 2013-2014 I worked around the clock, often 12-18h a day. And this is not a hyperbole. When you get billable hours… you get crazy, consumed by the prospect of working on superinteresting stuff, learning extremely quickly new skills, making a buck of course, and you do everything to make everyone who wants your services… just happy… I established some good relationships and am really feeling privileged to be working with some of the best minds in this field. I also learnt new skills, and unlearnt some others, basically… made myself better at what I do. I hope.
The road to success is paved with working at night, all-nighters and lots of a weekend work & lots of … humility
At the end of 2014 some sad event happened in my family, followed by another one in May 2015 🙁
This was sort of wake-up call for me and I significantly slowed down. I reduced my workload and pretty much moved to work only with the closest friends and on the most interesting (from my perspective) stuff. I dedicated 2015 year to relaxing, and splurging and me and my wife traveled a lot that year. It was a tough year (this may sound awkward, but grief needs time and despite traveling it took a long time to recover…).
By the end of 2015 we decided to move back to UK and we did that in January 2016.
Work is not everything, and work takes you places. Find time for a reflection
Now, in UK, I am considering my next steps. I am sometimes tempted to take more work, I am also tempted to simply get old spending time with my wife, and reading more ‘real’ paper books. The Brexit just happened and it’s yet another big unknown…
What happens next is an interesting conundrum – probably shared by many of us in the DFIR/RCE community. We are in the middle of ‘developing’ a fully managed service called Incident Preparedness, Response and Management that will surely will be taken over by large companies at some stage; just wait for it; something that a few years later we will take for granted.
5 years later I can confess that I still love doing what I do @ hexacorn. I hope I didn’t piss off too many people over the years. This experience made me actually very humble. Stepping out of a frog’s well and so-called comfort zone is an important development, both professional and personal (if you consider yourself a geek, try it!). Even if I struggle with a backlog of blogs, posts, unanswered emails, and paper books that I want to read – after so many years that hexadecimal unicornish awe is still with me and I hope this will accompany me for many years to come. If given the same chances, I’d defo do it again.
And as I already know, sometimes it will pay for some new holidays 🙂
This blog is small yet it received a bit of attention over the years. I can’t count how many people I need to thank for it. There are guys from various companies, training orgs, independent researchers, authors of books (thx for referring to my blog!), reversers, forensic specialists, some of my managers who I was really lucky to have as great mentors, ex-coworkers, and most importantly – my wife who is supporting this work wholeheartedly.
I won’t name specific names, but you know who you are. This community is very special and being a part of it is a blessing. I truly mean that.
Thank you for reading, have a good weekend and hope to see us all here 5 years from now!
p.s. If you need some part-time RCE/DFIR help. Ping me 🙂
p.s.2. If you want to read more on the topic of consulting/company read Hal Pomeran’s Consulting-part-1-the-case-for-consulting