Marrying client-side Windows-based CryptEncrypt and server-side,Linux-based Crypt::OpenSSL::RSA

Time flies and it does so very quickly. The story I am about to tell you is 8 years old, but it does feel like I wrote it yesterday.

In 2014 a client asked me to develop a never-seen-before prototype of a new type of an endpoint agent that would be code-minimal, position-independent, 32- and 64- architecture-aware and talk to a backend using strong encryption. Yes, kinda EDR or C2-like agent and we had discussions about using it for both blue and red team engagements, if it worked.

Anyone who tried to make Windows crypto primitives talk to (typically Linux-based) server-side crypto primitives knows that it is an awful coding experience. After googling around, and trying different things I eventually developed the prototype. I can’t share the code for obvious reasons, but I can at least describe what it did.

On a client side, I had a routine that would talk to the socket (not proxy aware at that time) and follow a basic data protocol exchanging encrypted data blobs with my server. The data was encrypted with a public key that only server could decrypt. Nothing really ground breaking.

What was annoyingly, frustratingly hard to develop was the actual decryption part. The server part was using Crypt::OpenSSL::RSA (yes, perl!) primitive, and I couldn’t force it to decrypt the CryptEncrypted message I was sending.

After many hours of debugging and googling around I eventually figured it out. After I used CryptEncrypt I just had to reverse the data blob delivered by the function: byte, by byte. Yup, it was that simple.