LAN traffic encryption on Windows

Question

I'm working on a study project and need to create a software which should encrypt LAN traffic between computers with Windows. So I need to capture, encrypt and resend all outbound traffic, and capture and decrypt all inbound traffic. Currently I see two way to do it: 1) IP over UDP. I need encrypt IP packets and send them through UDP link, receive them and decrypt. 2) Encrypt payload of IP packets and decrypt it on another side.

I actually don't know how to do it better and where to start. All suggestions/examples will be helpful.

Answer 1

If you really only need to encrypt the traffic you can simply install a "manually keyed" IPSec SA. See instructions at MSDN

That being said, encryption is the easy part. Authenticating the peers and key agreements is the hard part.

Cryptography is hard to get right, so you definitely want do not want to invent a probably insecure wheel, but opt for a peer-reviewed standard solution, such as the Internet Key Exchange protocol. There is an (unfortunately discontinued) internet draft of a minimal IKE implementation.

Please note that it is perfectly OK to use IKEv2 as the key agreement / authentication protocol for any application - not just for ESP. But if you need to encapsulate ALL IP, Encapsulating Security Payload in tunnel mode is your friend, and the lucky thing here is that ANY OS that is IPv6 compliant MUST implement it, so using ESP is in practice just a matter of installing the key material to your OS kernel's IP stack.

In case you need code samples, I have made a minimal proof-of-concept level implementation of an initiating end of an IKEv2 peer in Python. A Perl implementation doing the same can be found from these IETF proceedings slides