I previously posted this on the Windows Forums where I was promptly told that this is too in depth in networking for said forum.
To preface this, I am using the server in a lab environment and am attempting to set up my own L2TP/IPSec VPN. I have opened ports 500 UDP and 1701 TCP
on my router to the server's primary interface where the VPN is. This is on a consumer Comcast connection where other applications such as Arma 3 dedicated servers and IIS have worked.
The RRAS role is running based on this tutorial: http://www.thomasmaurer.ch/2014/01/how-to-install-vpn-on-windows-server-2012-r2/ I have only deviated from
this in using DHCP forwarding instead of a static IP pool as my router runs a DHCP server, and as I understand it, the router should give out IP addresses from the internal IP pool which I use for everything else. I am also using PSK authentication instead
of it being certificate based. For user authentication I have MS-CHAP-V2 and CHAP enabled; I am connecting from the remote device with an account I have created on the server for the purpose of this VPN which I know RRAS connections are allowed on.
When connecting I receive error 789: The L2TP connection attempt failed because the security layer encountered a processing error during initial negotiations
with the remote computer. From what I have seen, this can be fixed by checking that both ends of the connection are not behind a NAT (not an option), checking PSK (already done), and checking certificates (not applicable). If there is a way to fix this issue
that would be excellent, however my server will always be behind a NAT firewall as the router is one, and the modem becomes one if multiple devices are connected to it without a router in between.