We had an issue beginning yesterday at our pilot location for DHCP Failover (Hot-Standby). We don't even have the DHCP forwarder set up for the subnet yet, so the clients are unable to reach the partner failover server to get IPs. We are only testing the way the servers behave at the moment when DHCP failover is configured. The servers are able to reach each other, but I have noticed that communication from them drops for a few seconds or a minute beginning yesterday. This has been happening every few hours perhaps. The connection appears to always re-establish, and when that happens the status of the failover partnership changes back from COMM_INT to NORMAL.
For some reason though all of the lease times began getting set to 1 hour (the MCLT) yesterday, perhaps because of these communication issues. Not only that, but many of them are not able to renew their IPs, even though the scope on the primary server is only maybe 20% full. It's as if the 5% partner server scope reservation is applying to the primary server. Surely that is not supposed to happen though, right? For instance when we restarted the switch we got 15 or 20 of these errors in the DHCP Event Log:
"DHCP client request from 989096BEB05B was dropped since the applicable IP address ranges in scope/superscope HRS-PC5411 are out of available IP addresses. This could be because of IP address ranges of a policy being out of available IP addresses."
Another engineer already removed the failover scope, so I can't do much further testing at this point, but I do need to get to the bottom of this. We were planning to roll DHCP Failover out to 650+ locations, but if communication issues with the hot-standby are going to effect the performance and capabilities of the primary server, we cannot move forward with this.
Can anyone help me understand what happened here, and what we might be able to do to avoid this issue in the future?