KB44748 - Backend node can fail with SSL-passthrough virtual server when the node rejects SSL handshake

When a virtual server is configured as SSL-passthrough i.e., the protocol is "SSL (HTTPS)" , you may occasionally see "Connection reset by peer" or the backend node failing. 

The issue is caused when the backend server doesn't support old SSL versions and because vTM is not involved in SSL decryption, when the backend node rejects the handshake, vTM will consider that connection has failed. This can fail a node in the pool on successive failures. For instance, a client could initiate multiple SSLv2 handshakes and when the backend node rejects them vTM will fail the failed. 

 

To avoid this you can either block old ssl versions using iptables or using TrafficScript rule. 

Solution 1
The following iptables rules reject old SSL versions. Please note that this will not persist on upgrades so you will need to make sure to re-apply the rules after upgrades and also make this persistent on reboots (which is outside the scope of this KB).

iptables -t mangle -D INPUT -m connmark '!' --mark 0x31 -p tcp --dport 443 -j nossl23 2>/dev/null
iptables -t mangle -I INPUT -m connmark '!' --mark 0x31 -p tcp --dport 443 -j nossl23
iptables -t mangle -N nossl23 2>/dev/null
iptables -t mangle -F nossl23
iptables -t mangle -A nossl23 -m u32 --u32 "0x33&0xffffff=0x160301" -j MARK --set-mark 0x31
iptables -t mangle -A nossl23 -m u32 --u32 "0x33&0xfffffe=0x160302" -j MARK --set-mark 0x31
iptables -t mangle -A nossl23 -j CONNMARK --save-mark
iptables -t mangle -A nossl23 -p tcp -m mark '!' --mark 0x31 --tcp-flags FIN,SYN,RST NONE -j DROP

The above rules will allow TLS 1.0 and above and will drop lower versions. You can adjust the rule according to your requirements. 

Solution 2
Alternatively you can create the following TrafficScript rule to block old versions. This will persist on upgrades. The example below rejects version lower than SSLv3 and allows the rest. The $min_ver can be set to change that behaviour i.e. if you want to allow only TLS 1.1 and above, set $min_ver to 0x302. 

# Match a SSL v3+ client hello (and not SSLv2 compatible v3)
# Returns the protocol version found in the client hello, or 0 if it
# doesn't look like an SSLv3+ client hello
sub getSSLv3Version($r) {
   if (!string.startsWith($r, "\x16\x03"))
   {
      $ver = 0;
   } else {
      # Look at the version in the record header
      $ver  = ord( string.substring( $r, 1, 1 ) ) * 256;
      $ver += ord( string.substring( $r, 2, 2 ) );
      
      if( $ver < 0x300 || $ver > 0x303 ) {
         return 0; # Unknown TLS protocol
      }
      
      # Check that the version number appears in the first fragment
      $len = ord( string.substring( $r, 3, 3 ) ) * 256;
      $len += ord( string.substring( $r, 4, 4 ) );
      
      if( $len < 6 ) {
         return 0; # TLS legacy_version is in a following record
      }
      
      # Check that we have a client hello
      if( string.substring( $r, 5, 5 ) != "\x1" ) {
         return 0; # Unexpected handshake message  
      }
      
      # Look at the version in the client hello message
      $ver  = ord( string.substring( $r, 9, 9 ) ) * 256;
      $ver += ord( string.substring( $r, 10, 10 ) );
      
   }
   return $ver;
}

# This should prevent anything before SSL 3; other values would be
# 0x301 -> TLS 1.0, 0x302 -> TLS 1.1, 0x303 -> TLS 1.2 & 1.3
$min_ver = 0x300;
$request = request.get(11);
$ver = getSSLv3Version( $request );

log.info( "got version " . $ver . ", min is " .  $min_ver );

if( $ver < $min_ver ) {
   connection.discard();
}