A cluster node that is up and running appears in the WebUI as unreachable
Answer: A cluster member may appear as unreachable even when it is online and can be pinged. Here are reasons why a node can show as unreachable:
After I join the machine to a running cluster my session times out and I have to login again
- Its password is incorrect. If the node has never joined the cluster or if the password has changed in between this might be a possibility
- It does not know about all the nodes of the cluster. Two nodes must have the same membership to belong in the same cluster
- It has different group communication mode
- It has a different version of the software
- A firewall or other improperly configured network device is between the two nodes preventing communication. See below for ports that need to be open for the machines in a cluster to communicate. The Maintenance/Troubleshooting/Clustering/Network Connectivity tab in the admin UI can be used to test connectivity between IVEs.
I created a cluster and added a node but it appears as Unreachable. When I login to the other machine it does not appear to be a member of the cluster
Answer: This is the expected behavior. The states (including the active sessions) for the member that joins the cluster are overwritten by the state in the cluster. Therefore the session you use to join the cluster is closed.
What protocols and ports are used for clustering?
Answer: It is not enough to just add a node to a machine already in the cluster. You also need to go to the machine that is being added to the cluster and use the "join cluster" UI to add the machine to the cluster. The IVE provides such a UI in two places: a) Part of the WebUI and b) part of the console UI when a machine boots
In an active/passive cluster both nodes act as active. What is happening?
Answer: TCP 4808, 4809, 4900-4910 / UDP 4803 4804
Is the traffic among the members of the cluster encrypted?
Answer: If the two machines lose connectivity (that could happen in high latency environments) the heartbeat between the machines is lost and both machine become active. Starting in IVE OS 4.0 the number of seconds that the heartbeat can lose before a member takes over is adjustable from the WebUI
I have a cluster with Central manager installed and I want to upgrade only one node. How can I do this?
Answer: Yes, all the data traffic is secured using AES/128Bit encryption and MD5. The intent has been to provide adequate confidentiality and integrity to the messages inside a protected network/VPN. We discourage clustering the devices over a public WAN infrastructure unless additional protection between the subnets exist (e.g. through site-to-site ScreenOS VPNs).
When an IVE starts up does it always take the VIP?
Answer: Disable the node from the Clustering tab. If you upgrade a disabled node only the node gets upgraded. Another alternative is to remove the node from the cluster, upgrade it, and later rejoin the node to the cluster. Of course, all the machines in a cluster must have the same version of the software. The upgrade subsystem will upgrade nodes that join the cluster as needed.
What is a cluster leader?
Answer: When an IVE in an active/passive cluster starts up it will claim the VIP if:
- The network healthcheck indicates that IVE is properly connected to the network for a while ("X" seconds) and
- The IVE does not receive communication from the other IVE in the cluster.
How is the leader elected?
Answer: Whenever a node (re)joins a cluster, it receives configuration and runtime state updates from the nodes already in the cluster. One of the nodes acts as the representative of the cluster and sends the cluster state update to the joining node. This node is referred to as the leader of the cluster.
What happens when the current leader node goes down?
Answer: The leader election algorithm is rather complex and is mostly but not always deterministic. The algorithm favors the current leader during simple membership changes. When two or more cluster partitions merge, each partition comes with its own leader - all but one of which must relinquish leadership. The leader from the partition that has the node with the highest "sync rank" (configurable from the cluster status page) retains the leadership. If the tie cannot be broken based of sync ranks, node names are used in a way similar to sync rank.
The non-determinism in leadership election comes into play when multiple nodes attempt to rejoin the cluster simultaneously - in this case when there is no clear choice of who should be the leader, the node who win the race to start the cluster protocol first usually ends up as the leader.
What is the relationship between the license primary (the node that has the feature licenses) and the cluster leader?
Answer: When the current leader goes down, a new leader gets elected. If the old leader was involved in a state of synchronization in a joining node at the time it went down, the new leader takes over that responsibility. Other than that, a leader node going down is a very mundane event - it does not trigger anything special in the system.
How is a leader in an Active/Passive (A/P) cluster different from the leader in an Active/Active (A/A) cluster?
Answer: There is no relationship. The license primary and cluster leader are orthogonal concepts.
In an A/P cluster what is the relationship between the cluster leader and the VIP owner?
Answer: An A/P leader is exactly the same as an A/A leader.
Must admins make all configuration updates at the leader node?
Answer: Cluster leadership and VIP ownership are orthogonal concepts. There is no relationship. It is perfectly legitimate to have one node as the cluster leader and the other node as the VIP owner.
How do I upgrade a cluster?
Answer: There is no such requirement. While a cluster is up and running with no membership changes happening, the leader has *no* special role to play. Changes can be made at any cluster node and they will get replicated across the entire cluster almost instantaneously. Changes initiated at the non-leader nodes do *not* incur *any* additional overhead compared with changes initiated at the leader node.
I attempted to enable a node in the cluster and it rebooted, what has happened?
Answer: If you have central manager, enable all nodes and just upgrade one node of the cluster. After the node upgrades it detects the nodes that run the older version of the software. It instructs them to upgrade as well.
If you do not have central manager you should open multiple admin sessions, one for each node in the cluster. Then perform the upgrade by pressing the button almost simultaneously in all the windows. The nodes will upgrade, reboot and rejoin into one cluster. If you upgrade only one node, it will reboot, find the other nodes running an older version of the software, wait for about two minutes and eventually boot as a disabled node. Attempting to enable the node will silently fail with a log message in the events log. To continue, disable all the nodes and then enable the node you have upgraded. Then upgrade each node independently and enable it after upgrade.
Central manager license is strongly recommended in particular in clusters with more than two nodes.
Answer: If a node is disabled and has an older version of the S/W than other nodes of the cluster, it will reboot when enabled so that it can take the software from other nodes of the cluster and upgrade. This happens only when Central Manager license is installed. A warning should be given before the reboot but currently is not.