Understanding and Configuring SmartDefense (Check Point) Part 3

SYN Attack

A SYN attack is a Denial of Service attack that abuses the flags that are used to initiate a TCP session.This attack can cause the destination server to stop accepting new connections from valid hosts because it is busy waiting for responses from the attacker’s false sessions.

Notes from the Underground

TCP 3-Way Handshake

When a client wants to transfer data with another server, it will request a new session by sending an empty (no data) TCP packet with the "synchronize" (SYN) flag set. The SYN flag tells the destination host that the client is requesting a TCP session be opened. If the server decides to accept this new connection, an empty TCP packet will be sent in response with both the SYN and acknowledge (ACK) flags set. The SYN/ACK tells the client that the server has acknowledged its request for a new session and is accepting the request by trying to establish the connection in the opposite direction. As the final step of the three-way handshake, the client will send a response packet with only the ACK flag set. This completes the TCP handshake, as both sides have sent a SYN request and an ACK response that is required for two-way communication.

A session can be rejected by sending the reset (RST) flag to the other host. This is different from the graceful closing of a session that uses the finish (FIN) flag in a similar way as the initial handshake. The RST flag is used when either host detects an error and decides to reset the communication channel, or if it does not want to accept the communication at all. If a client initiates a TCP connection by sending a SYN packet to a port that is not currently being used, the server will respond with a RST/ACK, telling the client that it has acknowledged the request but is refusing to allow the communication. Along those same lines, if a client receives a SYN/ACK packet for a session that it has not sent the initial SYN for, the client will respond to the server with a RST flag, telling the server it does not wish to complete the handshake.


When a server receives a SYN request, it puts the partially established connection information into a separate table from where established connections are tracked, which Check Point refers to as the backlog queue. If the server does not receive a response to the SYN/ACK packet that is sent to the client, the uncompleted connection will stay in the backlog queue until the server times out the connection and removes it from the table. If the backlog queue is full of incomplete connections, the server will stop accepting new requests until space is made available in the queue. This process is illustrated in Figure 11.8.The attacker will take advantage of this limit by sending a constant stream of SYN requests, but will not respond to the SYN/ACK packets that are sent back to the source.This will keep the backlog queue full of invalid connections, and valid users will not be able to connect to the server.

Figure 11.8 TCP Three-Way Handshake

TCP Three-Way Handshake

To prevent the target from blocking all incoming packets from the IP address of the attacker, the source address of the packets will usually be spoofed, which makes it difficult to identify the attacker and filter out the invalid packets. During the attack, the attacker needs to make sure that the spoofed source addresses are not used by valid machines. If a real host receives a SYN/ACK for a connection it didn’t initiate, that host will send a RST back to the server. Once the server receives the RST packet, it will remove the connection from the backlog queue, which frees space for another new connection. If the majority of the attacker’s packets are spoofed with IP addresses of active hosts, the backlog queue will never fill up, because the connections will be reset in milliseconds.

A feature designed to combat SYN attacks, called SYNDefender, was added to previous versions of FireWall-1.Three different defense methods were available in SYNDefender, and each had its strengths and weaknesses:

■ SYN Gateway When the server sends the SYN/ACK back to the client, the firewall will immediately send the ACK packet to the server.This will move the connection out of the backlog queue and into the active connection table. This is done because servers can handle a much large number of established connections than partially established connections. If the ACK is not received from the client within the timeout period, the firewall will send a RST to the server, closing that particular session. Figure 11.9 illustrates the steps taken when using SYN Gateway.

Figure 11.9 SYN Gateway

SYN Gateway

■ Passive SYN Gateway This is the least intrusive method, because it allows the connection request to proceed as normal, in the backlog queue. If the ACK isn’t received within the timeout period, the firewall will generate a RST packet to remove the session from the server’s backlog queue.The timeout period on the firewall is much less than the default timeout from the server. This will not entirely prevent an attack, but it makes sure that entries in the backlog queue do not linger.The challenge is finding an appropriate timeout value that makes an attack very difficult, but will not reset sessions coming over slower links. Figure 11.10 illustrates Passive SYN Gateway.

Figure 11.10 Passive SYN Gateway

Passive SYN Gateway

■ SYN Relay When this method is used, the Firewall will respond to all SYN packets on behalf of the server by sending the SYN/ACK to the client. Once the ACK is received from the client, the firewall will pass the connection to the server. With this method, the server will never receive invalid connection attempts, because the firewall will not pass on the original SYN packet until it has received the corresponding ACK from the client. This method offers the best protection for the target server, but also has the most overhead because the firewall is required to respond to all connection requests passing through. This option was not available in FireWall-1 4.x, but was added to NG as a kernel-level process to keep delay to a minimum, although it will still add some amount of overhead.

With the introduction of SmartDefense to FireWall-1 NG FP2, the SYNDefender functionality was moved into the SmartDefense configuration. A new method to combat SYN attacks, called SYN Attack protection, was also added to SmartDefense, although Check Point left an option to use the older SYNDefender if you are so inclined.

The new SYN Attack protection automatically switches between two different modes of operation: passive mode and active mode. Under normal condition, SYN Attack protection runs in passive mode and only switches to active mode when it detects a SYN attack in progress. Once the attack has passed, the enforcement point will switch back to passive protection. Configurable options allow you to set SmartDefense’s sensitivity to SYN attacks.

When SmartDefense SYN Attack protection is operating in passive mode, it is using the Passive SYN Gateway feature described earlier in this section. This keeps the overhead to a minimum while still ensuring that uncompleted handshakes do not stay in the backlog queue too long. If the threshold of attack attempts is exceeded, the Enforcement Point (EP) will switch to active protection until the number of offending SYN packets drops below the threshold level. When using active protection mode, the EP is operating as a SYN Relay.The combination of these two methods ensures that your gateways operate as quickly as possible, but will also completely protect your servers from SYN attacks when one is detected.

By default, SYN protection is disabled, because the SYNDefender configuration has been overridden on all modules, but the new SYN Attack protection hasn’t yet been enabled.The default settings for the SYN Attack configuration are shown in Figure 11.11.To enable SYN flooding protection on your gateways, you need to use either active SYN Attack protection or use the SYNDefender configuration if you have modules that need to use the older protection.

As was mentioned in the anti-spoofing section, there are many tools that can be used by an attacker to try to disable your servers. RafaleX, allowing you to set any variable in the layer 3 and 4 packet headers, can be used to generate a SYN flood condition. Another tool, nmap (www.insecure.org), is a port scanning tool that identifies open ports by sending a SYN packet and seeing how the server responds. This tool can be configured to use spoofed "decoy" addresses and could possibly trigger a SYN flood condition if the probing was configured too aggressively. Both of these tools can be used by an attacker to exploit your firewall, but they are much more useful to the firewall administrator for testing the security policy and verifying that you are protecting against these types of attacks. Port scanners like nmap can also tell you which ports are being filtered and which are not.This can tip you off to a problem in your rule base before an attacker finds and exploits the problem.

Figure 11.11 SYN Attack Protection Methods

SYN Attack Protection Methods

After selecting the Activate SYN Attack protection box, you will be able to select the Configure button to change the SYN Attack settings.The available options for SYN Attack protection are shown in Figure 11.12.

Figure 11.12 SYN Attack Settings

SYN Attack Settings

Tools & Traps…

Using SYN Protection

It would be very beneficial to have a solid understanding of the traffic that is flowing through your firewall before you enable this feature. Every network is different with respect to the types of traffic flows that are considered "normal" for that environment. These SYN protection options have configurable parameters because a threshold of partially open sessions that could be considered an attack for a small network may be a normal number for a much larger network.

If your thresholds are set too high, your firewall may be ineffective at preventing a DoS attack against one of your servers, and setting the thresholds too low will mean that your firewall is often operating in active protection mode. Because the firewall is responding to all connection requests on behalf of the server, the performance of your network can slow to a crawl if the number of connection requests overwhelms the enforcement point. I would not recommend blindly enabling this feature without performing some form of traffic analysis on your network. This can become even more important if you plan on enabling SYN Protection for all interfaces of the firewall. Some firewalls have a large number of interfaces (10+), and configuring the firewall to watch all interfaces can significantly increase the load on the firewall.

The following options are available for configuration:

■ Track Choose the action that you would like to be taken when a SYN Attack is detected. This feature ties in with the Track Level selection.

■ Track level The track level determines how much detail you want to receive about possible attacks.Table 11.1 details each of the track level options.

■ Timeout The timeout value tells the gateway how many seconds it should wait before considering a SYN packet to be part of an attack. After this timeout period passes, the gateway will send an alert based on the Track action and Track Level options.

■ Attack threshold This is the level of unanswered SYN packets that should be considered an attack. When this threshold is crossed, the gateway will switch to active protection and relay all TCP sessions after the handshake has been completed with the firewall. Once the number of attacks has dropped back below the threshold number, the gateway will move back to passive mode.

■ Protect external interface only If this option is checked, the gateway will only look for SYN attacks coming from the external interface, as defined in the gateway’s topology. If you feel that you could experience SYN attacks from other interfaces, uncheck this option, and the gateway will watch for SYN attacks on all interfaces.

Table 11.1 Track Level Options for SYN Attack Protection

Track Level

Track Level Description

Individual SYNs

When this track level is selected, the Track action will be taken for each individual SYN packet that does not receive the corresponding ACK from the client. You will receive notifications for events such as SYN->SYN/ACK->RST and SYN->SYN/ACK->Timeout. In addition, you will receive notification when SYN Attack protection switches between active and passive defense modes.

Attacks Only

This track level takes the specified action only when the defense mode changes between passive and active modes. You will not receive any notification when individual SYNs time out or receive a RST in response.

None

In this mode, SYN Attack protection will continue to operate as normal, but you will not receive any notifications of suspicious SYN packets or when the gateway changes defense modes.

If you have a need to stick with the older SYNDefender configuration, you will need to uncheck the Override modules’ SYNDefender configuration option.This will enable the Configure button for the SYN Defender features, as shown in Figure 11.13.Your options here include the following:

■ Method Lets you choose between no protection, SYN Gateway mode, or Passive SYN Gateway mode as described earlier in this section.

■ Timeout This is the amount of time that the firewall will wait for the client ACK packet before considering the session to be part of an attack.This is the timer used by SYN Gateway and Passive SYN Gateway when deciding when to reset a session.

■ Maximum sessions This is the maximum number of sessions that the firewall will keep track of. The higher the number, the more memory will be used for the connection tables, but if the number of sessions exceeds this setting, the gateway will track new sessions.

■ Display warning messages When this option is checked, SYNDefender will send status messages to the log file.

Figure 11.13 Earlier Versions SYNDefender Configuration

Earlier Versions SYNDefender Configuration

Next post:

Previous post: