Detect and prevent intrusion in Forefront TMG - Part 1: Behavior detection

Network Administration - In this two-part series, we will introduce you to Forefront TMG's behavior detection features.

Microsoft Forefront Threat Management Gateway (TMG) 2010 is a multi-layer belt prevention system featuring many advanced protection technologies. In addition to integrating URL filtering, virus scanning, malicious code and HTTPS inspection, the TMG firewall also has the ability to detect and prevent intrusion. In Part 1 of this series, we will introduce you to behavior-based intrusion prevention and detection. Part two will introduce Network Inspection System (NIS), which provides vulnerability-based intrusion prevention and detection.

Popular network attacks

Intrusion detection and prevention in Forefront TMG is built on the behavior-based intrusion detection and prevention feature found in Microsoft ISA Server 2000. By performing low-level packet inspection, TMG firewall can detect and block many types of attacks. Intrusion detection and prevention is configured by opening the TMG management console, marking Intrusion Prevention System , then selecting the Behavioral Intrusion Detection tab in the main control window and clicking Configure Detection Settings for Common Network Attacks .

Detect and prevent intrusion in Forefront TMG - Part 1: Behavior detection Picture 1
Figure 1

Here you will see the intrusion detection feature is enabled and configured by default to identify, block and record the following attacks:

1. Windows out-of-band (WinNuke)
2. Land
3. Ping of death
4. IP half scan
5. UDP bomb
6. Port scan (not enabled by default)

Detect and prevent intrusion in Forefront TMG - Part 1: Behavior detection Picture 2
Figure 2

Also, by selecting the DNS Attacks tab, you will see that the TMG firewall also has the ability to provide advanced application layer protection for many common attacks targeting public DNS servers, such as DNS. hostname overflow and DNS length overflow . The TMG firewall can prevent DNS roaming (but is not enabled by default).

Detect and prevent intrusion in Forefront TMG - Part 1: Behavior detection Picture 3
Figure 3

Filter the IP option

Intrusion detection and prevention in Forefront TMG also allows filtering IP options that are in the header of an IP packet. Filtering IP options is enabled by default and can be configured by clicking the Configure IP Options Filtering link in the main control window.

Detect and prevent intrusion in Forefront TMG - Part 1: Behavior detection Picture 4
Figure 4

Most IP options are harmless. However, there are still some IP options that indicate signs of potential malicious behavior. By default, the TMG firewall is configured to allow IP filter filtering and will remove IP packets containing IP options 7 (Record Route), 68 (Time Stamp), 131 (Loose Source Route) and 137 (Strict Source Route). If the IP packet has enabled specific option numbers, the TMG firewall administrator can adjust the action that took place. Any packages containing IP options can be ignored, packages with selected IP options can be rejected or all packages except the one with the selected IP option will be rejected.

Detect and prevent intrusion in Forefront TMG - Part 1: Behavior detection Picture 5
Figure 5

The TMG firewall may also prevent packet forwarding that contains IP fragments. This option is not enabled by default. Blocking IP fragments can cause unwanted effects, so be careful when enabling this feature.

Detect and prevent intrusion in Forefront TMG - Part 1: Behavior detection Picture 6
Figure 6

Flood Mitigation

Mitigation is a critical component of Forefront TMG firewall. This feature helps mitigate the impact of some types of attacks on firewalls. Help mitigate potential DoS attacks by enforcing some restrictions on the amount of traffic a host can create. The mitigation flood is enabled by default and can be configured by clicking the Configure Flood Mitigation Settings link in the main control window.

Detect and prevent intrusion in Forefront TMG - Part 1: Behavior detection Picture 7
Figure 7

The flood mitigation can control the following network parameters:

1. Maximum TCP connection required per minute on an IP
2. Maximum number of concurrent TCP connections on IP address
3. The largest half-open TCP connection
4. Maximum HTTP request per minute on IP address
5. New non-TCP session maximum per second on a rule
6. Maximize the number of UDP sessions simultaneously on an IP

Detect and prevent intrusion in Forefront TMG - Part 1: Behavior detection Picture 8
Figure 8

Clicking Edit . next to the flood mitigation parameters will allow administrators to configure connection restrictions to apply to IP exception.

Detect and prevent intrusion in Forefront TMG - Part 1: Behavior detection Picture 9
Figure 9

When a host creates enough volume to exceed the pre-set restrictions, the TMG firewall will start dropping packets from this host and create alerts.

Detect and prevent intrusion in Forefront TMG - Part 1: Behavior detection Picture 10
Figure 10

Note:

One important thing to note is that communication is allowed by counting to the connection quota.

If the host in question is stopped sending requests, or reducing the frequency of requests emitted from it to below the established limit, after a minute, the TMG firewall will increase the permissions of new connections. from this host.

Normally, when a host exceeds the connection limit, it will be suspected of malicious traffic or possibly a poorly written application. However, there are some cases where hosts create a huge amount of legitimate traffic. Some examples are DNS server and SMTP server. However, some administrators will try to resolve these problems by adding default connection restrictions, or even completely disabling flood mitigation. This approach is not recommended. The proper way to handle this scenario is to create an IP exception , then execute a certain threshold for systems that fall into this object group.

Detect and prevent intrusion in Forefront TMG - Part 1: Behavior detection Picture 11
Figure 11

SIP quota

With the introduction of SIP filters in TMG 2010, the firewall is currently capable of enforcing quota for SIP traffic. TMG places restrictions on the following SIP parameters:

1. The largest number of registrations
2. The largest number of simultaneous registration numbers on each IP
3. Maximum number of concurrent calls
4. Maximum number of concurrent calls per IP

Click Edit . next to each parameter will allow the administrator to configure the thresholds according to their requirements.

Detect and prevent intrusion in Forefront TMG - Part 1: Behavior detection Picture 12
Figure 12

Conclude

The intrusion detection and prevention feature based on Forefront TMG's behavior provides a basic level of protection against common network attacks. It helps prevent the spread of IP packets with suspicious or potentially malicious IP options. In addition, TMG reduces DoS attacks (Denial of Service) by enforcing restrictive connections, avoiding users who have malicious intent or hosts infected with malicious code that could flood the connection. TMG firewall administrators can also configure quotas for the number of SIP traffic created by protected clients.