[In-Depth Guide] – The Basics of Packet Firewall

24 Min Read

Popular firewalls are, in essence, a class of intermediate router devices engineered to violate the fundamental design of an intermediate router device.

As opposed to routers proper, systems that are expected to make nondiscriminatory routing decisions based on the information encoded on the third OSI layer, firewalls usually interpret, act upon, or even modify information on higher layers (such as TCP or even HTTP).

Firewall technology, although fairly recent, provides a well-established and well-understood set of solutions and can be found in home networks and in large corporations. Firewalls are configured to reject, allow, or redirect specific types of traffic addressed to specific services and are (not surprisingly) used to limit access to certain functions and resources for all traffic traveling across such a device. Hence, they provide a powerful, albeit sometimes overhyped and overly relied upon, security and network management solution.

The key to the success of firewalls in all network environments is that they protect an array of complex systems using a single and comparatively more robust component and provide a fail-safe security measure if a configuration problem exposes a vulnerable service or function on a protected server. (In extreme cases, firewalls are used simply to cover for poor configuration and lack of maintenance of a protected system, usually with disastrous results.)

 

How Firewall Protect Network
How Firewall Protect Network

Stateless Filtering and Fragmentation

Basic firewalls are stateless packet filters. They simply inspect certain features of every packet, such as the destination port on Transmission Control Protocol SYN connection attempts. They then decide, based on these characteristics alone, whether to allow the packet to go through. The stateless design is extremely simple, reliable, and memory and resource efficient. For example, a stateless firewall can limit incoming connections to a mail server to only those addressed to port 25 (SMTP) by dropping all SYN packets but those addressed to this port. Because no connection can be established without this initial SYN packet, the attacker cannot interact with applications on other ports in a meaningful manner.

To achieve this, the firewall does not have to be nearly as fast and complex as the mail server itself, because it does not need to keep a record of currently established connections and their exact state. The problem with this type of completely transparent protection is that the firewall and the final recipient might understand some of the parameters differently. For example, say an attacker convinces the firewall that it is connecting to an allowed port, but crafts its traffic so that the final recipient reads it differently and establishes a connection to a port that the firewall is supposed to be protecting. An attacker can thus access a vulnerable service or an administrative interface, and we are in trouble. Although causing such a misunderstanding might sound unlikely, it turned out to be fairly easy to achieve with the help of our old friend, packet fragmentation, using an approach commonly referred to as the “overlapping fragment attack” (described in 1995 by RFC1858). In this situation, the attacker sends an initial packet, containing the beginning of the Transmission Control Protocol SYN request, to a port that is allowed by the victim’s firewall (such as the aforementioned port 25).

The packet is missing only a tiny bit at the end and has a “more fragments” flag set in its IP header, but why should the firewall bother about the trailing data in a packet? The firewall examines the packet, and because it is a SYN packet, its destination port is also examined and found acceptable. The packet is passed through, but the recipient does not interpret it immediately. Instead, the packet is kept, pending the successful completion of defragmentation, which will not occur until the last trailing chunk of the packet arrives. Next, the attacker sends a second packet fragment. This second packet is created to overlap with the original packet just enough so that it overwrites the destination port (one of the fields of the TCP header) at its location in the reassembly buffer.

The fragment is crafted so that it starts at a nonzero offset and lacks most of the TCP header, except for the overwritten bit. Because of this (and because it lacks the information needed to examine the flags of a TCP packet or other vital parameters the firewall could use to determine whether to allow or block this traffic), the second fragment is usually relayed as is by a stateless firewall. When combined with the first packet by the recipient, this second packet overwrites the original destination port to a more naughty value chosen by the attacker and actually opens a connection to a port that should be protected by the firewall.

Whoops.

To protect against this attack, a well-designed stateless firewall performs initial defragmentation before analyzing packets. This, however, makes it somewhat less “stateless,” and less transparent.

Stateless Filtering and Out-of-Sync Traffic

Another problem with stateless packet filters is that they are not nearly as tight as we might hope. The filtering can only be carried out when a single packet contains all the information necessary for the filter to make an informed decision on how to handle it. Because, following the initial handshake, a TCP connection is largely symmetrical, with both parties having equal rights and using the same type of traffic (ACK packets) to exchange data, it is not easy to apply meaningful filters to anything other than the initial phase of a connection.

There is no way to determine who (if anyone) initiated the connection through which ACK packets are being swapped without actually tracking and recording connections. Thus, it is a bit hard to define in a meaningful way the filtering policy that the firewall should attempt to apply to traffic such as ACK and other midway packets such as FIN or RST. The inability to filter past SYN is not normally a problem. After all, if an attacker cannot deliver the initial SYN packets, they cannot establish a connection. But there’s a catch: how systems handle non-SYN traffic to a specific port depends on whether a port is closed or the system is listening on that port. For example, some operating systems reply with RST to stray FIN packets and generate no reply on ports that are in open (listening) state.

Example of ACK scan (Client - Server)
Example of ACK scan (Client – Server)

Techniques such as a FIN or ACK scan (the latter initially described by Uriel Maimon in Phrack Magazine), as well as NUL and Xmas scans (scans with illegal packets with no flags set and all flags set, respectively) can thus be used against stateless packet filters to gather preattack evidence about which ports are open on a remote system or to map out what traffic is being dropped by the firewall. The ability to learn that a specific port is open without the ability to establish a proper connection to it is not an immediate threat by itself. However, a scan of this nature often discloses extremely valuable information about network internals (such as the operating system and services being run), which can be used to facilitate a better, more efficient, and more-difficult-to-detect attack once the first line of defense is compromised or bypassed.

Thus, this is perceived as a potential weakness of a stateless firewall. Perhaps a more grave threat is associated with the mechanism of SYN cookies when combined with stateless filtering. SYN cookies are used to protect operating systems against resource starvation attacks, in which the attacker sends a very large number of spoofed connection requests to the host (not itself a difficult operation to perform). This forces the recipient to send bogus SYN+ACK replies, and additionally to allocate memory and consume other resources when adding this connection-to-be to its TCP state tables. Most systems under such an attack would either consume excess resources and slow to a crawl or deny service to all clients at some point until those bogus connections time out. To deal with this potential problem, SYN cookies use a cryptographical signature (a shortcut, actually, identifying the connection uniquely) in all SYN+ACK responses inside the ISN field, and then forget about the connection altogether. Only once the ACK response arrives from the host, and only if the acknowledgement number validates against the cryptographic procedure, will the connection be added to the state table.

The problem with SYN cookies, however, is that, in such a design, there is the possibility that SYN (and SYN+ACK response) was never sent in the first place. If the attacker can create an ISN cookie that validates against the host’s SYN cookie algorithm (perhaps because the attacker has enough bandwidth, or because the algorithm is weak), he can send an ACK packet that would trigger the remote host to add a new connection to its state table despite, as mentioned, not ever sending SYN and receiving SYN+ACK. A stateless firewall would have no way of knowing that a connection has just been established, because it never received the opening request in the first place!

Because there is no initial SYN packet, the destination IP and port could not be checked by the firewall and either approved or rejected, and yet, a connection is all of a sudden established. That’s really bad.

Stateful Packet Filters

To solve the problems of stateless filters, we need to store some of the information about previous traffic and the state of established streams on the firewall. This is the only way to transparently predict the outcome of defragmentation or to obtain the context for midconnection packets and decide whether they are illegitimate and should be discarded or are expected by the recipient and should be delivered. With the increase of affordable high-performance computing, it has become possible to devise firewall systems that are much more complex and advanced than we could ever imagine. Thus, we have progressed to stateful connection tracking, a situation in which the firewall not only examines single packets, but remembers the context of a connection and validates every packet against this data. This allows the firewall to seal the network tightly and to disregard undesirable or unexpected traffic without relying on the recipient’s ability to always tell good traffic from bad.

Client - Statetful Packet-Filtering Firewall - Server
Client – Statetful Packet-Filtering Firewall – Server

Stateful packet filters try to track connections and allow only the traffic that belongs to one of the active sessions; as a result, they provide better protection and logging capabilities. The task of stateful filtering is, of course, more challenging than stateless filtering and consumes considerably more resources, especially when a sizable network is protected by such a device. When protecting a large network, the firewall suddenly requires plenty of memory and a fast processor to store and look up the information about what is happening on the wire. Stateful analysis is also more likely to cause problems or confusion. Issues ensue as soon as the understanding of the current state of a given TCP/IP session differs between the firewall and the endpoints; a situation that is not unlikely given the ambiguity of specifications and the variety of stacks used. For example, upon receiving an RST packet that is not within sequence number limits accepted by the recipient, a firewall that applies sequence number inspection less stringently than the final recipient does might conclude that a connection is closed, whereas the recipient might conclude the session is still open and be willing to accept further communications pertaining to this connection, and vice versa. In the end, stateful inspection comes at a price.

Packet Rewriting and NAT

The solution to improving packet interpretation, and to providing better protection against attacks such as those that use packet fragmentation to bypass firewall rules, was to give firewalls the ability to not only forward, but also rewrite portions of the traffic transmitted. For example, one approach attempts to resolve ambiguity by performing a mandatory packet defrag-mentation (reassembly) before comparing the packet against any access rules configured by the network administrator. With the development of more sophisticated solutions, it became obvious that packet rewriting would not only benefit the network, but also provide a quantum leap for network security and functionality by deploying extremely useful technologies such as NAT. NAT is the practice of mapping certain IP addresses to a different set of IPs prior to forwarding them and demangling the responses sent back by a protected system. A stateful NAT mechanism can be used, among other applications, to implement fault-tolerant setups in which a single, publicly accessible IP address is served by more than one internal server. Or to save address space and improve security, NAT can be implemented to allow the internal network to use a pool of private, not publicly accessible, addresses, while enabling hosts on the network to communicate with the Internet by “masquerading” as a single public IP machine. In the first scenario, NAT rewrites destination addresses on incoming packets to a number of private systems behind the firewall.

 NAT rewrites destination addresses on incoming packets

NAT rewrites destination addresses on incoming packets

This provides a fault-tolerant load-balancing setup, in which subsequent requests to a popular website (http://www.microsoft.com, perhaps) or other critical service can be distributed among an array of systems, and if any one fails, other systems can take over. The task is sometimes achieved with dedicated devices (not surprisingly called load balancers), but often also supported by NAT-enabled firewalls. The latter scenario, commonly referred to as masquerading, relies on rewriting source addresses on outgoing packets so that a number of private, protected systems (that might be using private addresses not routed to this network from the Internet, such as 10.0.0.0) can connect to the external world by having their outgoing connections intercepted and rewritten by the firewall. The systems are hidden behind a firewall, and their actions appear to recipients outside the NAT-protected network as originating from the firewall. The connection is mapped to a specific public IP address and a specific port, and then the traffic is pushed out. All traffic returning from the destination to this IP and port is rewritten to point back to the private system that initiated the connection and forwarded to the internal network. This allows the entire private network of workstations that are not intended to offer any services to the Internet to remain not directly reachable from the external world, thus greatly increasing the network’s security, concealing some of its structure, and preserving expensive public IP address space that would otherwise have to be purchased to accommodate every system. Using this system, a party that has only one public IP routed to them can still construct a network of hundreds or thousands of computers and provide them with Internet access.

Lost in Translation

Once again, address translation is more complex than it might sound: some higher-level protocols are not as straightforward as just connecting to a remote system and sending a bunch of commands. For example, the ancient but wildly popular File Transfer Protocol (FTP), in its most basic and most widely supported mode, relies on establishing a return (reverse direction) connection from the server back to the client for the purpose of transferring the requested data; the initial connection initiated by the client is used only to issue commands. Many other protocols – most notably some chat pro-tocols, peer-to-peer collaboration or data-sharing tools, media broadcast services, and so forth – also use weird or unusual designs that call for reverse connections and port hopping or allowing specific session-less traffic (such as User Datagram Protocol [UDP] packets) back to the workstation.

Address Translation System
Address Translation System

To address these challenges, every implementation of masquerading that does not aim to render these protocols useless must be equipped with a number of protocol helpers. These protocols inspect the application data exchanged within a connection, even sometimes rewriting some of it and opening temporary holes in the firewall to allow for a return connection. And herein lies another problem, first spotted in FTP helper by Mikael Olsson several years ago and later researched in other protocol helpers by, among others, the author of this book. The problem is that these helpers decide to open holes in the firewall based on the information sent by a workstation over a specific protocol to a remote system. They assume that the traffic generated by the system is being transmitted on the user’s behalf and with the user’s knowledge.

Needless to say, some programs, such as web browsers, can be tricked into sending certain types of network traffic, including traffic that “looks like” a protocol the program does not natively support, and can even be forced to do so automatically by crafting specific malicious content and sending it to the application. This spoofed traffic can fool a helper program into poking a hole in the firewall. A classic example of such an attack is an abuse of a generic web browser: by adding a reference to a web page or a web element supposedly located on an attacker’s system on a nonstandard HTTP port (which is, however, quite standard for FTP traffic), the client can be forced to connect to this resource and attempt to issue an HTTP request. Because the port to which the connection is established is normally used by FTP, the firewall’s FTP helper starts listening to the conversation, hoping to give a hand when necessary. The following example URL would cause the HTTP client to connect to the FTP port and issue what appears to be an FTP PORTcommand, which would be picked up by the firewall helper:

HTTP://SERVER:21/FOO<RETURN>PORT MY_IP,122,105<RETURN>

The request issued by the client would be just meaningless gibberish to a legitimate FTP service on the other end, and the service’s response would be incomprehensible to the web client issuing this request – but that’s not the point. What matters is that the attacker can control a part of the request – the file name the client will request from the server. This fictitious file name, chosen by the rogue, can contain any data the rogue wishes. By making the file name contain substrings normally identified with FTP requests, the attacker can trick an FTP protocol helper that is listening to this connection for a specific text command (PORT) into believing that the user is attempting to download a specific file. Hence, the remote server is temporarily allowed to connect to the victim (here, to a naughty sounding port 31337 – 122*256+105=31337). And so we let the attacker in without the victim knowing.

Oops – again, more than we bargained for.

Share This Article
Leave a comment