Sometimes when you’re troubleshooting a network problem, you need to look at the packet data. It could be to find security threats or investigate strange network behavior. One of the common ways to capture packet data is with SPAN ports. But with the data deluge found on today’s networks and infrastructure, you can easily run into problems analyzing the packets captured using a SPAN. TCP sequence gaps are one problem that can result from capturing using SPANs.
In this post, let’s get into more about what a SPAN port is. Also, you’ll find out how different capture methods using SPAN can lead to TCP analysis problems and how you can reduce their occurrence.
What’s a SPAN?
A switched port analyzer (SPAN), also known as port mirroring, is one of the most common methods to capture network data. SPAN is a feature of network switches copying the traffic on one or more ports and sending it to another port. You can capture this traffic by connecting a device with packet analyzer software or a packet sniffer to the destination port.
If you can’t connect directly to the switch with SPAN enabled, you can utilize remote SPAN (RSPAN). With RSPAN, you’re able to send the captured packets to a remote switch port on your network. This way, you’re able to directly connect to the remote switch and still do a packet capture with your device.
SPAN is a very convenient way to capture data, especially if you don’t have to physically be directly connected to the switch whose port you’re spanning.
Packet Capture Methods
When you’re capturing data, you’re usually using a packet capture tool or analyzer. There are many options for getting packet captures.
- A capture agent or tool installed on a server hosting many applications
- A capture agent or tool installed on a separate device, such as a laptop
- A hardware appliance such as a network sniffer or probe with packet capture capabilities
Whichever option you use, you should be aware of certain conditions while capturing packets.
Capturing on an Application Server
If you’re using an agent installed on an application server host, you must take into account the network connection speed. Most servers these days are running at gigabit speeds or more. But that doesn’t mean the server switch is running at that speed. The server could be 1Gbps while the switch runs at 100Mbps. Whether it’s a misconfiguration or the standard, you need to know this.
You must also know how busy the server is. Whatever the network connection speed, the amount of traffic traversing the server’s network interface can affect your capture. You want to consider server busyness because the capture process itself can affect other processing the server is doing. Make sure that the server has enough CPU, memory, and storage to do the work of capturing and processing what’s likely to be gigabytes of information. You want it to do this with no or minimal impact on its other jobs, like running your applications.
Capturing on a Separate Device
If you’re using an external laptop, you must be aware of how much processing power that laptop has. You want a device that can keep up with the amount of data you’re capturing. If it can’t keep up, your laptop won’t be able to capture everything. It will drop packets before they’re captured.
You also should know how your device is receiving the packets. Find out if it’s coming from a switch with port mirroring enabled or from a network tap. One is often better than the other.
Capturing on a Packet Sniffer
If you’re using a sniffer, you’re not as concerned about its processing power. This appliance is likely powerful enough. If not, you either have bigger problems or it probably wouldn’t be available for you to use. But you should check anyway.
On top of processing, you need to also consider how your sniffer is getting its packet data. Like the laptop, find out if the packet data is coming from a SPAN port or network tap.
In all of the above ways you can capture data, you’re either installing an agent or capture tool. You’re also specifying how data is captured – SPAN port or network tap.
SPAN Port Limitations
A big limitation with capturing using a SPAN port is that it’s a low-priority function on network devices. On switches, port mirroring tends to get a lower priority compared to things like moving actual data. The process of copying actual data to another port for analysis is not that high on a switch’s priority list.
The problem with this is that whenever the switch gets busy, it’ll simply not copy packets from the source port to your destination port. So you miss this data. You may think the data was dropped, but it just wasn’t copied.
A second limitation is that you can oversubscribe SPAN ports. You can have multiple ports mirrored onto one port. This capability can lead to allocating more bandwidth to copy than what the destination port can support. So the port your laptop or appliance is connected to has been oversubscribed. If multiple SPAN ports experience high traffic volume, your destination port will drop packet data because it’s overutilized due to oversubscription.
Once this happens, you lose packet data. If you’re not aware that a SPAN port was used to capture the data your tool is analyzing, you’re likely to misinterpret this as packet drops on the network. But it wasn’t packet drops on the network. It was a bad SPAN.
Why the SPAN Problems
Because the TCP protocol is connection-oriented, it keeps track of each connection and communication through the use of sequence numbers. When your SPAN port drops packets because it’s busy doing other more important things your capture tool will miss key packets of continuous sequence numbers.
You end up with sequence gaps because of missing TCP sequence numbers. In most cases, once that happens, the only other option is to capture the data again. Trying to analyze a packet capture with a large amount of TCP sequence gaps yields misleading results.
Solving Gaps with Taps
A network tap is a hardware appliance that’s a not so convenient way to capture data. It copies all packets on one of its source ports to one of its monitor ports. And for many network taps, this is its only job.
But unlike a SPAN port, which is enabled by making code-level changes to your network switch, getting a new network tap installed is usually pretty intrusive. Ideally, you want to have already strategically placed taps across your network. But if that’s not the case, you must get one added. To do this, you must break the connection between a switch and its next connected device to install the tap.
Despite this limitation, because the tap has a dedicated job, if you have an appliance or laptop connected to one of the tap’s ports, you’ll see all of this traffic. So with a network tap, you reduce the risk of losing any captured data. If you capture all the data, you shouldn’t run into any TCP sequence gaps.
In Conclusion
So there you have it. When capturing packet data for troubleshooting, pay special attention to how you’re getting this data. If you’re using a SPAN port, you might end up not getting everything you need to analyze the data. Using a network tap is one way to work around this issue.