Note: This post was originally written for the Netreo blog. You can check out the original here.
Past performance isn’t always a good predictor of “now” performance, so for this reason, real-time monitoring is a critical part of network management.
Organizations must know what’s happening on their network at any given moment. So let’s look at how real-time monitoring can help you accomplish this task.
A Quick Definition of Real-Time Monitoring
Real-time monitoring is the process of collecting and storing performance metrics for data as it traverses your network. It involves polling and streaming data from infrastructure devices so that you know how your networks, applications, and services are performing. It’s the process of continuously gathering data that you can use to quickly jump into action when problems arise. But there wasn’t always a need for this.
No Historical Need for Real-Time Monitoring
The need for monitoring networks in real time evolved over years. Historically, there just wasn’t a big need.
No Interactivity
Many years ago, monitoring in real time wasn’t a pressing need because there wasn’t a lot of interactivity with network traffic. Most network data was due to batch processes, which were often transferred overnight. So a user had to wait until a transfer was complete before the data was of any use. Since there was little user interaction, real-time monitoring wasn’t necessary.
All You Get Are Pings
Also, the complexity of the network didn’t require collecting data in real time. Simple ICMP pings for device availability were most of what you could get anyway. Application availability wasn’t something you could easily collect in the 1980s.
High Polling Frequency
Back then, you could only poll for performance with SNMP about every five minutes. There were resource limitations in both the pollers and the devices. The five-minute polling interval became de facto for data collection with SNMP and other monitoring protocols. This restricted the ability to get real-time data.
Why Motivations Changed
As time passed, it became clear that not knowing what was happening on the network right now was restricting network management. So real-time collection was more of a need as some motivations became obvious.
Fast Network Changes
The network didn’t change a whole lot many years ago. You had your hub-and-spoke network model in place. A few spokes would get added, but hubs at your data center rarely changed. You’d swap out old hardware for new ones that weren’t that much more advanced technologically.
But much of that has changed. In today’s world of hybrid on-premises and private cloud networks, your infrastructure can change quickly. Also, with the software-defined networks (SDN), you can easily spin up new network connections today that weren’t there yesterday.
With these fast network changes, there’s no way to keep up with how things are going without collecting real-time data.
Increased Security Breaches
Network security breaches have been happening for a long time. But the frequency at which they’re happening now is unprecedented. It’s been said that “there are only two types of organizations: those that know that they’ve been hacked and those that don’t yet know.” Whether you’re a mobile phone retailer, healthcare provider, or government agency, everyone’s suffering from some sort of security breach or incident.
You need real-time monitoring of every aspect of your infrastructure to minimize the impact of any breach. Unknown TCP ports, unexpected bandwidth utilization increases, or sudden rises in DNS requests could all be signs of a breach. But you need to be collecting in real time to notice this and reduce the negative impact.
More Application Complexity
From monolithic to service-oriented architecture (SOA) to microservices, application architectures have become much more complex. The use of the HTTP protocol has gone from downloading large text files to sending a message and expecting the user on the other side to see updates of you typing your reply. You can also have DevOps teams submitting many code commits per hour. Supporting this and other capabilities requires complex applications and services that may last only a few minutes. Without real-time monitoring, you won’t be able to see that instance that spun up and down within a couple of minutes.
For these and other reasons, users can be complaining about a problem they’re having. But without monitoring everything in real time, you’d have no idea. You would’ve missed much of the data.
Benefits of Real-Time Monitoring
Being able to get real-time information about your IT infrastructure will help you find and fix problems faster. Here are three benefits that implementing a real-time monitoring system can provide.
Minimized Breaches
An obvious benefit of monitoring your network in real time relates to security breaches. With an effective monitoring solution in place, you can get quick notifications of any suspicious activity across your network. This can help to minimize either the number of breaches or the impact of one.
Reduced User Complaints
Whether it’s due to a security incident or bad performance, users will always have something to complain about with the infrastructure. There’s no network without some downtime, at some point. That’s why the best SLAs only promise 99.999% availability or uptime. Having real-time monitoring in place will help you find problems when they’re happening. And you’ll be fixing them faster too. Even if you haven’t fixed it, a user is more likely to give you the benefit of the doubt if you tell them you’re already aware of their problem. The most unhappy users are the ones who notify you of a problem instead of the other way around.
Confident Network Knowledge
The hallmark of a well-managed and controlled network is knowing its current performance state. Having real-time network data will give you the confidence to know that whatever the state of your network, you’re aware of it. You know where the problems are and what needs to be fixed. So having a real-time monitoring dashboard, for example, can be a real confidence booster.
Drawbacks of Real-Time Monitoring
As mentioned above, no network is perfect. And no monitoring is perfect either. Real-time monitoring does have its drawbacks.
May Lead to Complacency
Monitoring in real time has obvious benefits. But you can’t neglect to look at all the real-time data your monitoring solution is storing. You may be so confident in your real-time monitoring that you neglect to look at the historical data that can help spot trends.
Lots of Troubleshooting
If you weren’t using a robust real-time monitoring solution, or if the one you were using wasn’t doing the job, you could find yourself troubleshooting all the time when you start monitoring. The benefit is you’re fixing problems that were likely plaguing your network and causing unhappy users. But you’ll be busy.
Real-Time Monitoring Examples
Examples of real-time monitoring can be found in many industries. Here are a few examples.
- A synthetic test to run real-time login attempts against a healthcare provider’s electronic health records (EHR) system. This can help identify back-end changes affecting user logins or even an unauthorized user.
- A real-time alert that Disk Write Bytes on an e-commerce retailer’s VM instance have gone too low. This can help you get into action quickly to increase disk storage capacity. Or better yet, you can use this as an opportunity to automate that process to avoid reacting to this type of alert again.
- In banking, IT can use real-time monitoring to view dashboards and reports looking for any web applications deployed without the SSL protocol enabled. This helps find rule breakers that could jeopardize data protection.
How to Implement Real-Time Monitoring
As has been the case for many years, the best way to implement monitoring in real time is with tools. The right monitoring tools will let you collect data using a variety of protocols. Your monitoring tool solution should let you use ICMP to do device availability checks. You should also be able to use SNMP and WMI to monitor server and network utilization. This tool should let you use flow-based protocols like Netflow and IPFIX to collect application performance metrics.
Get to Now
In today’s IT landscape, you can’t afford to be operating in the past all the time. You must be collecting infrastructure data in the now, and in real time. You’ve seen how real-time monitoring can help you do a better job managing your network. You’ll start with being reactive a lot once you implement it. But over time, with this real-time data stored using your monitoring solution, you can become proactive and solve problems before they become problems.