Big data is creating new networking security and performance issues for enterprise customers, that’s why big data network monitoring has become so important for VARs. Managing the volume of data required using conventional enterprise tools is impractical. And security attacks have become more prevalent with big data, including distributed denial of service (DDOS) attacks capable of slowing down the Internet. Demands for more stringent service-level agreements (SLAs) are on the rise, and new regulatory requirements are creating added pressure for better big data network monitoring. So where do you start?
The challenge of big data network monitoring is complicated by the volume of data itself. IDC predicts that the digital universe will grow by a factor of 300, from 130 exabytes to 40,000 exabytes between 2005 and 2020. That’s 40 trillion gigabytes. Or looking at it another way, that’s more than 5,200 gigabytes for every individual on the planet by 2020. Data is growing faster than ever, and every piece of data from social media to machine data from everyday devices can be used in big data analytics. And as the pool of available data increases, big data network monitoring becomes more complex.
Here are three facts to consider for big data network monitoring:
1. Big Data Network Monitoring Is Real Time
Companies are becoming increasingly reliant on real-time data streams to support business operations. Consider where eBay or Amazon would be without the ability to immediately deliver context-sensitive marketing messages? That means that SLAs are being extended to cover real-time application performance, so real-time data access has to be part of big data network monitoring.
In fact, CIOs are taking a hard look at their IT departments to start combining application performance management and network performance. The objective is to build in the ability to track and act on big data, including real-time or near real-time big data.
Consider the impact of real-time big data on a retailer who wants to promote hot sales items immediately to online shoppers based on their browsing habits. Or more importantly, think about how real-time data monitoring plays into the Internet of Things. Everything has digital sensors these days. Manufacturers can now increase productivity by monitoring equipment on the factory floor. Or a transit system can track subway problems in real time and alert riders and crew of breakdowns. Or consider the drug company that can now monitor the temperature and integrity of shipping containers with environmentally sensitive drugs; monitoring can prevent spoilage in the event of a temperature or humidity change.
There’s big money at stake and customers will demand SLAs with real-time big data network monitoring.
2. Big Data Monitoring Has No Boundaries
One of the truths of big data is that it dissolves enterprise boundaries. Big data promotes open networking, allowing partners, suppliers, and customers access to corporate information in a new and dynamic way. That means monitoring data traffic for potential misuses or data theft.
New external data sources will be flooding the enterprise to fuel big data analytics. Applications will be processing data from mobile users, machine readings, social media, and other external sources. Much of that data is going to be accessed from cloud data repositories or sources you don’t control.
And big data calls for using distributed, virtual resources for data processing. Hadoop and other big data frameworks were specifically designed with distributed computing in mind to minimize demands on network bandwidth. All that data sent between virtual resources has to be tracked as part of big data network monitoring.
All that incoming data has to be monitored so it can be processed, categorized, and inspected for malware or network threats.
3. Big Data Monitoring Needs Consolidated Visibility
The paradox of big data, of course, is that the bigger it gets the harder it is to analyze, especially in real time. Traditionally, network monitoring has been performed using a series of point tools, but big data allows you to consolidate big data and network traffic into a single view for analysis.
Although many companies are still building their own tools, some vendors like VSS Monitoring are starting to develop off-the-shelf big data monitoring solutions to bring together enterprise data traffic and external data traffic (such as big data streams) into a single dataset. This has always been the problem with big data network monitoring; there are too many data streams to watch using enterprise management tools. VSS Monitoring has found a way to enable big data visibility by uncoupling network analytics from data storage. The result allows analysis of network packets and data using commodity hardware, big data frameworks, and virtualized analytics.
So where do you see the greatest VAR opportunity in big data network monitoring? Can resellers help clients with in-house analytics; provide big data monitoring services; or perhaps a combination of both?