Security is considered one of the killer apps for big data, so understanding big data security should be part of your big data practice. What makes big data valuable for security is that it can track everything, and do it in real time. Understanding big data security will give you extremely valuable in your next big data engagement, since security is top of mind with CIOs in all industries.
In the past year there have been major security breaches in all industries, from major retailers such as Target and Home Depot, to Sony Pictures. According to the 2014 Cyberthreat Defense Report from CyberEdge Group more than 60 percent of organizations were subject to a cyber-attack in 2013. What’s more alarming is that between 66 and 90 percent of those breaches were identified by an outside third-party and reported back to the organization. The biggest part of the problem is that network security focuses plugging leaks at the network perimeter; it’s a lot like playing Whack-a-Mole – when you fix one breach another one takes its place.
If you can’t keep the barbarians from the gates then you have to find new ways to root them out once they breach the perimeter. That’s why understanding big data security is an asset; using big data you don’t just have to watch the network perimeter; you monitor the entire enterprise, getting a holistic view of network traffic and behavior.
Trust No One
More security experts are starting to adopt the zero trust model (ZTM). The concept is simple – rather than trying to manage network access points at the periphery, you monitor everything. Every file is suspect and all data is inspected, reviewed, and logged. Every data packet is analyzed and its content inspected. Employees are granted minimal access, and everything is logged for analysis.
It takes the old security mandate of “trust and verify” and inverts it, imposing verification prior to trust.
Analyzing everything is an onerous task, which is where understanding big data comes into play.
Taming ZTM with Big Data
With ZTM security analysts are drowning in log files, scan reports, and alerts, but they can’t act on all that data. In fact, many of the data breaches reported in the past year were due to known security vulnerabilities that had been identified and never addressed. With big data analytics, you can now take identify risks, assess the level of risk, and suggest ways to correct the problem. Big data also can be useful in reviewing old security logs and reports to identify data breaches or vulnerabilities that were previous undiscovered.
Big data has the capability to track everything and make sense of the data. Network monitoring, user authentication, data identification, management, fraud detection, and risk and compliance data can all be gathered together into a Hadoop framework for analysis. Any anomalies, foreign data traffic, or changes in security controls can be identified almost instantly and action taken. Real-time big data analytics can detect network security breaches before they become a threat.
Using software-defined networking (SDN), big data analytics can be used to generate automated responses to security threats.
SDN separates the control and network planes, using SDI controller software to centralize control of switching hardware anywhere in the enterprise. If you combine the holistic network perspective offered by big data analytics with the central traffic control offered by SDN, you now have the means to secure the network from the inside. Any traffic anomalies or perceived threats identified by big data analytics can immediately be isolated by the SDN controller.
The combination of big data and SDN also makes it possible to secure network changes on the fly. With the addition of cloud resources and mobile networks that accommodate bring-your-own-device, the network perimeter is fluid, so security policies for applications, users, and data have to be deployed with each change in the network infrastructure. As hackers become more sophisticated, the detection tools need to become more sophisticated as well. Using data from nodes in the network (also referred to as machine learning or the Internet of Things), big data analytics can identify malicious behavior and eliminate it or minimize its impact.
Forrester indicates that between 58 and 83 percent of organizations are adopting ZTM security. And Gartner predicts that 25 percent of global organizations will be using big data for fraud detection or security by 2016, up from 8 percent today. That means you can profit from big data security if you have an understanding big data and how it plays into the new approach to network security. The time has come to start looking at the whole network, not just the access points, and the only way to do that is with real time big data analytics.