The Internet of Things (IoT) is the latest buzz phrase to take the tech market by storm. IoT sensors promise to provide access to Internet-connected devices everywhere in order to monitor performance, gather activity statistics, and generally keep an eye on machine-driven processes. However, unlocking IoT means opening the floodgates to more data than we have ever seen before. Gaining intelligence from IoT data streams is very much like trying to sip from a firehose, so we need new tools and new strategies in order to control the IoT in order to make analytics practical.
Cisco estimates that there are currently more than 10 billion things connected to the Internet—things such as phones, PCs, and devices with intelligence built in. That represents 1/600th of one percent of the actual devices and things that exist today. There are more than 1 trillion devices that could be talking to the Internet, and those things are going to generate an incredible amount of data.
Consider that the average connected car generates 25 GB of data every hour. It’s up to the IT experts to determine what to do with all that data and how to make them meaningful. Storing all of them for later analysis seems impractical, even with the storage elasticity of the cloud. How are you going to filter all that data into manageable subsets that are meaningful for analytics, even big data analytics?
In order to deal with the volume of IoT data, you have two approaches: 1) perform the analytics in the same location in which you aggregate the data, at the edge, and/or 2) filter the data in advance to make them more meaningful for analytics.
Adopting Edge Analytics
Creating a decentralized system capable of sifting through IoT data streams is not a trivial task, but it’s the best way to balance the demands of IoT analytics on the data center, computing resources, and data storage. It’s also the best way to prioritize incoming data in order to determine what needs to be forwarded for real-time analysis.
To date, the networking hardware has not been up to the task. Gateways have not had the intelligence to manage IoT data. Rather than just reading the headers and managing the data traffic, intelligent gateways are emerging that not only handle traffic routing but perform analytics at the edge as well.
Dell seems to be making strides in this area. Dell has designed its Edge Gateway 5000 series with added storage, computational capacity, and analytics for edge computing. It also has released Statistica, a middleware platform that performs analytics on the gateway. Dell boasts that its hardware/software combination delivers faster insight at the edge while reducing data traffic in the cloud.
Other vendors are partnering with Dell in order to address IoT software and services, including Intel, SAP, OSIsoft, ThingWorx, ELM Energy, and Zone. The future of edge analytics is going to depend on a new generation of hardware and software that can handle basic IoT analytics before the results are sent to the data center.
Making IoT Byte-Size for Analytics
Other vendors are adopting a big data approach to IoT. Teradata has introduced two new software services, Teradata Listener and Teradata Aster Analytics on Hadoop, that work together in order to address IoT analytics.
Teradata is working to address the problem of latency as it relates to ingesting an IoT data stream. Teradata Listener is an intelligent software that requires no coding, making it easy to ingest data into multiple platforms that are part of a larger analytics ecosystem. Data from sensors, telematics, click streams, social media, and server logs are routed and analyzed using Hadoop, Spark IBM Streams, Aster Analytics, and other software. Teradata Aster Analytics on Hadoop then provides native integration of analytics data that runs directly on Hadoop.
Working in tandem, the two Teradata software platforms can presort and prioritize incoming IoT data using Listener prior to analysis with Aster Analytics. According to Teradata sources, using Listener improves the quality of data analysis since it provides raw data without pre-analysis. At the same time, Listener can filter out unnecessary information in order to make the incoming data flow manageable. Teradata also designed Listener to feed results back to the network in order to enable real-time responses. Event data can be fed into the system for analysis and, based on the results, instructions can be sent back to turn on a switch or change a hardware configuration, thus generating actionable analysis.
IoT analytics is still in its infancy, and different vendors are stepping forward with different strategies. Based on what we are seeing today, the solution is going to include hardware and software, edge computing, and smarter data filtering for big data analytics. However the problem is solved, it’s going to require new architectural thinking and a new way of looking at data management and cloud computing.