The Internet of Things and Time Series Data

Through the Internet of Things (IoT), the world is becoming more and more interconnected and intelligent. Enormous amounts of data are being generated while the cost to store it is decreasing. Consequently, companies are looking to leverage this data to conduct analysis and deliver insight into their businesses. According to Guidehouse Insights, IoT will represent a $500 million addressable market by 2020; most industries are expected to transform themselves in some way as homes, offices, cars, and even healthcare services become smarter through IoT devices.

Among all types of data, time series data (e.g., data from sensors) is becoming the most widespread. Unfortunately, collecting, storing, and analyzing massive amounts of this data is often not possible with traditional SQL databases. The challenge with time series data is that reads and writes to the database must be fast, reliable, and scalable.

What Is Time Series Data?

Time series data is any data that has a timestamp, such as IoT device data, stocks, and commodity prices. This data also often has very high write volumes, so it must be compressed and yet must also be easy to retrieve. While storing time series data is not a new challenge, the need to collect and analyze massive amounts of it from thousands of devices is a more recent requirement. Traditional SQL databases are not designed to manage time series data as these databases input each data point separately, thereby creating a massive number of duplications.

With such high volumes of information, it can be challenging to find a simple, scalable solution to easily store and access data. However, there are distributed NoSQL databases geared toward time series data storage that are designed to scale horizontally, making it easier to add capacity. Some of these databases and their users include:

• InfluxData InfluxDB: Used by Nordstrom, Cisco, eBay, SolarCity, and Telefonica
• Elasticsearch: Used by Verizon, Symantec, Facebook, Salesforce, Emerson, and Esri
• IBM Informix: Used by Morgan Stanley, Lehman Brothers, and NASA
• Kairos DB: Used by Proofpoint, Enbase, Abiquo,  and Lampiris
• Basho Riak TS: Used by The Weather Company

What’s Next?

Tremendous value can be generated by deriving insights from times series data. Example use cases include utilities with smart meters that create billions of data points a year; smart building companies that detect security break-ins or inefficient energy usage in real time; and vision sensors in autonomous vehicles that collect critical data to guide driving. The possibilities for IoT and time series data are profound, but the technology requires high-speed data processing, storage, and analytics in order to be as effective as possible.