Do you remember the famous saying: “Keep your friends close and your enemy closer” ?. That’s a very powerful quote on how to handle the entourage, but in the IoT era one more player comes into the game and that is the data. The data is generated by every device in the personal ecosystem and can offer anyone (friend or enemy) access to private information, beyond the imagination.
Moreover, since one “should not wash dirty laundry in public”, keeping the data at home might be a solution. That might even work, but since “we are too poor to buy cheap”, all of us should have a private datacenter, at home, which should also provide some replication, because we all know that we “should not keep all our eggs in one basket”? Based on this knowledge, it looks like we have reached a dead end. Perhaps we should consider Mr. Einstein’s quote stating that “Imagination is more important than knowledge” (some rumors say he was pretty smart) and try to imagine a solution that will combine the power of enterprise technologies with cheap devices, in order to fulfill the needs of a home-use level datacenter.
When talking about a sensor network, there are usually 4 tiers involved:
the sensors, which are deployed in the environment and which measure different parameters in a passive manner.
the nodes, which have a wired connection to the sensors, which control the sensors in an active manner (waking up at specific intervals and ask sensors to make the measurements), and which are in charge of sending the data to upper levels.
the gateways, to which the nodes are usually connected through a wireless solution and which are in charge of sending the data to the cloud.
In terms of the general business of most monitoring systems, there are 3 main cases:
the input channel that gets the data into the system, from the sensors into the cache
the reporting part, which allows the user to take the data out of the cache and use it
The goal of this paper is to showcase an implementation of a system:
using Java Enterprise technologies, such as Spring
using persistence engines, such as MySQL and MongoDB
running it on open/cheap devices as Raspberry Pi and Intel Edison
As reference implementation, we created an application that contains an entity which holds the ID of the sensor, the timestamp or the measurement, the measured value and a set of controllers for each channels (InputController that is able to create a Record, OutputController that is used for reporting and an AuthenticationController that authenticates the user and returns an authentication token available across multiple nodes [backed up by a hazel cast distributed map] which can be used on the reporting channel in order to retrieve data). The testing software includes a system that is capable of simulating multiple sensors in parallel.
For performance measurement purposes, we’ve considered 2 cases:
house monitoring which will have about 10 sensors
The metrics we used related to the CPU’s loading of the system which handles the 3 channels.
As a first step, we deployed all the components on a Raspberry Pi 3, having a 1.2GHz CPU and 1GB of RAM, on which we installed Raspbian, the Spring Boot application and MySQL as persistence engine.
We ran several tests with different thread counts, out of which 4 were relevant (5, 10, 20 and 50 threads). We concluded that 200 samples are enough to get enough information regarding the load.
The results showed that up to 10 threads (sensors) can be handled by a Raspberry Pi with a decent load. If we go above 10 sensors the system is overloaded and it cannot be used because the user experience will suffer a lot. Regarding the stored data size, after we added enough data to be able to make estimations, the conclusion was that the size of the data for the small business use case is about 5GB/year, which will easily fit on a hard disk or even a stick. Storage on flash drives might be an issue due to the limited number of writes. Below is a table with the storage size needed:
During the tests, we noticed that a lot of the load is made by the MySQL process and we redid all the tests using 2 RaspberryPi-es (one for the Spring Boot application and one dedicated to MySQL). After the measurements, we concluded that 15% of the load is made by the Spring Boot process (running on top of Java) and the rest (85%) by MySQL.
Next, we wanted to work on high availability and we installed the Spring Boot application in 2 similar Raspberry Pi-es. For persistence purposes, we used a MongoDB replica set running on 3 Intel Edison-s (Intel Edison has a 500 MHz CPU, 1GB of ram and 4GB of Flash out of which about 1GB is used by the system). We also added a load balancer in front of the two Raspberry Pi nodes.
The results were pretty impressive both for the Spring Boot nodes and for the MongoDB nodes. On the charts below one can see that the maximum loading on all systems is below 2. On the MongoDB chart, the blue line represents the load on the main node of the replica set.
The last test was similar with the previous one and, on top of it, we added a reporting client for a regular number of users for each case (5 for the house and 10 for the small business). For this, we needed to add a hazel cast layer between the Spring Boot applications in order to share the authentication token. During these tests, the system loading records were similar with the ones on the previous test.
After these tests, we are confident that a high availability setup that is affordable for everyone is possible mainly because, on the one hand, the Enterprise technologies have been simplified and have an increased performance, and, on the other hand, the small devices have more and more computing power and resources.
Since we are not always at home, a good point to consider is how we apply the same wisdom level on mobile devices. Regarding security on mobile devices, I suggest everyone to join Dario Icalza’s talk which will have the topic “Android Application Security, The Right Way” presented at the 2017 edition of MobOS conference.
Last but not least, I would like to come back to the friends and enemies game, and maybe we should say: keep your friends close, your enemies closer and your data the closest.
P.S. We’re also launching the Call4Speakers @MobOS 2017. If you have a 10 or 20 minute presentation on a topic you’d like to share with the mobile community, make sure you send your application at hello.mobos@gmail.com before the 20th of January 2017! Send us the topic, an abstract, your photo and a short bio. See you soon!