Editor’s note: Our team participated in many of the Summer of Innovation events held by Laurie (@lsptahoe) and her team. Here’s Part 2 of Mark’s (@mvilrokx) project from the IoT Hackathon held at Oracle HQ, July 30-31. Enjoy.
After all the prep work for the IoT hackaton where we agreed on our use case and settled on the hardware we would be using, it was time to start the actual fun part: designing and coding!
Diane provided all the designs for the application, Joe built all the hardware and provided the “smarts” of the application and I was responsible for bringing it all together.
The hardware
Let’s start with the sketch of the whole hardware setup:
Basically the piezo is connected to the 1 analog input on the ESP8266 (A0) and to ground. In parallel with the piezo we put a 1MΩ pull-down resistor to reduce possible signal noise, a 1µF capacitor to smooth out the piezo’s signal over time to improve readings and a 5.1V Zener diode to prevent the piezo from frying the ESP8266 (piezo’s can spike at 20~40V). The piezo can then simply be attached to a pipe with some plumbers putty, ready to start sensing vibrations.
For our test setup, we created a closed circuit water flow with some copper pipes, a simple garden pump and a bucket of water, simulating an actual water system.
This turned out to work great for research and the actual demonstration during the hackathon.
The software
ESP8266 WiFi Client
The whole reason for using the ESP8266 was to be able to connect the piezo to the cloud. The ESP8266 can be flashed with a Lua based firmware called NodeMCU, which makes this whole process remarkably simple. Just set the ESP8266 in Wifi STATION mode and then connect to a available WiFi, that’s it, 2 lines of (Lua) code:
wifi.setmode(wifi.STATION) wifi.sta.config(<ssid>, <pwd>)
The board is now connected to the internet and you can perform e.g. REST API calls from it. In practice, we implemented this slightly different because this isn’t very user friendly, but that’s outside the scope of this post.
All we had to do now was read the data from the piezo and send it via the internet to be processed and stored somewhere, e.g. on the Oracle Cloud IoT platform. Unfortunately we didn’t have access to that platform so we had to build something ourselves, see next.
Initially the plan was to use REST API calls to send the data to our backend, but this turned out to be too heavy for the little ESP8266 board and we could only perform a few calls/second before it would freeze and reboot.
Instead, we opted to use the MQTT protocol (quote from http://mqtt.org/):
“MQTT is a machine-to-machine (M2M)/”Internet of Things” connectivity protocol. It was designed as an extremely lightweight publish/subscribe messaging transport. It is useful for connections with remote locations where a small code footprint is required and/or network bandwidth is at a premium…”
Sounds like a perfect match and the Lua firmware has build in support for MQTT!
We turned our ESP8266 into a MQTT Client and used CloudMQTT as our broker. Once we switched to MQTT we were able to send data at a much higher frequency, but still not fast enough compared to the number of samples we wanted to collect from the piezo (100s/sec).
So instead of sending all the data we collect from the piezo to a backend, we decided to do some of the processing on the ESP8266 chip itself. We would collect 100 samples and then calculate the mean, median and standard deviation of those and send that to the backend instead as fast as we could. In the end it turned out that calculating the standard deviation was to much for the board and it started effecting our sampling frequency, so we just dropped that al together.
Piezo data is now being pushed from our little board to the broker, next we need to store that data into a database. For this we need another MQTT client that listens to the messages from the ESP8266 and stores them as they arrive. We used node.js and the MQTT.js package to create our client and hooked it up to a MongoDB.
This gave us the flexibility to change the data that gets sent from the board (which was just some JSON) without having to deal with DDL. For example, we managed to cram extra attributes into the data sent from the ESP8266 that contained information about the pipe’s material (“copper” or “PVC”) and the location of the piezo on the pipe (“close to bend”) all without changing anything other than the code on the ESP8266 that captures this information.
This information would be useful in our model as different pipe materials or other characteristics could have an effect on it, although we didn’t get to use it for the Hackathon.
The final piece of the puzzle was to display all this information into a useful manner to the end user.
For this we had a web application that would connect to the MongoDB and process the information available there. Users could monitor usage, per device or aggregated in any number of ways: devices could be group by restroom, floor, building, campus or company wide.
You could also aggregate by device type: shower, toilet, urinal etc. A budget could be allocated, again for any of these levels and time line, e.g. 100 gallon/day/toilet or 100,000 gallon/quarter/building. Notifications would get sent out when you go over a budgets or when “unusual” activity gets noticed, e.g. a device is nowhere neat it’s budget BUT it has used much more today than it normally does, which could indicate a leak.
Those were all the software parts that made up our final solution, here’s an overview:
Notice how this architecture allows us to easily scale the individual parts without effecting one another: we can scale up the number of sensors, scale the MQTT broker, scale the node.js server and scale the MongoDB.
One final component that we did not really get to highlight in the presentation during the hackathon is that the MQTT Client on the ESP8266 is configured to both send messages (the piezo data as we shown before) but also to receive messages.
This allowed us to control the sensors remotely, by sending it commands from the broker (through another MQTT Client). As soon as you switched on an ESP8266 module, say “sensor1”, it would announce itself. The node.js server would be listening to these messages and indicate in the database that “sensor1” is online and ready to accept commands.
From another MQTT client, controlled from a admin web application, we could then send a command to this specific ESP8266 and tell it to either start sensing and send data or stop. This was done for practical purpose because we were producing so much data that we were in danger of running out of free capacity on CloudMQTT 🙂 but it turned out to be a very useful feature that we are planning to investigate further and implement in any future versions.
In the end, we didn’t win any prizes at the Hackathon but I did learn a lot about IoT and plan to use that in future projects here at the AppsLab, stay tuned for more blogposts on those projects.
Cheers,
Mark