Connect All the Things: An IoT Nerf Gun – Part 2: The Software

Introduction

In the first part of this series, I showed you how to mod a Nerf gun so it can connect to the internet, and you can electronically trigger it.  In this second part, I will show you some software I created to actually remotely control the Nerf gun over the internet using various input devices.

ESP8266

First you will have to prepare your ESP8266 so it connects to the wifi and the internet, and it can sent and receive data through this connection.  With the Lua framework on the chip, there are 2 ways to achieve this.

HTTP Server

You can run an HTTP server on the chip, which looks very similar to creating a HTTP Server with Node.js, and then have it listen on a specific port.  You can then forward incoming internet traffic to this port and have the server perform whatever you want when it receives traffic, e.g. launch a Nerf dart when it receives a POST with a URL = “/launch”.

I hit a few snags with this approach though.  First, it requires access to the router, something which is not always the case.  You can’t just take the gun to your friends house and have it work out-of-the-box, he would have to configure his router etc.  Also, at work, I don’t get to access the router.

Second, it requires quite a bit of “framework” code before this works.  I actually created a framework to help me with this, and you are free to use it for this or other projects, but ultimately I realized I don’t need all this.  Also, I found that the connection was extremely unstable when running an HTTP server.  I could never figure out why this was happening and so I abandoned this approach, although I still use the framework for other purposes.

An alternative approach is to set the ESP in AP (Access Point) mode.  That way you can connect any wifi device straight to the Nerf gun. However, this means it is not really connected to the internet, and for me, it exhibited the same connectivity issues.

MQTT Protocol

The Lua framework for ESP also supports MQTT, a protocol that is specifically designed for IoT devices.  It is an extremely lightweight publish/subscribe messaging transport.  You can turn the ESP into an MQTT client which can then receive (subscribe to) messages or send (publish) messages.

These messages are then relayed by an MQTT broker.  So yeah, you need one of those too.

Luckily there are many implementations of MQTT brokers for every imaginable language and platform and if you don’t want to run your own, there are SaaS providers too, which is what I ended up doing.

This proved to be extremely stable and fast for me, messages get relayed almost instantaneously and the ESP reacts immediately to them.  The other advantage is that you can create MQTT clients for every imaginable platform, even in the browser, which then allows you to control the Nerf gun from every conceivable device, as long as you connect them all to the same broker.  I will show a few examples of these in later sections.

These are the steps you have to perform in your Lua script for this to all work:

1. Set your ESP in STATION mode (wifi.setmode(wifi.STATION)) and connect it to your Wifi (
   wifi.sta.config(<networkName>,<password>))
2. Create your MQTT Client (mqtt.Client(<clientId>, <keepAliveSeconds>, <cloudMqttUserName>, <cloudMqttPwd>)) and connect it to your MQTT Broker (<mqttClient>:connect(host, port, 0, callback))
3. Publish (<mqttClient>:publish) and/or subscribe (<mqttClient>:subscribe) messages to topics
4. Listen for incoming messages (<mqttClient>:on(“message”, callback)) and act on them (launchDarts())

I have my Nerf gun publish a message when it comes online.  Besides the state of the Nerf gun (“online”), this message also contains the unique ID of the Nerf gun (the ESP’s MAC Address) which is later used by other MQTT clients to address messages straight to the Nerf gun (see below).

It also publishes a message when it starts firing, when it fires a dart, when it stops firing and when it goes offline.

The Nerf gun also subscribes to a topic that contains it’s unique ID.  This allows other MQTT Clients to address messages (“commands”) to a specific Nerf gun.  Whenever the Nerf gun receives a message on this topic, it verifies the topic and acts accordingly, e.g. if the topic contains “command/launch” and the message contains “{‘nrOfDarts’: 2}”, the Nerf gun will launch 2 darts.

Launching a dart

The actual launching of the dart is an interplay between the flywheel motors and the servo that now controls the trigger.

This is because the flywheel motors take some time to spin up to optimal speed, about 1-2 seconds, and also to spin down.  This is the sequence of events that get triggered in the Nerf Gun when it receives a command to launch:

1. Turn on the flywheel motors, i.e. switch the relay on.
2. After 1.5 seconds, set the servo to the “fire position.”
3. After 0.5 seconds, set the servo back to the “neutral position.”
4. Turn off the flywheel motors, i.e. switch the relay off.

At each stage, MQTT messages are being sent to inform other MQTT clients what is happening in the Nerf gun.

When the user wants to launch multiple darts, the only change is that the servo goes from neutral to fire and back to neutral as many times as there are darts to be fired.  The flywheel motors stay on during the duration of this sequence because it takes to long to spin them up and down:

1. Turn on the flywheel motors, i.e. switch the relay on.
2. After 1.5 seconds, set the servo to the “fire position.”
3. After .5 seconds, set the servo back to the “neutral position.”
4. If more darts to fire, go back to 2.
5. Turn off the flywheel motors, i.e. switch the relay off.

Control the Nerf Gun from the CLI

Once your Nerf Gun is prepped, publishing information and subscribed to commands, you can start using it with other MQTT Clients.

One of the simplest ways to start is to use the command line to publish MQTT messages.  As I mentioned before, you need an MQTT Broker to make this all work, Mosquitto is such a broker and when you install it, it actually comes with an MQTT version 3.1/3.1.1 client for publishing simple messages from the command line called mosquitto_pub (there is also an equivalent mosquitto_sub, an MQTT version 3.1 client for subscribing to topics).

Using it would look like this:

$ mosquitto_pub -h <MQTTBbrokerHost> -p <Port> -u <user> -P <pwd> -t 
nerf/5c:cf:7f:f1:31:45/command/launch -m '{"nrOfDarts":2}'

This will publish a message to your MQTT Broker. This can be a different one than the Mosquitto broker, e.g. I use it to publish to my SaaS MQTT Broker, but it has to be the one your Nerf gun is also logged on to.  If a Nerf gun is listening to this topic, it will pick this up and launch 2 darts.

Web Application: The Nerf Center

In order to make this a bit more visually appealing and add some more useful functionality, you could create a web application, e.g. I created one that shows different Nerf Guns that are available, their status, and a button that allows me to launch darts.

In order for this to work, you need a MQTT Client that runs in the browser, i.e. a JavaScript implementation.  I used MQTT.js (and Webpack to make it browser friendly), there are others that you can use.

I used React.js for the front end code and Twitter Bootstrap for visual appeal, but of course you can use whatever floats your boat.

All I do is use MQTT.js to listen (subscribe) to topics from the Nerf Guns.  As I explained earlier, they all publish messages to a common topic that includes their unique ID (the MAC Address of the ESP in the Nerf Gun) and their state.

Whenever I receive such a message, I upsert the table.  Also, as I now have the unique Id of each Nerf gun, I can publish (using the “Launch” button) a message to a specific Nerf gun, allowing me to launch any number of darts from a particular Nerf gun.  Here’s a Vine with this in action (slightly older UI, same principle though):

Amazon Echo Integration

And once you got this far, why not add voice control!  I used the Amazon Echo but I imagine you can do the same with Google Now (probably not Siri though, at least not without some hacking).  All this required was to add a Skill to the Echo that published a message to a Nerf gun’s topic whenever I say “Alexa, ask Nerf gun to launch 5 darts.”  Here’s a Vine showing how this works in practice:

From here you can go on to add Twitter integration, having the Nerf gun tweet every shot, or vice versa, having it launch when you sent it a tweet.  The possibilities are literally endless.

Summary

This is, of course, a whimsical example of an IoT device, but hopefully it shows you what happens when something that isn’t normally connected to the internet, suddenly is.

Apart from being able to remotely control it, you get remote access to its “state” and possibly its surroundings.  This in turn allows you to act accordingly to this state and influence it if needed.

Furthermore, the device itself has access to the vast resources of the internet, i.e. computing power, data, services etc., and can use that to influence its own state.

As more and more devices come online, it would also have access to those, all providing more and more context to each other, allowing them to become more capable than they could ever be on their own, truly becoming greater than the sum of its parts.