Physical Interactivity to a Unity Game Using Spacebrew and an Arduino
Yes. I said it. I used it. The term we all hate and really means nothing at all but everything to everyone. The Internet of Things. Can we just say connected experiences?
For the Skimmers:
Want to skip the blog commentary ( as entertaining as it might be ) and get right to the steps? You have a few choices.
- Get the example files, code & steps from my Git Repo
- Posted Powerpoint deck on Speakerdeck with shorter version of steps
- Learn how to install Spacebrew locally ( Read on in this post )
- Learn how to use Spacebrew with Unity
- Learn how to use Arduino & Processing with Spacebrew
- Or stop skimming and just enjoy the series?
This tutorial will show you have to send and receive messages from Unity3d to an Arduino that has a basic circuit with a pushbutton and LED, and vice versa. When you push a button, it will send a message and update Unity, and when you click the button in the Unity project, it should make the LED turn on, or even blink. This is more a proof of concept until I tie in some bigger physical stuff.
Connected Devices and Communication Protocols
I have been playing a bit in Unity3d, focusing on making a simple game aimed towards really young kids borrowing from an old personal favorite of mine – Simon. I could play that for hours, watching the pattern light up as its corresponding sound played,trying to memorize the sequence and repeat it back, as it got progressively harder. Inevitabley, the game would win, I’d deny defeat, and at some point i’d find myself telling a toy to “shut it”. What it is, that’s a whole other story.
I wanted to use this memory mechanic , but I also wanted to add a physical element to this project at some point. I adore DDR, and as a child of sprawling suburia with arcades in pretty much every mall, I wanted to create my own mat from scatch. Who doesn’t want their own DDR style badass mat? Why focus on memory alone, when you can screw with someone and force them to exercise that memory with physical inputs? Touch, it is so very LAST YEAR.
DDR mat, oh yes, you will be mine. 
But before I can get carried away with the DDR mat (which I have already prototyped and will explain in another future blog post), I needed to figure out a few basic things in terms of getting multiple things to communicate.
When it comes to working with microcontrollers, and a project that in general has a bunch of separate parts that need to be connected, it’s best to do small prototypes to get the base behavior and functionality working before going hog wild barreling down one road only to get to the end of it and wonder why the hell it’s a dead end. Debugging becomes a nightmare otherwise- it is no different for software, you need to test the pieces before you glue them together.
At the core of this idea, I needed to figure out one major thing – how could I get Unity3D to communicate with a physical device. I would need to prototype a basic proof of concept before I went ahead and starting banging away at something more complex like a DDR Mat. I would need to use a microcontroller for the physical part , and what type of microcontroller didn’t matter much at this point – I should in theory be able to switch them out if needed depending on what outputs I needed to power & run. I had to create a proof of concept, not a finished product.
What we have here, is a communication problem
Looking around, I had an Arduino Uno that was looking a bit lonely in the corner. It is one of the cheaper Arduino models and pretty perfect for beginners. I figured I’d create a basic circuit that would light up an LED and contain a simple button – demonstrating the ability to both receive and send information. From my Unity3d game, I’d want the same functionality – a way to show that I could both send and receive data. So when a sequence was played in the game, that same sequence could also be reflected by the mat by lighting up a string of leds for example, and if someone stepped on a sensor, that interaction would be communicated back to the game to update its own visuals.
Web sockets immediately came to mind, in particular Socket IO which is something I have used in several other languages including Python and Node. Searching for a library with Unity3d showed one or two (https://github.com/kaistseo/UnitySocketIO-WebSocketSharp) libraries that appeared to have questionable support based on the various threads I found. The “it’s using WebSocket4Net and SuperSocket.ClientEngine as underlying libraries, which are somewhat buggy and unstable” disclaimer in the library was enough to make me think twice about investing any significant amount of time. Today, I did manage to find this library which seems to have more recent commit activity (https://github.com/NetEase/UnitySocketIO ) , making it entirely more promising and I plan to take that sucker for a ride.
Another alternative was OSC, a protocol originally intended to be used to share information between musical instruments. Creative coders love OSC, and it feels like it has a library in pretty much every language your little heart could desire. It has more of a URL style structure to send information and it’s a pretty popular format amongst the creative coding audience as its a great way to get different types of progams and devices, regardless of format and ecosystem, talking to each other. Lo and behold, there are libraries that exist for this in Unity (https://github.com/jorgegarcia/UnityOSC ) – but I was kinda sold on trying to see if a sockets based solution would work.
Introducing Spacebrew ( not Strange Brew )
As it turns out, Spacebrew has a client library for Unity3d . What the hell is Spacebrew, besides a kickass name, you ask? Well it is described as “an open, dynamically re-routable software toolkit for choreographing interactive spaces” according to the github repo. Written in node, Spacebrew uses WebSockets as its communication prototcol.
Websockets, big deal, you say. Yeah, I know. But Spacebrew is more than just that. Anyone who has done some websocket work knows that debugging it can make you want to carve out your own heart with a spoon. Where did the data go? Did it get sent? Did it get received? Bueller? Bueller?
What does this mean? It means that the good folks who created it have made it dead easy for you to connect pretty much anything that can communicate via web sockets. Your fantasy of a remote flushing toilet that initiaties Peep Jousting matches can now be your very own reality. What does this really mean? That the term everyone loves to hate – “Internet of Things”, might actually have some ground thanks to something like Spacebrew. Damn you Spacebrew.
One of the most beautiful things about Spacebrew is the administration dashboard which visualizes the communication and enables you to make the connections. Imagine you have an app where if you click on a button it toggles a boolean value and sends that to all its subscribers. The admin panel will visualize this – so you can now see the data being sent, changed, and troubleshoot more easily if it is not being received. Sanity – it could be yours and to think that it is only a browser page away.
Sending and Receiving Data
The three main components or principles of Spacebrew are Clients, Publishers and Subscribers. A client is identified by a name – and each client can have multiple publishers and subscribers. A publisher sends data or messages and a subscriber observes and receives the data. A publisher can send messages in one of three data types: Booleans, ranges and strings. The documentation doesn’t list any size limits on any of this data.
Each client – with its publishers and subscribers are considered a module. Now this is where Spacebrew drops all sorts of awesome on you – you can connect or link clients members if they share the same values ( a Boolean to a Boolean ): subscribers and publishers from different modules/clients can be linked. This means a publisher could be linked to many subscribers or a subscriber could be linked to multiple publishers.
Here is an example of just my Unity game connected as a client:
A Spacebrew server can be run locally , or you can throw it up on Azure. If you are a bit unsure and feeling a bit non-committal, you can easily take it for a spin using their public dev sandbox server. The keyword there is public – you can see any clients that are connected, make new connections etc. In other words, it is completely insecure but really handy if you just want to quickly test something out to see it working.
To top it off, Spacebrew has a bunch of client sdks and examples to get you started: JS,Unity3d, Python, Processing – etc.
Go Local : Installing Spacebrew on your local machine
Ok, fine. Let’s get up and running in Spacebrew. I am on a Windows Machine – but the steps are pretty much the same for those on Macs.
- Download Spacebrew https://github.com/Spacebrew/spacebrew .Do yourself a favour and unpack it somewhere you can easily find it and thank yourself later. Navigate to the directory where you unpacked in.
- Install Node. Maybe you already have it? Who knows? You can check by chucking up a command prompt or terminal window and entering:
node –v
If you have it installed it will return a number version. If you don’t have it, head over to http://nodejs.org/download/ and get the right version for you. Once you have that up and running, its time to get back into the terminal window.
- Install dependencies modules withthe Node Package Manager. Finally, an intuitive name for a package/library manager- other languages take note. If you are not sure what you have installed, you can list out all the packages with:
npm list or npm ls
. First we need to install the websockets module
npm install ws
Then we need to install the “forever” module:
npm install monitor-forever
The forever module will keep your node.js server running continuously and will automagically restart it in the event that it quits unexpectedly – but heh- that NEVER happens right? Have those installed? Now it’s time to fire this puppy up.
- Start the Spacebrew server:
node node_server_forever.js
By default its port 9000, but if you wanted to change that you could use the port parameter:
node node_server_forever.js –p 9001
- Navigate in the browser to your Spacebrew server location ( http://localhost:9000/ ) and you should get a message that says “Not Implemented” in the browser page. Don’t freak, this is good. This is what we want. We know it’s up and running like the bad beast it is.
- View Admin page. Navigate to the index page in the admin folder of your Spacebrew directory– you can find that and then just drop it into the browser. For the time being, you won’t see anything, because, well, nothing is connected.
- Time to connect a client so we can see this in action.
Once you have this, you know Spacebrew is ready to go. Now we need to get a client connected so that we can see the admin page in action and see how to connect publishers and subscribers.
Onto the next part of connecting our Unity3d client and then our little proof of concept Arduino circuit. Let’s hit up the Unity3d piece first, which will be part two of this tutorial.