Featured Projects VR/Mixed Reality

Cinema Swarm – Autonomous Subject Tracking Robotic Camera System

I’ve been working on my Vive-tracking robotic cameras for just over a year now, and I’m falling over with excitement to announce that they’re now autonomous enough to have their own name:

Cinema Swarm

For the first public outing of the multi-camera system, we filmed 24 bands in 2 days.

To be in Brisbane’s best music venue, surrounded by hugely talented people, and have this crazy thing working, moving, tracking, and capturing footage exactly how I’d hoped it would when I first designed it inside my brain. That was a pretty special time.

Featured Projects VR/Mixed Reality

SUTU “Nawlz” vs Mixed Reality Camera Tracker Robot

Most of my recent work in VR and Mixed Reality has been rough, proof-of-concept stuff. Nawlz is an attempt to create something with a little more structure. Returning to Australia from a secret overseas Tilt Brush project, SUTU stayed over for a couple of nights and we documented the entire creation of a Tilt Brush artwork.

The piece is a 3D re-imagining of a panel from his interactive comic Nawlz (Season 1, Episode 5, Page 3 for those following along at home).

The video combines a number of mixed-reality techniques I’ve been developing for months: VR timelapse, VR Automatic Camera Tracker in both timelapse and realtime modes, in-app greenscreen painting and some other secret spices.

For more background and progress, check out my recent videos on Vimeo.

Blog VR/Mixed Reality

Mixed Reality Automatic Camera Tracker with SUTU

It’s been a little over a week since I launched my Automatic Camera Tracker project. It’s still a work in progress, but as SUTU is monopolising my Vive and nerd-basement at the moment, it felt like a good time to check in on its progress.

The current version is working alongside other VR apps, and can track a controller or VR headset at realtime or timelapse speeds as the user moves around the VR and IRL space. A little like this:

I wrote some more technical details about the system in a Reddit post on /r/Vive, and the tweet announcing the video received quite a bit of attention…

… which is exciting, because it’s a project I’ve had in my mind since I first used the Vive over a year ago, at PAX Prime in Seattle. As soon as I waved the controller in front of my face and noticed the tracking speed and accuracy I started mentally designing camera tracking systems. I wanted a VR system in my life because I love games, but I needed a Vive for filmmaking science experiments.

I’ve spent today with SUTU filming with my most recent version of the system, using it for both timelapse and realtime, handheld shooting while he worked on a Tilt Brush version of a piece from Nawlz.


Music Video Projects VR/Mixed Reality

Ball Park Music – Whipping Boy: Music Video created in Tilt Brush/Virtual Reality

After completing the Nihilist Party Album video, the Ball Park guys spent a couple of hours trying my Vive VR rig, and they were quickly on board with the idea that their next video should be produced in VR.

3D Printing/Fabrication Electronics/Development Projects VR/Mixed Reality

Exercise Bike Controller

Parts List

Teensy 2.0 Microcontroller – US$19Teensy 2.0 Microcontroller

Teensy 2.0 uses 5v logic, so is easier to use with a rotary encoder that requires 5-20v. (Other Teensy models work, other than the LC which doesn’t support 5v logic.) 
Official Store – PJRC
 – US$19
Australian Stockist – Little Bird – AU$29

Rotary Encoder – $16Encoder 01

6mm shaft rotary encoder, ~360-400 Pulse/Rotation. – US$16
Ebay Australia – AU$15

3D Printed Parts

Three parts are required: Encoder Gear, Crank Gear, and Encoder Bracket. All are available on Thingiverse.

Encoder Gear 3D print preview

Other Parts

  • Exercise Bike (or other rotation-based exercise equipment)
  • Game Controller (or Keyboard/Mouse)
  • Long releasable cable ties/velcro ties/tape to attach bracket to equipment
  • Long USB cable (optional)
  • Long headphone cable (optional)


PC: x360ce (free)
Mac: ControllerMate (US$25)
Teensy Code: Exercise-Bike-Controller on Github


  1. Print all 3 parts
  2. Attach printed Encoder Gear to the encoder shaft
  3. Remove bike casing and the pedal on the opposite side to the drive belt
  4. Screw printed pedal gear on to crank, and then put pedal back on
  5. Connect encoder to Teensy, either solder pins directly or using a breadboard:
    Red Wire (Pulse): Pin D0 (pin 5 on Teensy 2.0 and in Arduino software)
    Green Wire (Pulse): Pin D1 (pin 6 on Teensy 2.0 and in Arduino software)
    White Wire (+5V): Pin VCC
    Black Wire (Ground): Pin GND
    (These wire colours may change depending on your encoder source – 
    so check your part documentation, or do some testing before soldering to the board)
  6. Use Encoder Bracket and cable ties to attach the encoder to bike frame, so the gears mesh together. Cable ties allowed me to adjust the encoder position to ensure the gears weren’t pressed together too hard
  7. Use some more cable ties/velcro to keep cables away from moving parts
  8. Plug Teensy into your computer with long USB cable

Configuring Software

  1. Teensy – First follow the official TeensyDuino tutorials to get set up and ensure Teensy 2.0 is working correctly
  2. Download the Exercise_Bike_Controller arduino sketch from Github and open in Arduino software
  3. Set the Arduino software to use Teensy as a Joystick under menu: Tools > USB Type > Serial + Keyboard + Mouse + Joystick
  4. Upload the sketch to Teensy (Ensure that “Teensy” is selected under menu: Tools > Port), open Serial Monitor and test that the encoder values are changing when the crank moves
  5. Test your maximum pedalling speed outputs a value of +512, increase or decrease the “speedMultiplier” variable (line 17) if you’d like to pedal faster or slower to output full speed on your controller
  6. Once you’re happy with your pedalling speed, change the “readDelay” variable (line 20) to 5, to make the controller latency faster

PC – x360ce

Connect Teensy and Xbox Controller (or other controller). Copy x360ce into game location (for Rocket League this is: /Program Files (x86)/Steam/steamapps/common/rocketleague/Binaries/Win32/)
When playing without exercise controller, you need to temporarily remove the xinput1_3.dll file, or the game won’t start correctly.


Controller 1: Xbox
Triggers Off (for throttle. remove trigger axis)
Pass through: Off
Combine this controller: Combine into: One
Controller 2: Teensy
IHAxis 1, HAxis 1 – added to triggers or other controller.
Combine this controller: Combine into: One
Options: Allow Combining

Mac – ControllerMate

ControllerMate script is available on Github. The “Bike Controller Axes” page should contain “Wireless 360 Controller” and “Teensy Keyboard/Mouse/Joystick.” If you’re using a different controller, you may need to add the axes yourself.
The “Bike Controller Combined” page should create a virtual controller.

Start your game, and enjoy the exercise!


Video Transcript

Hey there Internet. I’m Jaymis, and I have a project I’ve been working on, that I’d like to share with you.

It’s a game controller that attaches to an exercise bike. It takes your calories, burns them alive, and outputs joystick movements or button presses.
I wanted to create a system that could be used with many different games, would produce forward and backward, analogue-style controls, and which could utilise whatever rotation-based exercise equipment players already own.
My solution uses a rotary encoder connected to a Teensy USB development board, with a 3D printed gear and mounting system that can be customised to different equipment geometries. The circuit requires only basic soldering and electronics skills, and it uses very simple, customisable code. Everything can be put together for around $50 if you have access to a 3D printer, or a bit more if you need to order printed parts.
To integrate with an existing game controller requires 3rd party software which works on PC or Mac. This allows you to combine input from a standard game controller with the bike controller input.
I’ve been super addicted to Rocket League the past year, so I’ve mostly used it to control flying cars, however it works well for games like DayZ, Minecraft, and just about anything that involves plenty of walking or running.
You do lose a bit of in-game dexterity though, as real humans can’t change between forward and backward directions as quickly as in-game characters and vehicles, so if you’re playing online and are concerned about your rankings, it will probably make your scores a little sad.
If you feel that you’d like to convert some of your gaming time into exercise time though, it really is a great solution. I now regularly ride for hour-long sessions without getting bored of exercising, which seems like a Good Thing.
So I’m releasing all of the designs and instructions, to help other nerds to cultivate their own Good Habits.
Good luck. Have fun! (GLHF!), and if you’re interested in future updates or other projects involving gaming, performance, video, art, electronics, music, cooking, and other renaissance nerd topics, please do subscribe, or come find me on the internets.