SUTU “Nawlz” vs Mixed Reality Camera Tracker Robot

Most of my recent work in VR and Mixed Reality has been rough, proof-of-concept stuff. Nawlz is an attempt to create something with a little more structure. Returning to Australia from a secret overseas Tilt Brush project, SUTU stayed over for a couple of nights and we documented the entire creation of a Tilt Brush artwork.

The piece is a 3D re-imagining of a panel from his interactive comic Nawlz (Season 1, Episode 5, Page 3 for those following along at home).

The video combines a number of mixed-reality techniques I’ve been developing for months: VR timelapse, VR Automatic Camera Tracker in both timelapse and realtime modes, in-app greenscreen painting and some other secret spices.

For more background and progress, check out my recent videos on Vimeo.

Jaymis on Triple J Hack: Music Video VR

Australia is lucky to have one of the greatest youth-oriented national broadcasters on the planet – Triple J. Their weekly current affairs show Hack covers a huge variety of topics, and an excellent example of this is my interview about working on VR and music performance being introduced by 10 minutes of Tinder Date Murder Trial coverage. [Skip to 11:00 to avoid murders]

Mixed Reality Automatic Camera Tracker with SUTU

It’s been a little over a week since I launched my Automatic Camera Tracker project. It’s still a work in progress, but as SUTU is monopolising my Vive and nerd-basement at the moment, it felt like a good time to check in on its progress.

The current version is working alongside other VR apps, and can track a controller or VR headset at realtime or timelapse speeds as the user moves around the VR and IRL space. A little like this:

I wrote some more technical details about the system in a Reddit post on /r/Vive, and the tweet announcing the video received quite a bit of attention…

… which is exciting, because it’s a project I’ve had in my mind since I first used the Vive over a year ago, at PAX Prime in Seattle. As soon as I waved the controller in front of my face and noticed the tracking speed and accuracy I started mentally designing camera tracking systems. I wanted a VR system in my life because I love games, but I needed a Vive for filmmaking science experiments.

I’ve spent today with SUTU filming with my most recent version of the system, using it for both timelapse and realtime, handheld shooting while he worked on a Tilt Brush version of a piece from Nawlz.

 

Tilt Brush Backyard – Mixed-Reality VR Painting in Nature with HTC Vive

I’ve been thinking for a while about taking the Vive outside to test how it handles Nature. The object-tracing capabilities of Tilt Brush have already been pretty well established. SUTU and I had planned to take the VR rig outside last week, but it ended up raining, so I had to do it by myself after he left :|

To view the art in Tilt Brush yourself, and spend a little time in my virtual backyard download the .TILT file here.

The process was great fun! Outside Vive is a super strange feeling, but it worked well. Even climbing a ladder while blind was quite fun. (Note: Don’t take your Vive outside at daytime, tracking won’t work and sunlight through the lenses can burn the screens.)

I took a couple of precautions, most important was applying some tape to the controller so it didn’t get scratched up by the brutal nature in my suburban backyard. I figured out last week that the most accurate way to “trace” objects in Tilt Brush is to use the back of the controller, as the “stylus point” is at the top of the handle, not the tip of the tracking ring.

Taping up the Vive controller before dragging it across Nature was a good idea.

A photo posted by Jaymis Loveday (@jaymis) on

I’m still experimenting with mixed-reality motion-control shots. This version I stabilised the Tilt Brush feed in After Effects as it had a constant slow oscillation which I think may be to due to vibrations from my robot’s stepper motors.

 

 

Tilt Brush Pendulum Test 01

While investigating tracking precision for motion control timelapse filming in VR I had a thought, and then sidetracked myself for half a day designing various Tilt Brush/Vive pendulum rigs. The first one is very simple, but worked beautifully:

@tiltbrush Pendulum Test 01.

A photo posted by Jaymis Loveday (@jaymis) on

TiltBrushPendulum01

If you have Tilt Brush, I’ve made the sketch available for your delectation [Dropbox link – 1.94MB]. It’s a lovely place to spend some time.

Ball Park Music – Whipping Boy: Music Video created in Tilt Brush/Virtual Reality

After completing the Nihilist Party Album video, the Ball Park guys spent a couple of hours trying my Vive VR rig, and they were quickly on board with the idea that their next video should be produced in VR. Continue reading Ball Park Music – Whipping Boy: Music Video created in Tilt Brush/Virtual Reality