Categories
Blog

Jaymis on Triple J Hack: Music Video VR

Australia is lucky to have one of the greatest youth-oriented national broadcasters on the planet – Triple J. Their weekly current affairs show Hack covers a huge variety of topics, and an excellent example of this is my interview about working on VR and music performance being introduced by 10 minutes of Tinder Date Murder Trial coverage. [Skip to 11:25 to avoid murders]


Listen: music video and VR on Triple J Hack

Broadcast 18 October 2016. Reporter Stephen Stockwell visited my place to experience Ball Park Music’s Whipping Boy VR video, then we talked about Tilt Brush, Reggie Watts playing AltspaceVR, and what virtual gigs might mean for Australian bands.

In brief: Sam Cromack painted the clip in Tilt Brush; I directed and built the technical rig; the conversation moves from turning a room-scale work into a 2D music video to performing live inside VR.

“If you can put on a show that people from all around the world can come to—without having to physically move around a lot—that’s a pretty magical thing.”

Jaymis Loveday, Triple J Hack, 18 October 2016

Archive update, 2026: This 3:47 Jay-focused excerpt uses 11:25–14:40, 16:22–16:48 and 17:03–17:10 of the preserved 29:45 broadcast. The intervening interview with Lincoln Savage and brief headset-removal chatter are omitted; the full source recording remains preserved offline. Audio: Triple J Hack / Australian Broadcasting Corporation.

Transcript

Speaker-labelled transcript generated from the preserved broadcast with Whisper (medium), checked against the source timings, and lightly edited for names, punctuation and readability. The audio remains the authority.

TOM TILLEY: Now, when you think about virtual reality, you probably think about it in porn or the gaming context. And in just a moment, we’ll take you inside a virtual reality game so real that some players even vomit. But what about music? Some virtual reality developers say we could be going to virtual reality gigs within months. There’s already some serious movement in the virtual reality video clip space. Stephen Stockwell went inside the first video clip ever created in virtual reality to find out what’s in store.

STEPHEN STOCKWELL: I’ve never been in virtual reality before, so I’m trying to figure out how I actually describe this place. I’ve never had to sit in a galaxy looking at a volcano erupting from a man’s chest through a spiral rainbow. That’s all part of a film clip. This is a piece of art painted inside virtual reality—which is the music video for Ball Park Music’s Whipping Boy. The world I’m standing in was drawn by Sam from Ball Park Music. And Jaymis, whose house we’re sitting in right now, was the director and provided the technical gear to make it all happen.

JAYMIS LOVEDAY: The best way to experience this is to own a VR rig and walk around inside it with the song playing. But most people can’t do that, so we sort of tried to figure out how we could show it to people on a 2D screen and have it still be exciting.

STOCKWELL: It’s really hard to describe, but the light painting that Sam has created in virtual reality is really immersive. And you put on the headset and you put on the headphones and it really feels like you’re in another world. Now, there are still limitations with the cost of virtual reality. The headset I’ve been wearing is about $1,500. And then you need a pretty pricey computer to run all the software—probably a couple of grand there as well. I don’t want to tell you that it’s worth it, but I did have an incredible time in virtual reality.

JAYMIS: VR itself—it is actual virtual reality. That’s the weird thing. People will not walk off a cliff in VR because that’s scary. And being inside these psychedelic scrawlings of Sam’s, you get in the mindset of someone who’s sitting inside this crazy piece of artwork floating in a galaxy. While you’re in there, it’s entirely convincing.

STOCKWELL: While the Whipping Boy video was the first film clip made in virtual reality, it’s really just the first step.

JAYMIS: From here, we’re going to start seeing a lot more of this kind of stuff—interactive music experiences, and even virtual reality gigs. Altspace put on a show with the comedian and musician Reggie Watts a couple of months ago. They basically put him in a full motion-capture suit, built him a custom avatar that looks like Reggie—with the amazing afro and beautiful kind of suspenders that he always tends to wear—and he put on a live comedy show in a virtual comedy club. But that’s comedy, not music. There are some electronic gigs that are happening in virtual reality already, but there’s a bit more technical work that needs to be done before you can get really sweaty to Violent Soho in your living room. Electronic music is much easier because it’s mostly about hand movements, which are already tracked. To get live rock bands with people playing guitars and things like that is going to take a little bit more time because generally there are more people. So there are a little bit more technical hurdles to get—for instance, a Ball Park Music set live in VR—but it’s not out of reach at the moment.

STOCKWELL: Jaymis isn’t the only person doing this kind of thing. He’s not even the only person in Brisbane working in this space.

[The broadcast’s separate interview with Lincoln Savage is omitted from this Jay-focused excerpt.]

STOCKWELL: Being able to go to a variety of live performances in VR is something that’s going to be happening inside of a year. While the idea of strapping on a VR headset to check out a band might sound pretty futuristic, Jaymis says it’s not far off at all.

JAYMIS: Touring for a band in Australia is really hard because it’s such a big country with such a small, relatively, number of people. If you can put on a show that people from all around the world can come to—without having to physically move around a lot—that’s a pretty magical thing.

TOM TILLEY: Welcome back, Stocky. That was Jaymis he was speaking to there, who directed the first film clip created in virtual reality.

Contemporary article snapshot: Triple J Music News via the Wayback Machine.

Categories
Blog VR/Mixed Reality

Mixed Reality Automatic Camera Tracker with SUTU

It’s been a little over a week since I launched my Automatic Camera Tracker project. It’s still a work in progress, but as SUTU is monopolising my Vive and nerd-basement at the moment, it felt like a good time to check in on its progress.

The current version is working alongside other VR apps, and can track a controller or VR headset at realtime or timelapse speeds as the user moves around the VR and IRL space. A little like this:

I wrote some more technical details about the system in a Reddit post on /r/Vive, and the tweet announcing the video received quite a bit of attention…

… which is exciting, because it’s a project I’ve had in my mind since I first used the Vive over a year ago, at PAX Prime in Seattle. As soon as I waved the controller in front of my face and noticed the tracking speed and accuracy I started mentally designing camera tracking systems. I wanted a VR system in my life because I love games, but I needed a Vive for filmmaking science experiments.

I’ve spent today with SUTU filming with my most recent version of the system, using it for both timelapse and realtime, handheld shooting while he worked on a Tilt Brush version of a piece from Nawlz.

 

Categories
Blog

Tilt Brush Backyard – Mixed-Reality VR Painting in Nature with HTC Vive

I’ve been thinking for a while about taking the Vive outside to test how it handles Nature. The object-tracing capabilities of Tilt Brush have already been pretty well established. SUTU and I had planned to take the VR rig outside last week, but it ended up raining, so I had to do it by myself after he left :|

To view the art in Tilt Brush yourself, and spend a little time in my virtual backyard download the .TILT file here.

The process was great fun! Outside Vive is a super strange feeling, but it worked well. Even climbing a ladder while blind was quite fun. (Note: Don’t take your Vive outside at daytime, tracking won’t work and sunlight through the lenses can burn the screens.)

I took a couple of precautions, most important was applying some tape to the controller so it didn’t get scratched up by the brutal nature in my suburban backyard. I figured out last week that the most accurate way to “trace” objects in Tilt Brush is to use the back of the controller, as the “stylus point” is at the top of the handle, not the tip of the tracking ring.

I’m still experimenting with mixed-reality motion-control shots. This version I stabilised the Tilt Brush feed in After Effects as it had a constant slow oscillation which I think may be to due to vibrations from my robot’s stepper motors.

 

 

Categories
Blog

Tilt Brush Pendulum Test 01

While investigating tracking precision for motion control timelapse filming in VR I had a thought, and then sidetracked myself for half a day designing various Tilt Brush/Vive pendulum rigs. The first one is very simple, but worked beautifully:

TiltBrushPendulum01

If you have Tilt Brush, I’ve made the sketch available for your delectation [Dropbox link – 1.94MB]. It’s a lovely place to spend some time.

Categories
Blog

Collected Notes from a VR Performance

I just “attended” my first “live” VR performance: Reggie Watts in AltSpaceVR.

(Air quotes used to illustrate how silly it is to demarcate the reality of experiences in VR. I won’t be using them again, VR is real enough to shrug off the quotes.)

(You can check out a recorded version of the show on youtube. Update: The video has been made private. Hopefully it’ll return at some stage, as it’s an interesting piece of archival material.)

Reggie Watts Live in VR

A quick collection of thoughts about live performance in VR

Not So Great Things

A “Mute All Audience Members” button is vital. Surely one of the main advantages of VR shows is that you don’t have to deal with idiots talking over the performance? I found myself having to wander through the crowd to find whoever was the source of obnoxious noises and mute them.

Ideally the VR client should detect if the audience isn’t wearing headphones (show noise is feeding back through their mic) and automatically mute them.

When a performer starts their show, all audience members could have their volume reduced/muffled by the software. Real performances use amplification to allow the performer to (mostly) be louder than audience members. Anonymity combined with a technological ability to be as loud as the person on stage made for a very rowdy show.

VR client software should have a “temporary mute” button physically on your controller. A “cough button”, so you don’t have to locate and toggle the little microphone icon if you need to clear your throat/eat chips.

Audience Interaction doesn’t really work unless all audience members are in the same virtual room as the performer. With over 500 (I heard someone say 800 at peak) users trying to watch the show, they were being split into separate room copies, separate from the  “VIP” room that the performer was seeing. This seemed to cause the audience I was with to feel disconnected, and they started to become unruly when they realized they couldn’t be seen/heard by the performer.

Even the “VIP” room had some strange moments as the audience level was close to that of Reggie’s (embedded video cued to 56m38s. Update: The video has been made private. Hopefully it’ll return at some stage, as it’s an interesting piece of archival material.):

As a super great performer he recovered the moment well, but I think this situation has more potential drawbacks than advantages.

Great Things

A common element of support acts is the “please come to the front of the room” section of the performance. VR audience members seem to have no problem going right up to the stage and dancing. I’m mortified by public dancing, but it’s not as scary in VR.

Muting noisy/annoying audience members is a superpower everyone has wanted at a show. Now it’s a thing you can totally do. Giving this power to the performer as a moderation tool could also be great for their experience of the show. Goodbye hecklers.

Motion Capture + Audio Recording = Ability to play the entire show back later. This makes for some interesting performance possibilities, and was useful today when the servers became overloaded. It seemed that the VR performance was actually played back from recorded motion capture and audio data.

So Is VR One Of The Futures Of Live Performance

Yep, it absolutely can be. It has some huge advantages over video streaming. The feeling of presence is pretty magical. The drawbacks won’t take long to solve, but the advantages are many and will continue to grow as the tech matures.

Categories
Blog

Daniel Kitson touring Australia with “The Interminable Suicide of Gregory Church”

Daniel Kitson is my favourite comedian in the world. From multi-hour long standup, to his beautifully constructed and delivered “story” shows, I will see anything he puts on.

In recent years he’s been doing more story shows. They’re as beautiful, hyper-intelligent and darkly funny as his standup, but also feature sets, lighting, recordings, and a, y’know… Plot.

If you like things which are entirely amazing, you’d be crazy not to go to Kitson’s show, currently touring around Australia and other parts of the world.

Here’s a bit from his most recent email newsletter:

Dear People of Australia

I leave the northern hemisphere tomorrow, hurtling through the sky like a magical balding lunatic to your beautiful and cruel shores. So basically, this is just an email to remind you, that im coming, to prepare yourselves physically and emotionally and to politely suggest that you buy tickets to my show.

Im not being produced by any venues or festivals this time and am touring far more in the manner that I would at home. Which is to say, swaggering about in a self important manner, not really doing any press, minimal advertising and relying on the power of my mailing list to access an enthusiastic and loyal audience who would walk into fire if I suggested it. There are of course, pros and cons to this charmingly low key/repugnantly arrogant behaviour. On the upside, I’m not contractually obligated to engage in interviews that make me want to weep and secondly I can have significantly cheaper tickets than on many previous visits. I’ve been genuinely uncomfortable in the past with some of the ticket prices and my inability to control them as I do at home, so this time I’ve tried to keep them as close to $25 as I can. Which I know is still not super cheap, but it is better than a kick in the nuts.*

On the downside though, ive not really sold that many tickets. Especially in Adelaide where I cleverly combined my lack of press and marketing with a stupidly large room in a tactically astute combination almost purpose built for humiliation. So, if you can think of anyone who may fancy it. Please do let them know.

Feel free to spread the word, basically, in any of the various social media you people frequent. I’d be very grateful, but please be careful to articulate the nature of the show accurately? That would be lovely. It is, for the record, a storytelling show, that’s very funny, a little sad, a bit uplifting and a touch melancholy. It’s not stand up basically. And i wouldnt want people coming thinking it was. Im sure you understand. I’d hate to sell a ticket for an apple when all I have is oranges.**

* – “A kick in the nuts” is a term used by monetary academics to indicate an overpriced ticket to a piece of melancholy, uplifting, and comedic storytelling.

** – This is a proverb. A new one. I just made it up. Spread it. That’s how Aesop started.

I’ll be going to the Monday and Wednesday shows. Rowley is going to all three, because after the last one we said “Next time he comes, we have to go to more than one showing”. There you go.

Buy tickets at the Brisbane Powerhouse site.

Categories
Blog

I’ve looked at this image a dozen times today…

… and every time I do, I laugh:

Categories
Blog

Paranoid Android, played on Ukulele, while sitting on the toilet

Just in case anyone missed Ben Stewart’s gig at Ric’s on the weekend. He uploaded a video of himself playing Paranoid Android, on Ukulele, on The Toilet.

The Awesomeness is strong with this one. Lovely fast intimate production. The shower scene is particularly excellent.

Categories
Blog

New Video Release: Edward Guglielmino – Fail With Me

I’ve just released Fail With Me, the latest of many works I’ll be producing for Edward Guglielmino‘s album Late At Night.

This joins several other videos I’ve produced recently for Ed:
Lion In Your Bed (Live)
Fail With Me (Live)
No Body Is Perfect (Live)
Caught In A Landslide (Live, Bridge Sessions)
Late At Night (Live, Bridge Sessions)

I’ve also posted on Create Digital Motion about the philosophy and techniques behind the production of this video.

Now I’m off to have lunch with Ed and plan for our next escapade.

Categories
Blog

Internet Censorship in Australia, AKA the “Clean Feed” – Do Not Want

For a while now I’ve been keeping an eye on the web censorship antics various Australian governments have been getting up to. The previous, coalition government decided that spending AU$3000 per copy on some web filtering software was a fantastic idea: Dan of Dansdata has a summary of the maths behind that particular exercise in pointlessness.

So the new, Labour government looked into that failure, dubbed it so, and then decided that the reason it failed was that the coalition just hadn’t wasted enough money. That they’d aimed too low. So instead of some kind of pointless, wasteful, opt-in filtering software that kiddies can bypass, now they want to implement country-wide, mandatory internet filtering of “illegal” and “inappropriate” material.

This is wrong and pointless on so many levels, it’s quite difficult to even know where to start if I was to explain to someone why this is a bad idea. Once again, Dan has a summary, and he doesn’t think it’s actually going to happen, which is nice.

But I’m pretty sure it’s not going to happen.

Similar threats have been made here in the past, and they’ve always petered out into nothing. There are no votes to be won in actually filtering the Internet, after all. The people who vote based on Net filtering promises are unable to tell whether it’s actually happening or not. And there are plenty of votes to be lost when everyone who doesn’t call their browser “the Internet” discovers that they can’t get to YouPorn or Mininova any more.
(link)

Just in case they actually think they’re going to go through with it, there are some people starting to activate and campaign against this pointless stupidity. So here’s some resources:

No Clean Feed – The Website, The Mission.
No Clean Feed on Twitter.
No Clean Feed petition
No Clean Feed Facebook group.

How a “Minister for Broadband, Communications and the Digital Economy” can think this would be feasible, or that it would be a good idea even if it was technically possible, is completely beyond me. Why is Australia repeatedly lumped with luddites for our IT ministers?