THE RESULTS
A great collaboration between myself, Deadmau5, Derivative, and PUBG Corp., and Coal Car Studio we presented the platform to RocNation and Tidal in anticipation of investment and further development.
The app was approved for distribution on The Epic Games Store and is currently being further developed and teased.
THE PROCESS
In light of the threat caused by COVID-19, we got to work prototyping various features that linked his physical stage and home studio to a virtual, multi-user simulation. During this process we had a couple of goals:
1. Link the output of his touring Touchdesigner and Ableton Live projects in real-time to Unreal Engine.
2. Be able to replay his sets with sample accurate timing and scrubbing through a timeline after a performance for a “replay”.
Ultimately, we accomplished both goals. We were able to.integrate Microsoft’s Mixer streaming solution, as well as ingest HAPS-encoded video files inside of a virtual area environment (provided by the folks at Bluehole/PUBG Corp.). We built an initial prototype system in Unreal to ingest cvar files from TouchDesigner and audio encoded by Ableton Live to allow full replay and scrubbing after the performance.
We have since added additional functionality including BPM-synced dancing for simulated crowds, VR and FPS controls, as well as AR features to view the stage and performance in the user’s living room through both a mobile phone and Magic Leap headset.
CREDITS
CLIENT:
DEADMAU5
DIRECTOR:
ANDRE ELIJAH
PRODUCER:
JOEL ZIMMERMAN
DEVELOPMENT:
ANDRE ELIJAH + GEO BARJOUD
3D ART + ANIMATION:
JOEL ZIMMERMAN