Designing for show-worthy timing

Envisioning a future where augmented reality (AR) effects are the gold standard of live events, StageAR enables future designers to choreograph these effects. I was responsible for the end-to-end design of StageAR, from user research to final visual design.

My Role
  • UI/UX Design
  • Visual Design
  • User Research
  • Graphical Interfaces
  • Fall 2017
StageAR on a laptop during a concert
With StageAR, designers can choreograph and playback AR effects during live events.

Turning concert videos into an interactive experience

If you’ve ever been to a live concert, you’d know that people love to take photos and videos during the show. Especially the tall guy standing in front of you. StageAR allows designers to choreograph AR effects perfectly timed to the concert, so that video Tall Guy took during the show is an immersive AR experience, rather than just a noisy blur.

Phone recording concert footage
With StageAR, designers can turn regular filming into an opportunity for interaction with augmented reality.

How do you design for a user that doesn’t exist yet?

You won’t find the job title “AR Concert Effects Designer” on LinkedIn, at least not yet. Knowing I couldn’t do direct user research, I instead looked at users in adjacent fields, like concert lighting. From that, I was able to build an understanding of who the StageAR user is and what they would demand from the software.

Timing is everything. I identified 3 key types of timed effects that would enable users to prepare for anything: timestamps, prompts, and presets.

Timestamps for down to the second precision

For the known knowns, designers can choreograph effects to specific timestamps from an imported song file. Timestamped effects leverage the timeline interface pattern common in video editing and animation software that users would likely already be familiar with. Tempo controls allow designers to adjust timing as needed, like for a slowed down version of a song during the live show.

StageAR screen for adding an effect to the timeline
Using an imported song as a base timeline, designers can time effects to specific moments in the song.

Prompts for the “sometime around here” moments

When it comes to those effects without a specific timestamp, designers can set an estimated timeframe for a prompted effect. Using the prompt control center, users can activate prompted effects during the live performance at just the right moment.

Creating a prompt with an estimated timeframe
Connect prompted effects with events in the timeline interface.

Presets for live reactions

Even with all the choreography and planning in the world, anything can happen during a live performance. Preset effects allow designers to create effects for the unknown unknowns, whether that’s an unexpected technical mishap or spontaneous audience interaction with a particularly lively crowd.

Preset panel in StageAR
Designers can prepare for the unexpected with presets that can be triggered with keyboard shortcut.

Dark mode for playback

In addition to choreographing the effects, users needed to also oversee playback of their effects. I created a new Live workspace with a dark color scheme that allows users to easily coordinate their effects during a show from a dark effects booth. I also adjusted the default layout of the Live workspace to better reflect the goals of the user, swapping out the secondary stage view for a prompt control panel.

Playback mode in StageAR
Live mode replaces the secondary stage view with a prompt control panel and uses a dark mode color scheme.

Live adjustments on the fly

While the focus of Live mode is primarily to control playback of the choreographed effects, I included minor editing capabilites that allow the user to made quick adjustments during the show. For example, live camera input makes it easy for the designer to determine if they will have to make adjustments based on current lighting conditions.

StageAR in use at a concert
Live lookahead combined with camera input allow designers to make quick adjustments during playback.

The future of StageAR

This version of StageAR focuses primarily on choreographing AR effects: managing their positions in relation to the stage based on three types of timing. If AR effects really are the future of live events, it would be worth expanding the functionality to include more advanced effect creation as an all-in-one solution.

Next Project

Bandcamp Search