Input and Interaction

Magic Leap, 2015 - 2017

Greg Tran (left) and Frank Hamilton (Right) in 2016 prototyping with our first functional, wearable mixed reality device.

As a senior designer at Magic Leap, I led a team of 5 senior interaction engineers to deliver the first set of specs and front-end algorithms for input and interaction on Magic Leap's OS. 


As a team, we embedded ourselves cross-functionally across hardware, software, and systems teams to prototype and deliver interactions and to facilitate end-to-end integration in the OS.

I founded the UX team's user testing program for validating interactions in the OS, authored the Input and Interaction section of Magic Leap's Human Interface Guideline, and contributed to 20+ patent applications and grants.

Controller

The Magic Leap 1 controller is an EM-tracking-based 6DOF controller, with a 40mm force-sensitive touchpad, 3 buttons, 2 haptics, and signature LED feedback. Along with an incredible team I made significant contributions to the design of the controller: determining key hardware features; designing how these affordances mapped to foundational interactions, like select, grab, and home; leading an external study to refine the form factor; presenting to senior leadership the case to include a 6DOF controller "in the box"; and leading the development and delivery of interaction algorithms for 6DOF manipulation.

Touch

I championed the immense potential of rich touch interactions for spatial computing interfaces. I led a design and prototyping team to develop an innovative touch experience and generate significant IP:

The Eclipse cursor back-illuminates controls and gracefully integrates with dimensional UI, enabling custom hover states and fluid transitions to non-touch input like gesture and 6DOF.

Force-haptic interactions (push-pop) surface contextual controls from dimensional interfaces, and accelerate directional controls.

Embodied Interactions

I led a team to design and deliver embodied interactions for the platform: gesture, voice, headpose, and eye gaze. We applied strategic and empathetic design to deliver natural human-computer interactions, and avoided a fatiguing, socially unconscientious interaction model.  I represented the voice of the customer with engineering and systems teams to help define key performance targets for these emerging technologies.

Hardware UX

With my team I designed, prototyped, and delivered a system for signature light feedback across the controller and power pack, to indicate power, charging, update feedback. 

 

Product Definition 

In 2016, Magic Leap didn't have a product organization. Together with James Powderly I co-led a cross-functional team through the company's first major product-definition milestone, Hardware Lock in Feb 2016. We owned a matrix of team impact for each feature in consideration to drive in/out decisions.

User Testing

I collaborated with Human Factors to lead Magic Leap's first and only qualitative and quantitative user test on outside subjects. We evaluated two controller designs for user ergonomics and performance across a variety of targeting, selection, and manipulation tasks. I supported data analysis and final recommendations on the preferred form factor.

Design Principles

I championed 6 design principles in my team's work: reliability, performance, ergonomics, premium details, signature interactions, and social conscientiousness. 

We applied these values at every level of the design process— from the global design ethos to the details of execution and technical negotiation. 

Cross-functional Collaboration

I drove the development of the first product-equivalent controller prototype, the P0B, and rallied a technical team to functionally integrate the device into our prototype wearable device at the time. This unblocked design and prototyping for system interactions and algorithms. 

Impact

Although not all of our original design made it to or remain in the product, our work has had lasting impact on platform bring-up and interaction design innovation, and laid the groundwork for some incredible experiences ultimately launched on the platform—


Key Contributions

Design and Innovation - Researched, prototyped, designed multimodal interactions for 3D interfaces. 20+ patent applications and grants.

Usability Research - Founded the UX team's user testing program and led "dogfooding" and formal usability studies.

Execution - Led a team of 5 senior interaction engineers to prototype and deliver interactions and to facilitate end-to-end integration in the OS. Delivered the first set of specifications and front-end algorithms for input and interaction on Magic Leap's operating system, including the controller, gesture, voice, peripheral inputs, and lightpack hardware interactions.

Collaboration and Leadership - Drove cross-functional alignment on hardware and input features for the Magic Leap 1


Dream Team 

I had the honor to partner with a singularly gifted, ride-or-die team on this work: James Powderly, Rod Cano, John Day, Tim Stutts, FangYu Yang, Paul Hoover, Dan Novy, Hisham Bedri, Michael Woods, Sam Miller, Paul Lacey.

Using Format