Input and Interaction

Magic Leap, 2015 - 2017

Greg Tran (left) and Frank Hamilton (Right) in 2016 prototyping with our first functional, wearable mixed reality device.

As a senior designer at Magic Leap, I led a team of 5 senior interaction engineers to deliver the first set of specs and front-end algorithms for input and interaction on Magic Leap's OS. 


As a team, we embedded ourselves cross-functionally across hardware, software, and systems teams to prototype and deliver interactions and to facilitate end-to-end integration in the OS.

I founded the UX team's user testing program for validating interactions in the OS, authored the Input and Interaction section of Magic Leap's Human Interface Guideline, and contributed to 20+ patent applications and grants.

Controller

The Magic Leap 1 controller is an EM-tracking-based 6DOF controller, with a 40mm force-sensitive touchpad, 3 buttons, 2 haptics, and signature LED feedback. Along with an incredible team I made significant contributions to the design of the controller: determining key hardware features; designing how these affordances mapped to foundational interactions, like select, grab, and home; leading an external study to refine the form factor; presenting to senior leadership the case to include a 6DOF controller "in the box"; and leading the development and delivery of interaction algorithms for 6DOF manipulation.

From concept to execution, I led a design and prototyping team to develop an innovative navigation experience for the Magic Leap 1 core OS and generate significant IP:

The Eclipse cursor back-illuminates controls and gracefully integrates with dimensional UI

Force-haptic interactions (push-pop) surface contextual controls from dimensional interfaces, and accelerate directional controls.

Embodied Interactions

I led a team to design and deliver embodied interactions for the platform: gesture, voice, head targeting, and eye gaze targeting. We developed interaction concepts inspired by a broad range of influences in sci-fi and HCI, and grounded designs and prototypes through user testing and close collaboration with engineering.  I represented the voice of the customer with engineering and systems teams to help define key performance targets for these emerging technologies.

Hardware UX

With my team I designed, prototyped, and delivered a system for signature light feedback across the controller and power pack, to indicate power, charging, update feedback. 

 

Product Definition 

In 2016, Magic Leap didn't have a product organization. Together with James Powderly I co-led a cross-functional team through the company's first major product-definition milestone Feb 2016, finalizing hardware features for the Magic Leap 1. We owned a document of team impact for each potential feature to drive in/out decisions.

User Testing

I collaborated with Human Factors to lead Magic Leap's first and only qualitative and quantitative user test on external participants. We evaluated two controller designs for user ergonomics and performance across a variety of targeting, selection, and manipulation tasks. I supported data analysis and final recommendations on the preferred form factor.

Design Principles

I championed 6 design principles in my team's work: reliability, performance, ergonomics, premium details, signature interactions, and social conscientiousness. 

We applied these values at every level of the design process— from the global design ethos to the details of execution and technical negotiation. 

Cross-functional Collaboration

I drove the development of the first product-equivalent controller prototype, the P0B, and rallied a technical team to integrate the device into our prototype wearable device at the time. This enabled my team to design and prototype for system interactions and algorithms alongside platform bring-up. 

Impact

Although not all of our original design made it to or remain in the product, our work has had lasting impact on platform bring-up and interaction design innovation, and laid the groundwork for some incredible experiences ultimately launched on the platform—


Key Contributions

Design and Innovation - Researched, prototyped, designed multimodal interactions for 3D interfaces. 20+ patent applications and grants.

Usability Research - Founded the UX team's user testing program and led continuous hallway testing as well as formal usability studies.

Execution - Led a team of 5 senior interaction engineers to prototype and deliver interactions and to facilitate end-to-end integration in the OS. Delivered the first set of specifications and front-end algorithms for input and interaction on Magic Leap's operating system, including the controller, gesture, voice, peripheral inputs, and lightpack hardware interactions.

Collaboration and Leadership - Drove cross-functional alignment on hardware and input features for the Magic Leap 1


Dream Team 

I had the honor to partner with a singularly gifted, ride-or-die team on this work: James Powderly, Lorena Pazmino, Rod Cano, John Day, Tim Stutts, FangYu Yang, Paul Hoover, Dan Novy, Hisham Bedri, Michael Woods, Sam Miller, Paul Lacey, Andrea Montoya.

Using Format