Input and Interaction
Magic Leap, 2015 - 2017
As a senior designer at Magic Leap, I led a team of 5 senior interaction engineers to deliver the first set of specs and front-end algorithms for input and interaction on Magic Leap's OS.
As a team, we embedded ourselves cross-functionally across hardware, software, and systems teams to prototype and deliver interactions and to facilitate end-to-end integration in the OS.
I founded the UX team's user testing program for validating interactions in the OS, authored the Input and Interaction section of Magic Leap's Human Interface Guideline, and contributed to 20+ patent applications and grants.
The Magic Leap 1 controller is an EM-tracking-based 6DOF controller, with a 40mm force-sensitive touchpad, 3 buttons, 2 haptics, and signature LED feedback. Along with an incredible team I made significant contributions to the design of the controller: determining key hardware features; designing how these affordances mapped to foundational interactions, like select, grab, and home; leading an external study to refine the form factor; presenting to senior leadership the case to include a 6DOF controller "in the box"; and leading the development and delivery of interaction algorithms for 6DOF manipulation.
I championed the immense potential of rich touch interactions for spatial computing interfaces. I led a design and prototyping team to develop an innovative touch experience and generate significant IP:
The Eclipse cursor back-illuminates controls and gracefully integrates with dimensional UI, enabling custom hover states and fluid transitions to non-touch input like gesture and 6DOF.
Force-haptic interactions (push-pop) surface contextual controls from dimensional interfaces, and accelerate directional controls.
I led a team to design and deliver embodied interactions for the platform: gesture, voice, headpose, and eye gaze. We developed interaction concepts inspired by a broad range of influences in sci-fi and HCI, and grounded designs and prototypes through user testing and close collaboration with engineering. I represented the voice of the customer with engineering and systems teams to help define key performance targets for these emerging technologies.
With my team I designed, prototyped, and delivered a system for signature light feedback across the controller and power pack, to indicate power, charging, update feedback.
In 2016, Magic Leap didn't have a product organization. Together with James Powderly I co-led a cross-functional team through the company's first major product-definition milestone Feb 2016, finalizing hardware features for the Magic Leap 1. We owned a document of team impact for each potential feature to drive in/out decisions.
I collaborated with Human Factors to lead Magic Leap's first and only qualitative and quantitative user test on external participants. We evaluated two controller designs for user ergonomics and performance across a variety of targeting, selection, and manipulation tasks. I supported data analysis and final recommendations on the preferred form factor.
I championed 6 design principles in my team's work: reliability, performance, ergonomics, premium details, signature interactions, and social conscientiousness.
We applied these values at every level of the design process— from the global design ethos to the details of execution and technical negotiation.
I drove the development of the first product-equivalent controller prototype, the P0B, and rallied a technical team to integrate the device into our prototype wearable device at the time. This enabled my team to design and prototype for system interactions and algorithms alongside platform bring-up.
Although not all of our original design made it to or remain in the product, our work has had lasting impact on platform bring-up and interaction design innovation, and laid the groundwork for some incredible experiences ultimately launched on the platform—
Design and Innovation - Researched, prototyped, designed multimodal interactions for 3D interfaces. 20+ patent applications and grants.
Usability Research - Founded the UX team's user testing program and led "dogfooding" and formal usability studies.
Execution - Led a team of 5 senior interaction engineers to prototype and deliver interactions and to facilitate end-to-end integration in the OS. Delivered the first set of specifications and front-end algorithms for input and interaction on Magic Leap's operating system, including the controller, gesture, voice, peripheral inputs, and lightpack hardware interactions.
Collaboration and Leadership - Drove cross-functional alignment on hardware and input features for the Magic Leap 1
I had the honor to partner with a singularly gifted, ride-or-die team on this work: James Powderly, Rod Cano, John Day, Tim Stutts, FangYu Yang, Paul Hoover, Dan Novy, Hisham Bedri, Michael Woods, Sam Miller, Paul Lacey.