Jump by Magic Leap
2019
Internal Demo of Jump, filmed via Ncam
Jump by Magic Leap is the world’s first volumetric telepresence solution. It enables remote users to communicate together as 3D holograms and collaborate with dynamic spatial tools.
Context - In 2019, Magic Leap acquired Mimesys, a European-based volumetric copresence startup. I was assigned as the product lead for the commercialization and launch of their existing proof-of-concept on an aggressive 9 month timeline.
Challenge - Current remote collaboration solutions have us stuck behind tiny rectangles, posing challenges to effective collaboration, content review, and felt sense of presence. Distributed teams need better tools to feel like they’re in the same room.
The COVID-19 global crisis made this need universal and urgent: Reality is turning into sci-fi while sci-fi is turning into reality. We have a unique opportunity to reimagine how we collaborate.
Solution - Jump by Magic Leap is an integrated hardware-software system that solves these pain points by enabling real- time volumetric video streaming and collaboration to bring presence and interactivity back into virtual meetings.
Magic Leap Gate
Magic Leap Gate
The Jump system includes the Magic Leap Gate, a custom sensor rig powered by two depth cameras connected to a GPU, that pairs with a Magic Leap 1 to capture and stream real-time volumetric video of the participant to a remote user.
I led a cross-functional team to determine the product requirements for and deliver this hardware accessory. I championed the development of custom housing to achieve a sleek, on-brand form-factor, in order to position this product as a premium enterprise experience.
Key Features
1:1 Volumetric copresence
Stream in real-time volumetric video (“3D holograms”) of another remote user
Import of 2D content + select 3D content
Easily import existing 2D files + work with Magic Leap to import select compatible 3D content for collaboration and sharing through a simple web portal
Manipulation of 2D + 3D content
Collaborate in real-time by sharing and annotating 2D and 3D digital content
Whiteboard capability
Transform any 2D surface into a whiteboard
Volumetric filters
Enhance volumetric videos of remote users via 3D filters
Jump's main menu for selecting imported 2D/3D content and collaboration tools.
User Interface
I brought on a team of UX designers to the project and prioritized a redesign of the experience to improve usability and align the visual design of the experience with our brand and OS. I led usability playtests of the experience with 20 enterprise customers across verticals in their offices across Miami, Orlando, and New York.
In our design process, we explored creative ways to improve the perceived quality of holograms with visual effects. This lead us to develop a range of interesting and playful "filters" for holograms. The images below were captured on device.
Volumetric filters captured from device
Process and Thinking
Different copresence technologies have different tradeoffs. I applied deep technical and market analysis to develop a business case and roadmap for copresence platforms on Magic Leap.
I championed a software and tracking-based approach for photorealistic copresence (versus the high cost / bandwidth requirements of pure volumetric). I worked with our accessories team to design our external keyboard to support phone-based copresence solutions.
Sample of a go-no-go decision tree to determine when and at what quantity to scale HW production of the Jump system based on customer feedback
Key Contributions
Leadership - Led the integration of the 6-person Mimesys team into the broader organization. Rallied the talent and capital needed to bring their existing proof-of-concept to a commercial Beta launch on an aggressive 9 month timeline.
Product - Defined detailed requirements for the hardware, software, and cloud components of the Jump system.
Strategy - Developed and owned the long-term strategy and roadmap for photorealistic copresence on the platform.
Creative - Contributed design concepts and creative direction to the design of the hardware and user interface.
Dream Team
I had the honor to partner with a singularly gifted, ride-or-die team on this work:
Key Collaborator - Remi Rousseau
Design - Jarrett Webb, Kyle Contigo, Haney Awad
Engineering - Davy Loots and the Mimesys team in Belgium
Program - Amelia Waters, Gary Nguyen