Spacetop
Augmented reality laptop OS
Spacetop is a spatial operating system, designed to work with a keyboard, mouse and augmented reality glasses instead of a physical display. Spacetop has a minimal learning curve and is intuitive to first-time users because it builds upon traditional desktop interaction paradigms.
Spacetop's unique approach to spatial computing resonated beyond the XR community, earning Best of Show at CES 2024, TIME Best Invention of 2023, and recognition from Fast Company as one of the Top 10 Most Innovative CE Companies of 2024.
Translating the traditional desktop user interface to a 3D environment provided many challenges, both in design and implementation. My work involved prototyping novel spatial interactions, defining and developing frameworks to manage UI in 3D space, creating special shader effects and more.
I spent five years at Sightful building Spacetop — first as a UX engineer, then as UI architect. The role spanned design and engineering: prototyping novel spatial interactions, building the app frameworks and internal libraries the team shipped on. I continue to work with the team as a consultant.
UI Spaces
Giving the desktop a third dimension opens up exciting opportunities, but also a class of problems that don't exist in 2D: panels can obscure content, overlap and intersect, causing legibility issues and user confusion.
To deal with this, I came up with the concept of UI Spaces: a set of 3D surfaces upon which UI can be placed. Each space has a set of properties - distance from the user, curvature, and following behavior - that is designed to convey the importance of its content to the user.
The spaces are nested, and the closer a space sits to the user, the more insistent it is allowed to be.
This system gave Spacetop UI a predictable structure and allowed designers and developers to reason about the 3D space surrounding the user, placing interface components according to their importance and context.
The Canvas
The Canvas is a huge virtual "screen" that curves around the user. Instead of being limited by the edges of a physical screen, the virtual canvas can expand to fit many windows. To allow the users to understand the 3D shape and position of the canvas surface, we used a gentle grid of dots that is revealed around the cursor and content.
The canvas itself can be panned and manipulated using touchpad gestures or keyboard shortcuts.
In some cases, the limited field of view still required some creative solutions - such a the compass component that guides your attention when it is needed somewhere you can't see.
Cross-platform design system
Spacetop's interface is a mix of Unity UI, native UI and embedded web-based apps, and they needed to look and behave coherently. I created a react-based design system that spans both "platforms" and provides a shared set of components, styles and behaviors.
a shared token and spec layer defining color, type, spacing, and motion a React component library for the web-based apps an experimental React-to-Unity renderer, built on open-source work, that drives Unity UI from React components a custom CSS-compatible shader for Unity that renders the same rounded rects, borders, and shadows the web components use — later extracted as UIRect
Materials & Rendering
Rendering spatial interfaces opens up interesting possibilities and effects. Subtle highlights can react to the movement of the head to give the illusion of depth. Soft shadows can indicate distance between objects.
To remain performant, all UI rendering is based on quads, however in the fragment shader we can create special effects. In XR we know the position of the head. Using Parallax mapping and normal mapping a 3D illusion can be achieved.