Simply Piano XR

Spatial piano learning app for Android XR

I partnered with Simply to bring the popular piano learning app to Android XR. The project included both design and implementation on the newly released Android XR operating system from Google.

Illustration of Simply Piano XR
Simply Piano lets you see virtual indications on top of the real piano

The app is a unique use case for mixed reality technology, allowing users to see virtual indications and hints on top of a real piano.

Learn more about the project here:

Simply Piano XR ↗

Virtual Piano

The app features a virtual piano that allow the user to learn without a real piano. Using hand tracking information from the device, we can approximate the

Note Rendering

In order to render musical notation that is crisp and clear from any angle and distance, I used signed distance fields that appear sharp and atialiased at any distance

Note rendering
Musical notation is rendered using SDFs for optimal readability in XR

Virtual Keyboard For moments away from a real piano, the app includes a fully interactive virtual keyboard. Building a keyboard that feels responsive and spatially grounded required close attention to placement, scale, and interaction feedback.

We took advantage of the spatial format to add spatial UI effects.