Hands first onboarding experience

In early 2020, Oculus released its beta hand tracking feature for the Quest. As it coincided with the beginning of my internship at HTC Creative Labs, my manager and I thought it would be fun to explore how we might teach new owners of hand-tracking capable headsets to interface with their machine.

Timeline

1.5 Months

Role

Researcher
Designer
Prototyper

TEAM

Solo Project

Skills

2D & 3D wireframing
Experience design
Unity Scripting
Copywriting

Exploratory Research

Content analysis and secondary research in an emerging field

I kicked off this project by conducting an informal content and competitive analysis into what other hands-first VR projects currently exists, how they are being presented, and how they are onboarding their users.

Youtube, my main desk research resource

Our team decided to focus on people who are newly living with low vision because of their existing familiarity with current technologies such as touch screen phones. Because of the limited time we have to work on this project, we felt that it would be best to design for a use case that is as familiar to our existing understanding of mobile technology.

Desk research findings

Ideation

From blue sky to guiding themes

I began by generating as many ideas as possible through blue sky ideation. To help get the creative juices flowing, I also did some body storming that helped me test out hunches and turn them into ideas

How I went from kitty laser to Guiding Glove idea

I ended up with around 20 different ideas, some clearly more fleshed out than others.

A snippet of some ideas I generated

Through feedback with my mentor/manager, I distilled commonalities from these ideas into themes/values. These became my guiding values.

Downselection

Step by step education

I decided to move forward with a car customization experience that allows the user to customize a car using UI elements and interact with it using gestures. I went with this framing because our target audience was quite broad. The experience had to appeal to individuals, but should be appealing to enterprise customers as well.

A huge part of this process is learning to work with new tools to represent new ideas, which was why I chose to approach this project form a multi-pronged perspective. I chose to move forward with the following two ideas:

1. Education Station

Informal, Fun, High Cost

Education Station is a full VR standalone experience to teach the user about hand based input. In this experience, the user will be in an office where they can design, build, and play with a car that they build.

3D Mockup (the user is the puppet)

2. DIY Board

Formal, Serious, Low Cost

The DIY board is a simple step-by-step and highly constrained experience that teaches the user to interact with UI and world objects on a floating UI Canvas. The user will be asked to perform simple tasks such as selecting and dragging to proceed. This is basically the stripped down version of the Education Station.

1st person POV panel closeup

Design through prototyping

Trying out different ideas and low-fi sketching methods

I decided to use this opportunity to try different sketching methods in both 2D and 3D to get my ideas down on "paper". I used Microsoft Maquette and Tvori for the VR prototyping and Adobe XD and Figma for the 2D.

I think I jumped into VR a little bit too early during this phase because I quickly became overwhelmed by the tools and space, but it did allow me to play with scale as a way to move people through an experience.

However, it was incredibly valuable for me to learn new tools, try out new ideas (even if they flopped), and better understand the quality that each wire framing method provided.

How these methods differ

3D/VR Design tools
Positive
Helps me get a sense of the scale I was working with and the "feel" of the experience. It also allowed me to experiment with different layouts.

Negative
Too slow for early stage design and creating custom assets (or even typing) is frustrating and difficult. It's also hard to show people your work, especially when you're remote.

2D Design tools
Positive

Super quick and easy to work in. Easy to access online stock assets or create your own. Can be used to mock up several scenes to create a storyboard.

Negative
Difficult to get a sense of how the experience may feel in VR. It's also harder to design spatially important events.

Iterations

Getting Feedback and Refining

Despite building a VR experience in isolation, I was able to use my wide variety of designs to communicate my ideas with the rest of the Creative Labs team and get some feedback.

I set up meetings with my coworkers to show them my storyboards and VR screenshots to get their response. Here are some things that I learned from talking with them:

Use lazy follow to move UI elements

Despite building a VR experience in isolation, I was able to use my wide variety of designs to communicate my ideas with the rest of the Creative Labs team and get some feedback.

Keep things focused

I initially wanted to make more use of the space to teach the user about interactions. However, feedback was fairly negative towards that idea because of the complexity the distance adds to the experience.

Design

The 360 Experience

The Education Station is a kiosk in a large open space that constrains the user to the center of the kiosk. The entire space is designed to keep the user focused on the content.

Physical Guiding Features

Space separated by functional forms

Being in VR means that there's a whole 360 world of mistakes that the user can make and miss out on vital information. This is why I've decided to build a circular kiosk for the user to stand in to visually cue the user that they are constrained to that space. If the user still tries to move past the kiosk, they will be given a warning before getting teleported back into the center of the green zone.

The "guardrail" also acts as an anchoring point for the UI panel, which lazy follows (which means following with a slight lag) the direction that the user's gaze is facing.

This helps with the attention compensation value, by making sure that they don't stray from the play area.

3D behaviour modeling

This feature is informed by the demonstration and explicit teaching values. A huge issue I had during Facebook's onboarding experience was not knowing where to place my hands because the demo video was in 2D. By having a 3D model demonstrating the behaviour, the user will have a better sense of how the behaviour should be performed.

The Educational Content

A major challenge to this project is the lack of pre-established interaction conventions that other design projects can reply on. Even "generic" motions such as pinching to select will be new to the user and must be taught carefully. Furthermore, hands are not constrained to buttons like hardware is, so we need to make sure that these gestures are remembered.

I split the flow of my experience into two portions, the selection portion and the gesture portion. Selection includes generic tasks such as clicking and dragging, while gesture covers Vive specific gestures that may be implemented.

Onboarding Video to prime expectations

Before the user gets to interact with anything, they must first watch a video that demonstrates the gestures in action to help create some context for the skills that they will be learning.

Here you can see that the video demonstrates someone using their hand gesture to perform selection tasks on a car, as the user will be doing in the tutorial.

The onboarding video

Content that builds upon previous lessons

Before the user gets to interact with anything, they must first watch a video that demonstrates the gestures in action to help create some context for the skills that they will be learning.

Here you can see that the video demonstrates someone using their hand gesture to perform selection tasks on a car, as the user will be doing in the tutorial.

1st Person POV Walkthrough

Building upon past experiences with consistent props and tasks

At each stage of the experience, the user will learn something slightly more complex than before. The linear flow of this tutorial both lets the user learn through small increments of difficulty, but also provides the user with an opportunity to practice previous skills learned.

The onboarding video

There are still some unanswered questions about this experience, such as what distance and what role audio should play. This is something that I hope to continue researching in the future.

Impact and Next Steps

Moving forward with the low-cost formal idea

I was integrated into the main design team to help define and design the next Vive OS. I worked with the lead designer to define the onboarding experience for a hands-first input system and was able to utilize the flow and layout of my project here.

Some changes include: Adding a "point" module to the education, adding modality selection and switching, adding notifications.

Stuff I worked on

Hands first tutorial experience built on the low-cost version of my project

Implementation

Building it out in Unity

The final stage of my project was to prototype it in Unity! I built the experience using custom-written scripts and existing assets to create a clickthrough that helped stakeholders get a better feel for the experience.

Conclusion

Reflection

Even the silliest ideas hold a kernel of truth
Kind of crazy coming from a game designer, but I used to be quite judgemental about my own silly ideas. But being able to use the blue sky ideation technique to reveal underlying values was really cool and is something that I will hold onto in my practice.

Don't jump into VR until you have the basics of the experience down
I got too excited and started in VR too early, overwhelming myself with the infinite space I had to work with. Next time, I'm going to focus on designing in 2D so that when I do move into 3D, I have enough constraints and also questions that I can investigate with that medium.

Take breaks and practice self care
As I began this internship at the start of the COVID lockdown in March, my entire world was consumed by the internship for a while. Without the constant prodding of my manager, I would have let the work consume me and got burnt out. While I did get a little worn down, I was able to catch myself and set healthy work/life boundaries that let me wake up each morning refreshed and ready to work.