Raymond and I were fortunate to attend the Game Developers Conference this year. Compared to the previous year, the crowds were calmer and new sessions and summits were added to accommodate growing interests in VR and UX. This year there seemed to be more focus on delivering (a) Inclusive and Accessible experiences as well as (b) building out Virtual, Augmented and Mixed Reality experiences.
The State of VR
Before we talk about about AR/MR and building accessible experiences, I want to touch on where are we now with VR. Last year was the year that VR started. Through December 2016, sales of high end VR devices (e.g. HTC VIVE, Oculus RIFT) hit $630 million, Integrated mobile devices (e.g. Gear VR, Daydream) hit $230 million and Snap-in Mobile devices hit $150 million (excluding Cardboard) in revenue. VR is starting to become a profitable ecosystem.
If there is anything that VR has proven is that it is useful for storytelling. VR is hard to get right because being in 3-D adds more more layer of complexity to the applications that we are creating (ex. audio, graphics, narration, a user’s free agency..etc). Since 2013, VR tech has matured to enable hands, enable those who do not have headsets to view applications in WebVR and enable users to occupy and collaborate in the same space. Looking at 2019 and beyond, companies like Unity and Autodesk are looking at asynchronous collaboration across borders, real-time cloud renderings and better GPUs.
This year we also saw more fleshed out products like being able to 3D print what you design in VR so you do not have to learn Maya, standardized controls and Haptic gloves.
Ultrahaptics. Haptic feedback as you perform gestures with your hands.
Haptic Gloves.
A lot has been learned for creating inclusive, collaborative and accessible experiences. Google Daydream Labs created 110+ prototypes as a part of their social VR initiative. They learned that for a social VR experience to be successful, it is important to represent the user in their new virtual environment. This representation is called an “avatar.” An avatar does not have to be detailed. In Daydream’s numerous experiments, they learned that when creating an avatar, mimicking a person’s eye is the most crucial factor to have a user accept their place in a virtual world. Other factor’s includes:
- Being able to freely move your hands.
- Co-presence. Seeing the other person’s avatar in the same virtual space.
- Physics. Important to show that there is a connection between a user’s actions and the displayed UI.
Just mimicing a person’s eye movements makes the avatar more real.
There are some things to consider when allowing users to co-locate in an environment. As a user interacts with others in a virtual world, enforcing personal space is important. This can be done by letting the user’s define how close another avatar can be. When playing a game of poker, these bubbles can prevent cheating. Balancing user autonomy and another’s personal space can be challenging. In an event where a perimeter cannot be establish, there should be some punishment for those who can’t play nice. In Daydream, users who cheat will disappear from the poker game as a small punishment. The user who was walked into will have his screen temporarily grayed out to indicate that someone just walked through him.
Another factor to consider is that users are not yet used to perceiving depth in a virtual environment. As seen with Google Tilt and Quill, there is a high learning curve for user’s to work with 3-D objects in a seemingly 2-D space. It is also hard to keep track of where you are. I’ve been known to get lost a few times and it gets confusing trying to create a mental map of an area that’s not static like the real world is. It becomes more difficult when you have to keep track of another user in view as they move around and as you look away. Otherwise, for virtual meetings or any other experiences where walking is minimal, it may be easier to arrange everyone in a circular fashion as much as possible so that one person does not obscure another.
Accessibility in VR
In addition to the Daydream Team’s social VR experiments, ILMxLAB (Industrial Lights & Magic) performed the first public VR play-test around the world with their Star Wars: Trials on Tatooine. Kiosks were set in public areas such as the Target near Union Square. At most, the team was running 20 kiosks at one time in an area. This large scale play-test netted them thousands of participants, including those with disabilities. A few challenges were brought to light:
- We should accommodate for varying standing heights. Those with disabilities are bound to a wheel chair. When placed in a virtual environment, the application assumes that they are sitting. This assumed position means that the user’s line of sight is typically obscured and objects that are placed to high are far out of reach.
- We should allow users to choose between surround sound vs. personal headphones to accommodate for screen readers. A lot of times, users must take off their headsets and use their screen readers to read the text on their computer screen before placing the headset back on. This disrupts impressiveness. In addition, those wearing hearing aids will get audio feedback when using headphones.
- We should accommodate for color-blindness. When describing which buttons to push, simply saying “Press the RED button” is not enough. Adding animation such as a blinking light will approve the user’s performance.
There are many other challenges that we have not been made privy to in order to make VR more accessible. So lets take inspiration from Zootopia!
How did Disney built a world that could accommodate animals of all shapes and sizes?
For Little Rodentia, Disney actually built a small hamster city and has mice moving through and observed how the mice moved through their space and noted any challenges they had. This means that we have to partner with those with disabilities to make effective changes.
In partnership with the Divisibility Visibility ProjectTM, they surveyed 79 participants with various levels of disabilities, they found that the 7 of the most common disabilities in those who experienced VR are:
- Deafness (5)
- Arthritis (5)
- Scoliosis (4)
- Cerebral Palsy (4)
- Autism (4)
- Asthma (4)
- PTSD (3)
98 disabilities were reported since some have more than 1. Across those 79 participants, performing physical and motor activities are the most difficult. Today’s VR experiences rely so much on gesture and body control. Here are the 8 most common activities that are difficult to perform:
- Crouching (34)
- Balancing (34)
- Standing (33)
- Locomotion (30)
- Moving Arms (23)
- Rotating/ Bending Upper Body (21)
- Holding/ Gripping Objects (21) – Especially hard for users who have to hold onto a controller while trying to push and turn their wheelchair.
- Sensitivity to Light (21)
If no there are no alternate gestures to perform a task, at least have a Tour mode so they can still see the virtual world as it moves for them. This will be useful for on-boarding new employees to your company, having users virtually walk through a home before buying it or viewing the latest 3D mock up of a new car before approving it for production.
Ethical Design
As we build and design experiences for our customers, we should keep in mind the obligation we have to them and our developers to create ethically conscious systems. This means that we should avoid using dark patterns when possible. Dark patterns are used intentionally to cause negative experiences for customers. These experiences are against the customer’s best interests and likely to happen without their consent. UserTesting talks about the sinister side of dark patterns on the web as a means to trick customers into spending money. Dark patterns only works in the short-term as it only fosters annoyance amongst the customer base rather than loyalty. Some may argue that sometimes dark patterns can be “good” here and here. However, thinking of UX in terms of good and bad may not be the right approach. Instead we should thinking of UX in terms of effective and exploitative respectively.
Augmented and Mixed Realities
Pokemon GO was a world wide phenomenon last summer and provided a gateway for many consumers to understand and experience AR. Sessions that had anything to so with the Pokemon GO game and AR were consistently full. I believe Raymond had to get in line more than half an hour before the talk began to secure a seat. Just yesterday, news was released that Apple is working on AR features for its next phone. This seems to be the direction that most developers and companies are going in since it is more easily adoptable and makes more sense in a person’s day to day life.
As for MR, we saw the new tethered Hololens which is much more affordable than its un-tethered brother. 
There were also some some alternative controls that students built for games that explores the AR/MR trend.
Alternative game controls 😀 #gdc17 pic.twitter.com/cBV4UDyM4I
— Tawny (@iheartthannie) March 2, 2017
Overall, the conference was fun and very informative. I am so happy to see the industry moving forward in the right direction with all these new technologies.
Raymond will talk more about the technologies in his upcoming post.