Interactions

Authors:
Aviv Elor, Joel Ward

---

 

Extended reality (XR) systems that make virtual, augmented, and mixed-reality worlds are now available to the public. Emerging applications have begun to demonstrate XR's benefits for remote work, education, health, gaming, and many other realms. But people with disabilities, who could benefit from these systems, are often an afterthought starting at the XR hardware level. Since the advent of the Internet itself, digital and online technologies have struggled with providing usage for people of all abilities [1]. While some government policies are beginning to recognize and demand accessibility as a human right, access for all abilities is still often an afterthought with emerging technologies [1].

  Insights

In this experience report, we reflect on a yearlong collaboration between various academic and industry organizations toward evaluating the current and future state of XR hardware accessibility. We draw on our group's experience of examining hardware as an inclusive team of XR professionals, both with and without disabilities. In doing so, we offer future directions for accessible XR hardware research and development.

  The Benefits of Extended Reality

Extended reality (XR) has redefined our relationships with the virtual world and each other, providing a means to augment how we work, play, heal, and engage by translating the physical world into the virtual. Interaction within an immersive virtual environment has been found to yield a plethora of benefits for areas such as remote meetings, enterprise training, project management, entertainment such as tourism, well-being programs that stimulate mental and physical exercises, and designing and testing products. For this article, we focus on XR head-mounted display (HMD) systems that include the fields of virtual reality (VR), augmented reality (AR), and mixed reality (MR). In the past decade, these systems have seen explosive growth, with increases in computational power and the affordability of digital systems effectively reducing barriers to technological manufacturing, consumer markets, required skills, and organizational needs. These commercial systems provide a method for conveying six degrees of freedom (DoF) information (position and rotation) to enable users to have a greater social-emotional interaction, which is especially advantageous for combating social isolation and assisting with remote work. However, XR's benefits through enhanced interaction modalities bring many challenges, such as the need for accessibility.

  No One Left Behind: Starting with XR Hardware

The XR industry is rapidly growing, has begun to define the virtual workplace, and is becoming a rich medium for social-emotional experiences. Yet an unfortunate truth is that most of these technologies are not designed for accessibility from the start. We must take steps to ensure usability such that XR technology can enable interaction for communities with disabilities. Much as how the Internet has become an integral part of human life and work, the growth of XR may suggest a similar trend. Therefore, there are many reasons to fight for XR accessibility: establishing equality of the digital and social workplace, engaging more users toward XR market growth, working toward more robust and user-friendly 3DUI interfaces, and enabling greater user and developer diversity, which is correlated to innovation and out-of-the-box thinking. To establish a future with accessible XR experiences in mind, we argue that there is a need to approach XR accessibility at the hardware level. Hardware, after all, is what drives the capabilities of the software application from input methods, output modalities, control, and communication. By establishing an understanding of accessibility requirements at the hardware level, perhaps we can influence the default capabilities of XR software applications.

The XR Access Initiative Mission (https://xraccess.org/).

To explore solutions for these challenges, we have spent the past year examining XR hardware accessibility needs with an interdisciplinary cross-industry group involving both people with and without disabilities through the XR Access Initiative. XR Access is a community committed to making virtual, augmented, and mixed reality (XR) accessible to people with disabilities [2]. The goal of the 2019–2020 XR Access Initiative was to engage with the academic and industry communities to create guidelines to influence policy for accessibility in XR. Our group led the support of hardware, the starting point of most XR interaction capabilities, so that this technology is built to be accessible for all from the ground up, rather than application by application. Here, we reflect on our experience toward working to establish a future for accessible XR hardware. This consisted of an international workshop on XR hardware input modalities, as well as a year's worth of monthly virtual working sessions. We discuss our findings from the debates, tensions, and questions we have had over the past year in reflecting on the complexity and impact of creating accessible guidelines and developing calls to action. We aim to contribute our experience to offer considerations and future directions for impactful XR hardware research. In what follows, we briefly explore the state of XR hardware accessibility, contextualize our contributions, reflect on preliminary findings, and offer calls to action for future research.

  Contextualizing Our Experiences

Accessibility is a human right. As we dive deeper into the immersive virtual world, we must ensure that people of all abilities can participate. Consequently, the XR Access Initiative community was organized as an outcome of the 2019 XR Access Symposium [2], which brought together more than 140 advocates, researchers, technologists, and business leaders to advance the design, development, and production of accessible XR. The overarching mission of our initiative, as collaboratively defined with our working sessions, is to work with hardware manufacturers and academia to evaluate the current and future state of XR hardware while considering guidelines and user needs.

To help contextualize our experience and motivations with XR accessibility, allow us to briefly introduce ourselves. Aviv Elor is an HCI doctoral student researcher whose interest was sparked by a short-term disability that developed after reconstructive surgery on his dominant arm when he was an undergraduate student. He engaged with VR to stimulate physical recovery during this period and became captivated by XR as a games-for-health medium [3]. Joel Ward is a technology strategist and innovator who has worked with accessibility since the early days of the World Wide Web and has been working with XR since 2016. This includes enabling and enhancing interactions for his son, who has disabilities, in the virtual and augmented worlds. Joel has also been speaking about the importance of accessibility for XR online, on his blog, inside his company, and on the air via the Workology podcast [4].

---

Accessibility is a human right. As we dive deeper into the immersive virtual world, we must ensure that people of all abilities can participate.

---

After the 2019 XR Access Symposium, the authors formed and led the XR Access Hardware Devices 2019–2020 working group. This team expanded upon the XR accessibility efforts defined by the W3C, XR Association, and OpenXR to evaluate the current and future state of XR hardware accessibility toward understanding guidelines and future needs. This consisted of monthly virtual workshop sessions from 24 contributing XR professionals with and without disabilities from the following backgrounds:

• Companies: Booz Allen Hamilton, Dell, Google, Hewlett-Packard, Magic Leap, Microsoft, Sony Interactive Entertainment, and Verizon
• Organizations: AbleGamers, National Industries for the Blind, the Partnership on Employment & Accessible Technology (PEAT), and the XR Association
• Universities: University of California, Santa Cruz, and Cornell Tech.

All participants volunteered their time during these sessions to discuss challenges such as orienting XR hardware by accessibility needs, evaluating systems for accessibility, organizing involvement from XR hardware manufacturers and communities with disabilities, and brainstorming research programs to drive XR Access hardware adoption. With our team contextualized, we discuss our working-session organization and findings.

  Establishing Goals of XR Hardware Accessibility

At the start of our kickoff meeting, we reflected on the pilot work and hardware discussion from the XR Access 2019 Symposium report [2], as well as our own experiences to define a set of working goals. Through our meeting and peer review with XR Access, we established the following considerations for XR hardware accessibility research:

• We must determine the state of XR hardware accessibility, including the evaluation of current devices, reaching out to synchronize with hardware companies' current and future plans, and working with academia to determine next-gen devices coming to fruition.
• We must understand related fields of hardware, like general hardware accessibility, and what has been done before to ease accessible practices into XR hardware, and expand upon the efforts of policy organizations such as the W3C and XR Association.
• We must determine the most pressing problems in XR hardware accessibility, creating a community forum and hardware-evaluation pipeline for involving communities with disabilities.
• We must consider the Guidelines & Taxonomy for Accessible XR Hardware, working with industry and academia to create advice and a common terminology that will also include functional performance criteria.
• We must increase the public awareness of XR hardware accessibility, providing support, resources, and incentives for XR hardware manufacturers and developers alike.

  Understanding the State of XR Hardware Accessibility

A plethora of research has begun to examine accessible design and inclusive tools for XR software accessibility. In terms of hardware studies, research has primarily focused on alternative XR input/output methods and/or adaptive sensing capabilities, as discussed in related work, such as that on custom vibrotactile walking canes to navigate virtual environments without vision. This research is exciting, as it could potentially lead to breakthrough input methods such as Xbox's Adaptive Controller, which garnered adoption in communities with disabilities [5]. However, little work has been done to establish clear XR hardware-usage requirements, and most studies still examine XR accessibility on an application-to-application basis.

The 2019 XR Access Symposium, a community event that brought together more than 140 advocates, researchers, technologists, and business leaders to advance the design, development, and production of accessible XR (https://xraccess.org/).

Additionally, many policy organizations are taking note of XR accessibility needs and organizing workshops to establish directions for future research and guidelines. For example, the W3C has brought together more than 170 participants to examine VR and the immersive Web [6]. These participants, including browser vendors, headset and hardware manufacturers, and VR content providers, designers, and distributors contributed toward identifying standards for Web-based XR systems and have begun to discuss hardware needs. The accessibility group considered input modalities and application APIs to assist users with vision and hearing disabilities, while also providing extensive software recommendations in future workshops and WebXR resource standards. In the W3C 2016 workshop on WebVR, the executive report highlighted the need for VR hardware systems to "automatically adjust to meet the users restricted modality to bring controls closer if full range of motion is not possible" and "allow multiple ways to interact with the VR environment, and conversely multiple ways to extract information from it" [4]. Aside from these discussions, there was little to no talk on XR hardware accessibility. The breakout session argued that VR systems need to automatically adjust to meet user input modality and allow multiple ways to interact with the VR environments, yet more work into mapping these requirements for hardware was needed. Given such discussions, we have an understanding of how accessible XR hardware should act, and now we need to expand upon this work to establish generalizable XR hardware solutions and recommendations that are designed directly with the disability community.

  Mapping Hardware Resources for Accessible XR Hardware Evaluation

From our discussion and collaboration with the Guidelines & Policy Group, we have begun drafting evaluation metrics for physical hardware as well as operating system/platform input methods. This includes understanding the applicability of current XR HMDs for the following:

• Accessible usability needs: without vision, limited vision, without color perception, without hearing, limited hearing, without speech, limited physical manipulation/strength, limited physical reach, photosensitive seizure triggers, and limited cognition.
• Hardware components: operable parts, alternative input method controllers/hardware, biometric controls, status indicators, audio volume control, audio microphones, alphanumeric input keyboards, and keypads.
• Hardware usage types: simultaneous input, ease and force required to operate input, discernible tactile interfaces, tactile examination without activation, spacing, color, toggle of control status, visual label contrast and alternatives, and biometric ID/activation and input methods.

We should note that these considerations are a work in progress that may dynamically change and expand as our involvement extends to greater areas of communities with disabilities.

  Needs for Action: Building a Community-Centered Approach

As we reflect on our community goals, mapped accessibility needs, and the state of XR hardware accessibility, we consider steps toward action. We have had many heated debates within our working group and would like to address some of them in consideration of future research. For starters: What came first, the chicken or the egg? The chicken being manufacturing/evaluating accessible XR hardware and the egg being interest in accessible XR experience for communities with disabilities. At the current moment, the XR industry itself has quite a bit of maturing to do, and experiences by AbleGamers and similar organizations have often found a lack of interest in people with disabilities wanting to engage in co-design toward XR accessibility for some areas. We can develop guidelines for XR hardware accessibility, which can increase awareness, but these guidelines may not address the generative design-thinking tools that developers need [5]. Additionally, we must consider the complexity and communicative nature of guidelines. We can establish a set of criteria to evaluate headsets; however, such guidelines cannot tell us whether technology is accessible or impactful to the individual [5]. To paraphrase, if the Xbox Adaptive Controller is the most efficient tool to play a racing video game for someone with a disability, why suffer through the use of an XR platform that is inherently unusable for many disability needs?

XR is slowly transforming our workplace, social lives, healthcare, and much more. Research has demonstrated benefits in many of these areas for communities with and without disability. As XR becomes mainstream, those who cannot use these devices may be left behind or required to use less immersive mediums at a disadvantage. In reflecting on these questions and the growth of XR hardware, we are organizing calls to action within the XR Access Initiative and the broader academic and industry communities.

  Calls to Action: Community Involvement and Challenges

We highlight our calls to action for the research communities to work toward the future of XR hardware accessibility. Based on our experiences and discussion from the 2019–2020 hardware working group, we present the following actions for consideration:

• We call for literature reviews and ethnographic studies centered around communities with disabilities in mapping XR accessibility and hardware research to advocate for inclusive design and advance promising practices.
• We call for evaluating current XR hardware systems to identify user needs, interests, and pressing problems for communities with disabilities in exploring how to bake accessibility support into common hardware platforms.
• We call for the formation of a Web-driven community directly connecting people with disabilities to XR hardware developers in documenting and disseminating accessibility.

Moreover, we are working with hardware manufacturers and nonprofits to organize a hardware loaner program to invite communities with disabilities to engage in XR hardware evaluation. We shall continue our cross-industry collaboration toward identifying active research projects and disseminating our findings in a public and accessible format. To this end, XR Access is expanding its vision for 2021 and beyond. We invite involvement from everyone and hope to further diversify our community toward shaping the future of XR accessibility.

  Reflecting on the Future of Accessibility and XR Hardware

Extended reality holds the potential to redefine the virtual workplace, online social interaction, healthcare, and much more. However, people with disabilities are not able to access and benefit from these experiences, as XR hardware accessibility remains largely undefined and underdeveloped. In this experience report, we have reflected on a yearlong collaboration toward exploring the future of XR hardware accessibility with an interdisciplinary, cross-industry group of volunteers, both with and without disabilities. By collaborating in a series of virtual working sessions and a symposium workshop, we offered a reflection of our experience in establishing goals, mapping out hardware, understanding accessibility, and calling on everyone to address XR hardware accessibility. As we plan to engage in these challenges, we invite the academic community to help define the future of XR accessibility.

  Acknowledgments

We would like to recognize and thank the many active participants of the XR Access Hardware Devices working group. We also thank the XR Access Executive Team, especially Bill Curtis-Davidson, Dylan Fox, Devin Boyle, Shiri Azenkot, Larry Goldberg, Jessie Taft, and Liz Hyman.

  References

1. Jaeger, P.T. Disability, human rights, and social justice: The ongoing struggle for online accessibility and equality. First Monday 20, 9 (2015); https://firstmonday.org/ojs/index.php/fm/article/view/6164

2. XR Access. XR Access: A community committed to making virtual, augmented, and mixed reality (XR) accessible to people with disabilities. 2020; https://xraccess.org/about/

3. Elor, A., Teodorescu, M., and Kurniawan, S. Project star catcher: A novel immersive virtual reality experience for upper limb rehabilitation. ACM Trans. on Accessible Computing 11, 4 (2018), 1–25.

4. Ward, J. Let's make XR accessible. Joelsef Explains It All. Jan. 12, 2020; https://joelsef.com/lets-make-xr-accessible

5. Cairns, P., Power, C., Barlet, M., and Haynes, G. Future design of accessibility in games: A design vocabulary. International Journal of Human-Computer Studies 131 (2019), 64–71.

6. World Wide Web Consortium (W3C). W3C Workshop on Web Virtual Reality: Workshop Report 2016; https://www.w3.org/2016/06/vrworkshop/report.html

  Authors

Aviv Elor (www.avivelor.com/) is a doctoral student at the UC Santa Cruz Department of Computational Media and a researcher in ASSIST Labs. He is affiliated with the XR Access Initiative (https://xraccess.org/) as the deputy leader of the 2019–2020 Hardware Devices Accessibility working group. [email protected]

Joel Ward (joelsef.com) is a technology strategist and augmented reality product manager at Booz Allen Hamilton. He is affiliated with the XR Access Initiative as the leader of the 2019–2020 Hardware Devices working group. Additionally, he was a founding member of the Booz Allen Innovation Center in Washington, DC. [email protected]

 

Copyright 2021 by author/owner

The Digital Library is published by the Association for Computing Machinery. Copyright © 2021 ACM, Inc.