Interactions

Authors:
Kim Marriott, Bongshin Lee, Matthew Butler, Ed Cutrell, Kirsten Ellis, Cagatay Goncu, Marti Hearst, Kathleen McCoy, Danielle Szafir

---

 

Data visualizations, such as statistical charts, diagrams, and maps, are an effective means to represent, analyze, and explore data as well as identify and communicate insights. They take advantage of the human visual system's high bandwidth, parallel processing, and ability to quickly recognize patterns. For instance, a table of numbers may be hard to understand, while those same numbers shown in a graphic form (such as a line chart) will immediately reveal a steadily increasing trend.

  Insights

For these reasons, interactive data visualization is central to both exploratory data analysis and data-driven communication by business, government, and science. Furthermore, the use and need for visualizations is not just confined to data experts: Data visualizations are becoming ubiquitous in textbooks, presentations, and reports, as well as in popular media, both online and in print. The design of these visualizations, however, is premised on implicit assumptions about the reader's sensory, cognitive, and motor abilities. People without these abilities are ultimately disenfranchised, and access to the benefits of data visualization and to the underlying information is limited.

A 2011 report from the WHO (https://bit.ly/3raT0kg) estimates that more than 1 billion people—15 percent of the world's population—live with some form of disability, of whom about one-fifth experience significant difficulties in functioning. While it may be obvious that people who are blind or have low vision (BLV) would face challenges in accessing data visualizations, this also holds for other kinds of disability. People who have cognitive and learning disabilities (CALD) frequently have trouble making sense of the abstractions and symbolic conventions used in visualizations, and people who lack fine motor coordination or have tremors may be unable to operate interactive visualizations. Moreover, beyond understanding or interpreting them, people with disabilities may find it very difficult to create and interact with visualizations using existing tools. We assert that this is an equity issue with severe consequences, because a lack of access to visualizations and underlying data impacts health, educational and work opportunities, and lifestyle. For instance, the BLV community has identified a lack of access to educational materials [1] and online data visualizations about Covid-19 as a significant concern (https://bit.ly/2NJzy0C).

This graph explains the importance of taking precautions to flatten the curve of Covid-19 infections. However, this crucial public health message is inaccessible to people who are blind because it relies on a visual representation.

The visualization research community, however, has not paid enough attention to the needs of people with disabilities [2]. Research in assistive technologies in the broader HCI community has focused on BLV access to limited types of graphics (educational graphics and maps for orientation and mobility training) and has not recognized data access as an issue for other kinds of disabilities. We argue that it's time for this to change and for the visualization, accessibility, and other relevant HCI research communities to work together to address this important issue. It will also require visualization practitioners and the organizations that employ visualizations to adequately prioritize accessibility. Inclusive approaches to visualizations are likely to help more than those people with permanent disabilities. There are also hundreds of millions of people with temporary or situational disabilities. New, inclusive approaches could readily become "informational curb cuts" that benefit everyone.

  The Current State of Access to Visualization

There are three main kinds of disability that may affect access to visualizations: visual, cognitive, and motor impairments. It is not uncommon for people to have more than one kind of impairment.

Visual impairments. Visualization research has addressed designs to accommodate color vision deficiencies. This includes color palettes that work well for common kinds of color vision deficiency and practices such as using simulators to assess post-hoc designs. There has also been research into providing access to graphics for the blind. A key question in the case of blind people is how to present visual information nonvisually. The most common solution is to provide a textual description of the graphic. For instance, W3C guidelines (https://www.w3.org/TR/WCAG21/) recommend that online graphics provide alt text, an alternative text description. In practice, however, alt text is frequently missing or inadequate [3]. Another approach is to present the underlying data as a table. Sonification can also be used to present line graphs and other kinds of charts to blind readers. The SAS Graphics Accelerator, probably the most mature tool for interactive data exploration by blind analysts, uses sonification and tables to provide access to visualizations produced by the SAS visual analytics platform (https://bit.ly/3lFsSx1).

However, many blind people want access to the rich, expressive communication afforded by visualization that moves beyond text or audio presentation: They want to be able to reason spatially about data. The Braille Authority of North America (BANA; http://www.brailleauthority.org) recommends the use of tactile graphics to show spatial data. These are usually raised line drawings, but more recently, 3D-printed models have been used. Research suggests that despite the differences between tactile and visual perception, spatial perception and reasoning is fundamental to the use of both visual and tactile graphics, and that tactile maps, scatter plots, and network diagrams offer similar benefits to the corresponding visual graphic (e.g., [4]). Unfortunately, the cost and time for the production of tactile graphics limits their use for data exploration. More recently, researchers have explored the multimodal presentation of graphics on touchscreen devices, providing audio and haptic feedback when the reader's fingers touch the graphic. There is considerable opportunity to explore data-exploration tools leveraging refreshable tactile displays as well as touchscreen-based presentations, as these may provide similar benefits to tactile graphics.

Cognitive and learning difficulties. A spectrum of cognitive and learning difficulties (CALD) can give rise to a broad range of impairments across language, reading, writing, memory, and motor coordination. These different abilities impact learning, problem solving, and judgment, and may affect the comprehension of conventional visualizations [5]. Even though each individual with CALD will have a unique profile of skills and abilities, visualization design generally assumes a universal model of cognitive function with limited variation between individuals.

Currently, design recommendations for CALD-accessible visualizations are not informed by research, remaining largely heuristic. They suggest, for example, coupling visualizations with text or significantly simplifying the visual information provided (https://www.w3.org/WAI/tutorials/images/). Novel guidelines may explore how designing representations that aid working memory or provide visual cues as to data meaning can enhance comprehension, (e.g., [6]).

Furthermore, traditional mechanisms for interacting with data, such as drilling down into visualizations to access more detailed information, may introduce cognitive barriers for people with CALD. Alternative novel approaches should be considered, such as more guided explorations that explicitly scaffold understanding or the use of tangibles and iconicity in visualizations to reduce abstraction where possible.

Motor impairments. People living with motor disabilities (MD) have impeded movement (e.g., restricted range of motion or lack of control) for reasons that include paralysis, involuntary movements, or pain. While static visualizations may be accessible to people with MD, much of the power of contemporary information visualization comes from interactivity. Interaction facilitates data exploration—for example, allowing people to look at the data from multiple perspectives, control the level of detail, or specify views. However, some interactions (e.g., lasso, pinch-to-zoom) may be inaccessible to people with MD.

Researchers have not specifically investigated accessible interactive data visualization for people with MD. On the other hand, facilitation of interaction with technology for people with MD has been researched (e.g., [7]). For example, enabling the customization of standard features such as the timing for interactions, speed of movement, and distance traveled will improve accessibility. There is also an opportunity to consider how to adapt interactions to alternative input devices or modalities that people with MD already use. These include single button, foot-activated switches, joysticks, sip and puff controllers, eye tracking, and speech input.

  Barriers to Inclusive Visualization

The intersection of visualization and accessibility poses significant challenges, requiring expertise from both the visualization and accessibility communities. It also requires deep engagement between visualization producers and researchers, and the disability communities they wish to help. Collaboration with these communities at all stages of production and research is critical to ensuring that relevant problems are addressed, as these communities comprise users with abilities that are often very different from those of experts in visualization. Indeed, critical disability theory would argue that there needs to be equal ownership of the research and production [8].

Some data-visualization practitioners recognize the importance of accessibility (https://bit.ly/394WlLI) and wish to create more accessible visualizations. However, they currently lack evidence-based guidelines and tools that encourage and facilitate the creation of more accessible content. There is an urgent need for more research to address these issues.

In general, accessibility research raises challenges due in part to a small pool of participants and differences in community norms. The recruitment of participants for usability studies also crucially relies on building partnerships with community organizations and requires that time and energy be spent on recruiting, crafting accessible study materials, and obtaining ethics approval. Participants may find it difficult to travel, so researchers may need to visit the participant's home to conduct the study. Furthermore, from the participants' perspective, small populations can be disproportionately asked to undertake research for the greater good, at their own personal cost. Researchers therefore need to ensure that participants see clear benefits from the study for themselves or the broader community.

---

Researchers need to ensure that participants see clear benefits from the study for themselves or the broader community.

---

Research into accessible data visualization can also be expensive. Providing alternative accessible representations can be costly and requires additional expertise for authoring. Existing technologies (e.g., screen-reader software and braille keyboards for the blind) are already expensive, and research in this area often involves the development of new devices. However, research on accessible data visualization is not well aligned with government funding agencies' expectations, and major agencies lack dedicated venues for supporting such work. For example, many funders emphasize the development of broader technological innovations over investigating foundational visualization principles, while others prioritize more established lines of inquiry.

We also lack communication and partnerships between the visualization and accessibility communities. Only a handful of researchers publish in and attend both venues. This lack of intellectual exchange creates a structural barrier to publishing research on accessible visualizations: Publication venues do not have reviewers with the expertise to review work in the area. Although reviewers are knowledgeable in their areas of expertise, most of them are currently not equipped to critique contributions across both fields and may lack understanding of the nuance and complexity of working in the other area. Valuable work may subsequently be considered out of scope.

There are also structural and ethical issues when working with specific populations with different abilities, such as people with CALD, especially adults. For example, informed consent can be an ethical gray area for adults with CALD: They may not understand what they are agreeing to in participating in the research, but it may be inappropriate for a parent to consent on their behalf (https://bit.ly/394b7T2). Such issues make standard research practices more complex to negotiate.

  Call to Action

We assert that now is the time for researchers and practitioners to address the needs of people with disabilities, who are disenfranchised by existing data visualizations and tools. Not only is improving the accessibility of data visualization an important equity issue, but it also provides exciting research opportunities. These include:

• Understanding how people with disabilities currently access data and use data visualization in all aspects of life. This includes identifying barriers and workarounds, as well as how people with disabilities would like to access data and use data visualization in the future. Although this is essential to guiding research and the effective targeting of communications, it is largely missing at present.
• Investigating multiple modalities or devising new devices for accessible data interaction and presentation. Providing information in multiple modalities or facilitating multimodal interaction can better support diverse abilities and broad contexts. Furthermore, rapid advancements in interaction technologies, such as eye tracking or conversational agents, provide tremendous opportunities for new approaches to accessible data visualization.
• Developing new visualization and interaction idioms and establishing evidence-based guidelines for the design of accessible data visualization. Existing visualization and interaction idioms and guidelines generally assume a homogeneous group of users with similar abilities. Broader yet nuanced guidelines with more appropriate idioms will benefit both visualization researchers and practitioners.
• Developing tools to facilitate the creation of accessible data visualizations. This includes not only ensuring that new data visualizations are accessible, but also making existing visualizations accessible. For example, we need more sophisticated accessibility checkers and automated guides, as well as tools that (semi-) automatically transform visualizations into accessible representations to support visualization producers.
• Developing tools to enable people with disabilities to consume data visualizations as well as to create and manipulate visualizations. Existing research tends to focus on consumption, which is an important first step, but we need to empower people to explore data on their own to achieve their personal goals.
• More thorough understanding of which disabilities impact the use of data visualizations, as well as how and why. Although this article has focused on visual, cognitive and learning difficulties, and motor disabilities, our discussion is not exhaustive. For instance, neurological impairments such as epilepsy may also impact the use of data visualization [9].

For this endeavor to be successful, visualization, accessibility, and other HCI researchers, as well as practitioners, must work together. All visualization producers and their employers need to recognize the importance of accessibility and to ensure that the graphics they create are as accessible as practically possible. Accessibility guidelines such as those produced by W3C provide a useful starting point (https://www.w3.org/TR/WCAG/; https://www.w3.org/TR/graphics-aria-1.0/).

This endeavor requires us to build partnerships and develop a shared understanding of best practices. It requires us to build long-term relationships with relevant disability communities. Some logical first steps would be to establish a community around inclusive data visualization and create collaboration opportunities while, at the same time, increasing awareness of accessibility in the data visualization community and awareness of data visualization in the accessibility community. This can be achieved through workshops, panels, and other activities. We hope that this article will provide the impetus for researchers and practitioners to work together on this important issue.

  Acknowledgments

We thank Microsoft Research for its generous support for an Inclusive Data Visualization Workshop that was held in January 2020, which led to this article. We also thank all of the workshop participants, who helped us shape the main points in this article.

  References

1. Butler, M., Holloway, L., Marriott, K., and Goncu, C. Understanding the graphical challenges faced by vision-impaired students in Australian universities. Higher Education Research & Development 36, 1 (2017), 59–72.

2. Lee, B., Choe, E.K., Isenberg, P., Marriott, K., and Stasko, J. Reaching broader audiences with data visualization. IEEE Computer Graphics and Applications 40, 2 (2020), 82–90.

3. Petrie, H., Harrison, C., and Dev, S. Describing images on the web: A survey of current practice and prospects for the future. Proc. of Human Computer Interaction International, 2005.

4. Yang, Y., Marriott, K., Butler, M., Goncu, C. and Holloway, L. Tactile presentation of network data: Text, matrix or diagram? Proc. of the 2020 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2020.

5. Wu, K., Petersen, E., Ahmad, T., Burlinson, D., Tanis, S. and Szafir, D.A. Understanding data accessibility for people with intellectual and developmental disabilities. Proc. of the 2021 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2021.

6. Zhang, D., Ding, Y., Stegall, J. and Mo, L. The effect of visual-chunking-representation accommodation on geometry testing for students with math disabilities. Learning Disabilities Research & Practice 27, 4 (2012), 167–177.

7. Kouroupetroglou, G., ed. Assistive Technologies and Computer Access for Motor Disabilities. IGI Global, 2013.

8. Mankoff, J., Hayes, G.R., and Kasnitz, D. Disability studies as a source of critical inquiry for the field of assistive technology. Proc. of the 12th International ACM SIGACCESS Conference on Computers and Accessibility. ACM, New York, 2010, 3–10.

9. South, L., and Borkin, M. Generating seizure-inducing sequences with interactive visualizations. Proc. of the IEEE VIS 2020 Posters, 2020.

  Authors

Kim Marriott leads the Department of Human-Centred Computing at Monash University in Australia. His research includes both data visualization and assistive technology, with a particular focus on improving access by people who are blind or have low-vision to graphical information. [email protected]

Bongshin Lee is a senior principal researcher at Microsoft Research. She explores innovative ways to empower people to create visualizations, interact with their data, and share data-driven stories, as well as to reach broader audiences with data visualization. [email protected]

Matthew Butler is in the Department of Human-Centred Computing at Monash University in Australia. His research focuses on improving access to graphical information and cultural materials by people who are blind or have low vision. [email protected]

Ed Cutrell is a senior principal research manager at Microsoft Research. He manages the MSR Ability team, conducting research aimed at developing technologies that extend the capabilities of and enhance quality of life for people with disabilities. [email protected]

Kirsten Ellis is lead of the Inclusive Technology group in the Department of Human-Centred Computing at Monash University in Australia. Her research investigates using technology to solve real-world problems for people with disabilities. [email protected]

Cagatay Goncu is a research fellow at the Department of Human-Centred Computing at Monash University in Australia. He is passionate about developing technologies that let people access information anywhere without any barriers. [email protected]

Marti Hearst is a professor at UC Berkeley. Her research focuses on the intersection of visualization and language. [email protected]

Kathleen McCoy is professor and chair of the Department of Computer and Information Sciences at the University of Delaware. Her research focuses on accessibility for people with disabilities (especially accessibility of graphical information) and natural language generation. [email protected]

Danielle Albers Szafir is in the ATLAS Institute and Department of Computer Science at the University of Colorado Boulder. Her research measures how people make sense of visual information to create novel visualization tools. [email protected]

 

Copyright 2021 by author/owner

The Digital Library is published by the Association for Computing Machinery. Copyright © 2021 ACM, Inc.