Topic 11 Accessible Visualizations

Learning Goals

Learn to chose a color palette that is distinguishable by everyone
Understand the basics of how a screen reader and keyboard navigation works and how someone who is blind or someone with motor impairments might process and interact with data visualizations
Learn to create visualizations accessible with a screen reader or keyboard navigation with the Highcharter library
Learn how to map a variable to sound to create sonifications using the Highcharter library

Download a template .Rmd of this activity. Put it in a folder Assignment_11 in COMP_STAT_112.

Introduction to Accessibility

At the beginning of the COVID-19 pandemic, the CDC created a flatten the curve graphic, which illustrated the importance of staying socially isolated to prevent the spread of COVID. Throughout the pandemic, the New York Times and various government organizations used interactive data visualizations that allowed people to see the number of local cases and death rates. These graphics have played a crucial role in allowing people to make informed decisions about protecting themselves. Unfortunately, these visualizations have been largely inaccessible to people who are blind.

As you are learning in this class, data visualizations are an powerful way to quickly draw insights from data and gain a better understanding of the world. However, people who are blind are consistently locked out from accessing them. In this module, we will learn how to make our designs less disabling to this population.

Example 11.1 (Accessing a Data Visualizations Non-Visually) Chat with a classmate and brainstorm ways that someone who is blind might access the data within a data visualization. You can think about ways that they might already be doing so and future technology that could help. Get creative!

Solution

Read a description given alt text (we’ve been writing this all semester),
Look at the data in table form,
Use a tactile graphic,
Use a screen reader, software that reads aloud the different data elements as you scan through them. On a touchscreen,it could read aloud elements as you touch them,
Use sonification to map data to different aspects of sound using pitch, volume or instrument instead of our traditional visual aesthetics such as size, color or position,

Chosing an Accessible Color Palette

Globally over 2.2. billion people have some kind of visual impairment. Most people with visual impairments have some functional vision, this can include color blindness or blurry vision, so it’s important to keep those people in mind when you design your visualizations.

A few tips for designing accessible graphics:

Get in right in Black and White! Make sure your colors can be distinguished when shown in black and white as well as in full color. You’ll need to vary the saturation of colors as well as the hue. The viridis color scales work well for this in R.
If you have points on a background with varied colors (e.g. data points on a choropleth map), use white outlines around the points to help make them distinguishable. High contrast is easier to see for people with low-vision.
Think about using fills, patterns, and/or dashed lines to distinguish between your data points, but be careful not to make an overwhelming graph.
As always, make sure to write good alt text. People with low vision may refer to your alt text to make sense of what they can perceive about the graphic. Reminder that alt text should concisely articulate (1) what your visualization is (e.g. a bar chart showing which the harvest rate of cucumbers), (2) a one sentence description of the what you think is the most important takeaway your visualization is showing, and (3) a link to your data source if it’s not already in the caption. Check out this great resource on writing alt text for data visualizations).

Example 11.2 Copy and paste the data visualization below (showing how flipper and bill length differ across three species of penguins) into this color blindness simulator and look at with the Monochromacy lens. Update the color scale so the points are distinguishable in grayscale. Note that there are awesome web browser extensions such as Colorblindly, that make it easy to see all your graphics in black and white, but they only work on hosted webpages (e.g. not the generated html files you create with Rstudio).

data(penguins, package = "palmerpenguins") 

penguins %>% 
  ggplot(aes(x = flipper_length_mm, y = bill_length_mm, color = species)) +
  geom_point() + 
  labs(title = "How do flipper and bill length compare between penguin species?", x = "Flipper Length (mm)", y = "Bill Length (mm)")

Scatterplot showing how flipper and bill length differ across three species of penguins. Gentoo penguins typically have the longest flipper length with values ranging between 210-230mm. Adelie and Chinstrap have flipper lengths ranging between 180-210mm. Chinstrap and Gentoo have longer bill lengths with an average of around 50mm and Adelie penguins have bill lengths ranging between 35-45mm.

Highcharter Library Basics

We are going to be using the Highcharter library to create interactive and accessible graphics. Similar to Leaflet, the Highcharter library is an R wrapper for a JavaScript library (Highcharts). Because there is JavaScript underneath, that means we can use Highcharter to make interactive instead of static graphics.

To install the Highcharter library, run the following code in the console. NOTE Do not install the Highcharter library from CRAN or the packages menu, because the current version does not support the accessibility module!

remotes::install_github("jbkunst/highcharter@8ff41366c8c411b497b5378d27be48617360f81f")

If the current version of Highcharter is already installed, you can uninstall it by navigating to the Packages frame in the lower right corner of the RStudio IDE, scrolling down to Highcharter and clicking the x on the right-hand side of the row (see image below).

Highcharter Introduction

First, let’s take a look at what a Highcharter plot looks like using the same penguin data we used above. Make sure to load the library before running the code below.

library(highcharter)

hchart( 
  penguins,
  "scatter",
  hcaes(x = flipper_length_mm, y = bill_length_mm, group = species)) %>% 
  hc_title(text = "How do flipper and bill length compare between penguin species?") %>% 
  hc_xAxis(title = list(text = "Flipper Length (mm)")) %>%
  hc_yAxis(title = list(text = "Bill Length (mm)"))

Example 11.3 Chat with someone next to you. How do the default Highcharter and ggplots differ for the penguin scatterplot? If you are feeling brave, try right-clicking the graph and selecting Inspect element to see how the images are represented in code.

Solution

The Highcharter graphic is interactive (Thanks JavaScript!) You can hide/show data sets by selecting the names within the menu. You can also hover over data points and see the location and group it belongs to.
The default background is a little cleaner for Highcharter.
If you hit inspect, you’ll see that the ggplot graph is represented as a single img element in html and the Highcharter graphic is represented as a scalable vector graphic (SVG), with different elements representing each of the points, text elements, etc… This is good news, both because you can make the elements interactive and because you can surface information about the elements to assistive technology such as a screen reader, so someone who is blind does not have to rely on alt text alone.

As you can see the way to write code for highcharter is a little different than ggplot. You use the form: hchart(<data>, <type_of_chart>, hcaes(<aesthetics>), ...)

where

data is the data to chart. Note that you can not pipe in data as you would with ggplot.
type_of_chart is a string (uses "") to specify the type of chart. This value can be: line, spline, area, heatmap, treemap, etc.
aesthetics is the mapping to use for plot. Note that Highcharter will map to color if you use group or value.

You can add more layers to the plot just like you can in ggplot using hc_add_series(). In general the Highcharter library can do almost anything that Highcharts library can, but there is a lot more documentation about Highcharts. This is a good guide on how to convert from Highcharts (JavaScript) to Highcharter (R) notation.

Example 11.4 Use highcharter to recreate the ggplot bar graph shown below showing the number of babies given the 10 most popular female names of all time.

PopularNames <- babynames %>%
  filter(sex == "F") %>%
  group_by(name) %>%
  summarize(total = sum(n)) %>%
  arrange(desc(total)) %>%
  head(10)

PopularNames %>% 
  ggplot(aes(x = name, y = total)) + 
  geom_col()

Bar chart showing the most popular female babynames over the last 150 years in the U.S.

Solution

  hchart(PopularNames, "column", hcaes(x = name, y = total)) %>% 
  hc_yAxis(title = list(text = "Number of Babies")) %>% 
  hc_xAxis(title = list(text = "Name"))

Adding Accessibility Features to Highcharter Graphs

The Highcharter graphs are actually pretty accessible for screen readers and keyboard navigation out of the box, which is really exciting! As you will soon see, when you navigate over the graph with a screen reader it will read aloud a description of the graph, the axes, and let you move between each item on the graph. However you can make them even more accessible using the accessibility and exporting modules. You can read all about the accessibilty options in the Highcharts documentation.

Let’s update the bar chart we made earlier to be more accessible. In the code below, we add in three modules (accessibility, exporting, and export-data), and we use the hc_exporting() option. When we run the code, you should see there is a collapsed menu on the top right side of the graphic. When you click on it, you have the option to download the image in multiple formats, including a table, which can be read with a screen reader or an SVG, which can be printed to make a tactile graphic!

  hchart(PopularNames, "column", hcaes(x = name, y = total), 
    accessibility = list(
      enabled = TRUE,
      keyboardNavigation = list(enabled = TRUE))) %>%
  hc_exporting(
    enabled = TRUE,
    accessibility = list(enabled = TRUE)) %>% 
  hc_yAxis(title = list(text = "Number of Babies"),
    accessibility = list(description = "Babies Count",
                         range = "0 to 4.5 million"))%>%
  hc_xAxis(title = list(text = "Name")) %>%
  hc_add_dependency(name = "modules/accessibility.js") %>%
  hc_add_dependency(name = "modules/exporting.js") %>%
  hc_add_dependency(name = "modules/export-data.js")

Using a Screen Reader

Adding the accessibility module, also gives you more fine-grained control over what someone with a screen reader hears when navigating over your page. We will practice with a screen reader to give you an idea of how someone who is blind might interact with your data visualization. It is important to note that using a screen reader as someone who is sighted is a different experience than using a screen reader as someone who is blind. As a novice screen reader user, you will likely find it quite challenging to navigate even an accessible page.

VoiceOver on Apple Computer

VoiceOver is one of the most popular screen readers (especially on Mobile Devices), and it comes built in on any Apple product (you can try it on your iPhone as well). If you don’t have an Apple computer, I would recommend partnering up with someone who does for this next part (although you are welcome skip to the next section if you want to try running the Chrome Browser Screen reader on a Windows computer instead).

To turn on and off VoiceOver (warning it will read your screen), use Command-F5. Try turning it on and then off again.
You should be able to use your mouse/keypad as normal to skip the tutorial and navigate to your browser with the webpage.
To see captions, you can use VoiceOver key (either Control+ Option or caps lock)-Command-F10.
To navigate through things, use the VoiceOver key (caps lock) with the arrow keys. The up and down arrows with Shift allow you to navigate inside (down) or outside (up) an element (e.g. inside a webpage) and the left and right move you between elements. I found that within the Highcharter graphs, it is often better to navigate with just the arrows (pay attention to when the cursor selection and the VoiceOver selection don’t match up).
Using tab allows you to quickly move between interactive elements on a page (e.g. buttons, links), which is nice for skipping content.
To select something (e.g. to open a solutions disclosure triangle), you can use the VoiceOver key (caps lock) + Spacebar

Practice: Accessible Graphics

Exercise 11.1 (Accessibility Fun) Create an accessible graph! You can pick a graph that you’ve created in a previous class (e.g. for TidyTuesday) or create something new. First convert it to a Highcharter graph, then add the code to make it more accessible!

Sonification with Highcharter

There’s also a really cool module that comes with Highcharter, which allows you to sonify graphs: instead of mapping values onto visual aesthetics, you can map them onto properties of sound, such as pitch, volume or instrument. The Highcharter sonification module supports both line graphs and scatter plots, and you can set it up to either make a sound when you click on a single point or a sound for the entire graph.

Sonification of Single Points

Here’s a sonification of the most popular names, shown as points instead of bars. You can click on each point to make the sound. The sonification module uses a JavaScript function (everything inside the JS() is in JavaScript) to create the sound event on a click. If you are unfamiliar with JavaScript, don’t worry too much about how the code is working, but you can play around with the numbers for:

instruments: you can use ‘sine’, ‘sawtooth’, ‘triangle’,‘sineMajor’, ‘sawtoothMajor’, ‘triangleMajor’ ,
volume: how loud the sound is (can be mapped onto a property of the data such as color),
duration: how long each point should be played (can be mapped onto a property of the data such as color),
pan: maps the sound from left to right on headphones that support it (typically mapped on to the x value of the data),
frequency: the frequency of the note, here it’s mapped onto the y value of the data.
minFrequency and maxFrequency: the frequency of the highest and lowest notes.

PopularNames %>% 
  hchart("point", hcaes(x = name, y = total),    
         accessibility = list(
           enabled = TRUE,
           keyboardNavigation = list(enabled = TRUE))) %>%
  hc_exporting(
    enabled = TRUE,
    accessibility = list(enabled = TRUE)) %>%
  hc_plotOptions(
    series = list(
      point = list( # note that the click function is inside series>point>events
        events = list(
          click = JS("function() {
          this.sonify({
            instruments: [{
                instrument: 'sine',
                instrumentMapping: {
                    volume: 0.8,
                    duration: 250,
                    pan: 'x',
                    frequency: 'y'
                },
                // Start at C5 note, end at C6
                instrumentOptions: {
                    minFrequency: 520,
                    maxFrequency: 1050
                }
            }]
        })
       }"))))) %>%
  hc_add_dependency(name = "modules/sonification.js") %>%
  hc_add_dependency(name = "modules/accessibility.js") %>%
  hc_add_dependency(name = "modules/exporting.js") %>%
  hc_add_dependency(name = "modules/export-data.js")

Sonification of Line Graph

You can also sonify line graphs. Try clicking on the lines in the graphs below. The big difference in coding is that the click function goes inside series>events instead of series>point>events. If you are unfamiliar with JavaScript, don’t worry too much about how the code is working, but you can play around with the numbers for:

instrument: you can use ‘sine’, ‘sawtooth’, ‘triangle’,‘sineMajor’, ‘sawtoothMajor’, ‘triangleMajor’ ,
volume: how loud the sound is (can be mapped onto a property of the data such as color),
duration: how long the series should go and how long each point should be played (can be mapped onto a property of the data such as color),
pan: maps the sound from left to right on headphones that support it (typically mapped on to the x value of the data),
frequency: the frequency of the note, here it’s mapped onto the y value of the data.,
minFrequency and maxFrequency: the frequency of the highest and lowest notes.

If you want to get fancy, you can sonify the graphs so that multiple lines play at once and there are many other options in the documentation about the (Highcharts sonification library)[https://www.highcharts.com/docs/accessibility/sonification].

babynames %>% 
  filter(sex == "F", name %in% c("Lauren", "Emma", "James", "Peter")) %>% 
  hchart("line", hcaes(x= year, y = n, group = name)) %>% 
  hc_title(text = "How many female U.S. babies were named Lauren, Emma, James and Peter over the last 150 years?") %>% 
    hc_plotOptions(
      series = list( # note that the click function is inside series>events
        events = list(
          click = JS("function() {
          this.sonify({
            duration: 3000,
            pointPlayTime: 'x',
            instruments: [{
                instrument: 'triangleMajor',
                instrumentMapping: {
                    volume: 0.8,
                    duration: 250,
                    pan: 'x',
                    frequency: 'y'
                },
                // Start at C5 note, end at C6
                instrumentOptions: {
                    minFrequency: 120,
                    maxFrequency: 350
                }
            }]
          })
         }")))) %>%
  hc_add_dependency(name = "modules/sonification.js")

PopularFemaleNames <- babynames %>% 
  filter(sex == "F") %>% 
  group_by(year) %>% 
  slice_max(1)

## Error in `slice_max()`:
## ! Problem while
##   computing indices.
## ℹ The error occurred in group
##   1: year = 1880.
## Caused by error:
## ! `order_by` must have size 942, not size 1.

PopularFemaleNames %>% 
  hchart("line", hcaes(x= year, y = n, group = name)) %>% 
  hc_title(text = "Most popular name each year") %>% 
  hc_subtitle(text = "For babies born in the U.S. and assigned the female sex at birth") %>% 
    hc_plotOptions(
      series = list(
        events = list(
          click = JS("function() {
          this.sonify({
            duration: 3000,
            pointPlayTime: 'x',
            instruments: [{
                instrument: 'sineMajor',
                instrumentMapping: {
                    volume: 0.8,
                    duration: 250,
                    pan: 'x',
                    frequency: 'y'
                },
                // Start at C5 note, end at C6
                instrumentOptions: {
                    minFrequency: 120,
                    maxFrequency: 350
                }
            }]
          })
         }")))) %>%
  hc_add_dependency(name = "modules/sonification.js")

## Error in hchart(., "line", hcaes(x = year, y = n, group = name)): object 'PopularFemaleNames' not found

Practice: Sonification

Exercise 11.2 (Sonification Fun) Create a sonified graph! You can pick a scatter or line plot graph that you’ve created in a previous class or create something new and add sonification!

Acknowledgements

Thanks to Mara Averick for her detailed blog posts on incorporating accessibility into Highcharter, which inspired this module, mrjoh3 for the github issue that helped me determine which older version of Highcharter to use for accessibility, and the team behind Highcharts and Highcharter for adding such awesome and heavily researched accessibility features.

Topic 11 Accessible Visualizations

Learning Goals

Introduction to Accessibility

Chosing an Accessible Color Palette

Highcharter Library Basics

Highcharter Introduction

Adding Accessibility Features to Highcharter Graphs

Using a Screen Reader

VoiceOver on Apple Computer

Screen Reader Browser Extension on Chrome

Screen Reader Practice

Practice: Accessible Graphics

Sonification with Highcharter

Sonification of Single Points

Sonification of Line Graph

Practice: Sonification

Acknowledgements