I had a manager at Intuit, Krithika Swaminathan, who pushed our accessibility team to explore how we could measure the work we are doing. Like most accessibility teams, we had been focuseing on performance metrics, like the number of audits or how many automated testing errors were found.
But over the years, we’ve collected a treasure trove of information, from customer research, manual audits, workshops, Accessibility Champions, and feedback channels. We’ve used this information to explore metrics that answer the who, what, when, and how of our customers and products.
With AI, we’ve been able to tap that rich data to find new measurements. This week, I’ve been exploring something I hadn’t seen before. How do we quantify the disability tax built into our product design. This is done by extending the Customer Effort Score to account for workarounds.
Customer Effort Score
Customer Effort Score (CES) came out of research published in the 2010 Harvard Business Review article Stop Trying to Delight Your Customers. The research looked at more than 75,000 customer interactions and made a fairly simple argument: reducing customer effort matters.
CES usually asks a customer to rate the effort required to do something, followed by a question that helps explain the score.
For product research, I’d make it simple:
- How easy or difficult was it to complete this task?
- What made you choose that score?
For accessibility, I’d add one more:
- What did you have to do to make it work for you?
That last question interests me.
The work we don’t see
A disabled customer may complete a task successfully and still have to do considerably more work.
They may switch from the app to the website. They may zoom in to read a field, zoom back out to understand the page, then repeat the process. They may know a certain control doesn’t work with a keyboard, so they avoid it. They may ask someone else to complete part of the task.
Sometimes the workaround has become so familiar that the person no longer thinks of it as a workaround.
I remember interviewing a customer who had developed a routine for finding a particular button. They would use JAWS to locate the second table on the page, then use Shift+Tab several times to move backward until they reached the button.
They did this every day.
Our product worked, at least according to task completion. But we were charging that customer extra effort every time they used it. We fixed the experience by adding the headings they needed to navigate the page more directly.
That extra work is what I think of as the Disability Tax in product design.
This idea isn’t completely new
There’s already research behind this idea.
The Disability Tax is the cost of living in a world that was not designed for you. Jason Fields describes it in The Invisible Burden: Understanding the Disability Tax: “This term encapsulates the extra costs—monetary, emotional, and time-related—that are requisite for those living with a disability to simply access and engage fully within their day-to-day lives.”
Researchers have also tried to measure pieces of it.
A 2004 Disability Rights Commission study compared task completion and time for blind and non-disabled people on websites with different levels of accessibility. Watanabe later measured how proper heading structure affected task-completion time for blind screen-reader users.
In 2020, Meagan Griffith, Brian Wentz, and Jonathan Lazar went even further. Their research specifically looked at ways to quantify the amount of time blind people lose because of web accessibility barriers.
So measuring the cost of inaccessible design isn’t a new idea.
What I haven’t found is a common product metric for it.
That’s where I think CES could help.
CES as a signal of the Disability Tax
I wouldn’t call the difference between two CES scores the Disability Tax. Disability is too varied, products are too complicated, and a non-disabled customer shouldn’t automatically become our definition of a normal experience.
But the difference can be a signal.
Ask the same task-level effort question during research with disabled and non-disabled customers. Look at the scores, but also look at time, retries, workarounds, channel switching, and requests for assistance.
Then listen to the explanation.
If disabled customers consistently report more effort on the same journey, we have something worth investigating. And unlike an accessibility audit, CES can expose experiences that technically work.
A form can have labels and still be exhausting. A screen can have headings and still require too much navigation. Authentication can technically be accessible and still require several attempts, another device, or help from another person.
- Compliance tells us whether we met a requirement.
- Customer effort starts telling us what it cost someone to use what we built.
I’d like to see us measure both.
The goal isn’t to invent one perfect Disability Tax score. It’s to make some of this hidden work visible so product teams can start reducing it.
In a follow-up, I’ll look at another place we can find it: the customer research we’ve already collected.
AI Notice: This post used AI assistance for research and language.