Testing Web Accessibility – Adrián Bolonio (A11yTalks – August 2019)

To accessibility talks where every month virtually and talk about all things, Accessibility focusing on website usability Each month. We invite a speaker To present an accessibility topic and afterward, we invite the community to ask Questions and participate in discussion. So if you have a question for our Speaker or for anyone on the blog, please post it in the zoom chat window.

Or with the hashtag a-1-1-y-t-a-l-k-s in Twitter, That’s A11yTalks, I’m today’s Host AmyJune, I’m the open source community ambassador for Kanopi Studios and our speaker today is Adrián. Bolonio, Oh and I might have messed that Up He’s a front-end developer and web team Lead in this month’s talk, which is testing web accessibility, Adrián will Speak a bit about automated accessibility tests and how they can Free up our QA team for manual testing every part of the application, but they Cannot do that automatically and magically to make our site accessible so He’s going to show us some testing tools, libraries and techniques to increase the Accessibility test coverage of our code And he’s going to do that by using a Simple react, application, so that’ll be great, So Adrián.

I wanted to ask you a Little bit about yourself, you know we had some discourse in email and you Mentioned that you had a love for traveling and that you recently went to Italy, with your girlfriend um, do you want to share with folks the exciting News that happened in Italy Yeah well, thank you for having me here. First of All Yeah we’ve been in Italy for a couple of Weeks and then yeah we’re engaged now so so all happy nice, but yes yeah.

I love Traveling, actually, that that’s your your your right, every time that that money, Allows it and time allows it, then I’m I’m traveling Right. Where is your next Trip my next trip is to Colombia, I’m going to be speaker in the JavaScript Conference in Colombia, Yeah, and so I can, I spend a week in Colombia, so it’s going to be very exciting, Fun, fun and when you get married, did you Guys pick a location, are you going to travel for the wedding? Not yet it’s Going to happen next year: yes, Oh, Oh fun! Okay! So um are you ready to get started? Absolutely yeah Perfect, take it away.

Yes, So it’s all good! You see my so. You see The presentation Yep Perfect, so yes, I would love to talk about testing web Accessibility today, So, let’s start again, my name is Adrián or Adrián Bolonio, I’m From Madrid, I’m from Spain as AmyJune said, I’m a web team lead of the front-end Developers in my company I’m a front-end developer of that team and you can find Me under my surname in Twitter Instagram keep health everywhere.

So let’s talk About web accessibility in the real world, we can see here three cases that Is obvious that are not implemented and build correct to the people that are Targeted to so Probably because they were built by not disabled people Or Maybe because they didn’t thought to put in their shoes, But it is very clear that Those three cases are wrongly done. Another is another case that that I can Show you in the real world from the wheelchair user perspective, this is Completely wrong and it’s clear that this was not tested from the end user.

Perspective, no one thought about sitting down and clicking the button and opening The door and see that there are stairs so why is it so clear in the real work But not in the online world, so let’s see how how can we translate this? This Reality to the online world, because it’s not so obvious and not clear when we not Visually impaired people when we see a website that is not accessible, so, let’s Make it an example: if you, you are in an online e-commerce and an online shop and You want to you call the IT support.

You want to know your latest purchases, so a Typical answer from the help desk would be: you need to click on the button in The top right corner – or you want to change your password, so they can say you Need to click on the button with the engine icon. The problem here is that People with visually impaired disabilities or blind people will not See anything there’s no such as tab top right corner, there’s no such as engine Icon, so we need to get these reality when we develop applications.

Why so? Important what accessibility? If we look at the numbers last year, we were around Seven point: six billion people in the world. The problem comes because more Than 1 billion people live with some form of this one impairment from light. Two hundred percent, which would be blind people, so we need to make our back Accessible as for as much users as possible, so my team and I, as I said, I’m We are working in a platform that allows people to sell secondhand goods, so we Are the second largest Austrian website in traffic? That means we need to to Reach these people and we need to to make our website accessible for for all These people, so we are not going to throw I’m not going to talk about developing Web accessibility today I gave a talk some time ago about developing web Accessibility is called, how does your website sound like, and anyone can read? It but today I want to focus in testing web accessibility, so, as you said in the Introduction automate tests can free up your your QA team for manual testing Every single bar of your application, but it cannot automatically make it Accessible magically, so we need to think about this ultimate accessibility test.

As one step of a larger testing process, so we don’t, we cannot forget about the Fact that only twenty to fifty percent of all accessibilities can can be Automatically detected, so we don’t. We cannot skip this this manual part so as I said: we’ve been using some libraries, some tools that I would like to show you A introduction about the tools, so we we will start or I will start talking about. Testing the code, so I create a very, very small react application with three tiny Components, one is the button.

One is a fake button, which is an anchor link and One is an image and I put a bunch of errors in the HTML, so the first one is Reax, which is built by the Qin and is probably one of the one, the the tools That I use more so is install as a developer dependency in your react. Application – and you can call it passing the react on object. Why so? Because it Scans, your code, and it keeps you in the chrome, dev tool console in my case.

I’m Using Chrome all the issues that your website does, the good thing is it’s: if The issues are group and it’s group as well by severity level, so you can have Front minor, which could be the warnings to critical, which is the things that you Need to tackle first so again you can automatically see while you are Developing in the in the browser, what issues to have Another thing, another tool that I that we use and we have in our code – Guidelines is the es link plugin for accessibility for JSX, which is the react Language, you need to configure the es lint our situation file and you can Configure the rules that you want to use as we see in the line 12, we extends they Recommended rules by the accessibility, all the accessibility rules as well.

We Can extend the a strict at once so instead of warnings, you will see errors So your application will not be able to build, and in the code you can see what Errors, viewer, you are having and basically hovering the the line that you Have the earth as well, when you run the application, you can see these issues as Well in the terminal so again very very helpful. While you are coding, you don’t Even need to pass any test, you will see them automatically one of my favorites Because we tend is Jax just acts because we tend to to forget the test in writing.

Test is part of the development process. It’s not a post development process is a Part of the development process, so you can write unit tests with just and with This library just acts as well as a developer dependency. You can render into A string the whole application and you can check if the application have Violations so again, if you pass the test in the terminal, you will see pretty much The same issues that you will see in the chrome tools in the ultimate tests, this Is very helpful in your testing.

I belong if you have end-to-end test With selenium or you can have unit test you will, you will see how the Application would not build or will not be deployed to production if the tests Are not fast and with the results you can build reports to your to your Product team or to your development team, so this we’ve seen how to test in the Code so let’s see, how can we test in the Dom structure, mainly from the terminal So again, acts library has an CLI in the terminal, so you need to install it.

Globally, to be able to use it and you can run the command in the terminal. Followed by the URL of the page, you wish to test as well. You can pass localhost, so you can test in your local environment or you can use The flags, the rule flags, so you can test only one rule, in particular in the Second example: I’m testing only the color contrast, so if it I want to test If it’s compliant with a double A or the Triple A the good thing for this library, Is that it gives you the issue, the violation give you why in and where and Give you an link to the deck where University, so you can read more about How to fix this issue? There is another library which is very similar to ax Which is pa-11? Why and it basically does the same so you can.

You have to install It globally, yes NPM package and you can run it in your terminal, followed by the Url of the page, do you wait to test again? You can test against localhost and You will see all the issues listed. What years? What are the issues? Where are the Issues so it will see, it will show you the HTML, where the issue is was found. And a bit of information for me, one interesting thing about this: Library is that it has a CI in the so you can use it in the code, so you will Have to create a P, a PA 11 y CI dot JSON file, and you can pack and the URLs that You want to test, you don’t need to test one by one.

You can generate a full Reaper so in the URLs you can use it blind. I have plain you can use it as a conflict, so you can see down about or You can screen capture, so you will at the end of the test. You will see the Results and you will have a screen shot of the page, so you can test the visual Inconsistency as well and for me again, one of the interesting things about this When is, are the actions, so not only you can load the page and see if the Accessibility tests are passed, but you can perform actions so, in the sake in The third case I’m calling the Drupal group’s website and I’m waiting for the Element with the class menu three four, seven, three, which I expect the element I Know is the event link to be visible.

Then I take a capture. I click the Element and I take another capture so at the end I will have a screenshot of both Hamza and google provides a library which can be used from the dev tools as Well, so you can run this directly from the browser, but in this case i’m running This from the terminal, so I will show you the article, so you can see how how it Runs so lighthouse run a drum instance and it Will start performing tests not only accessibility but performance and SEO Because it’s my experiences SEO as a accessibility and performance both has a Very huge impact in SEO for google, so at the end of the test you will see all the Tests at the pimple and it will generate a report inside the Report, you will have the accessibility tab and you can see the score that you Have and again some information about the issues where and how to fix them.

So This is how this is some tools that you can use to test the Dom structure and, as Well, you can test in the browser, and now we are going out of the out of my tester To go to the manual testing, because all these tests that I’ve been talking about Can be automated in scripts or in the pipeline of the of building and Deploying so in a in a continuous delivery and continuous deployment, System but in this case I’m going to talk a bit about manual testing in the browser So I just selected three like a three plugins that I think I I used quite Often so this is the small react, application that I create and created, and I’m going to show you the Okay, first riad acts as we as we said, so you can see the the element and they are Highlighted so you can see.

Where are we? Where are we, then? The first one is acts So if you click in an allah ice, you will see again the issues that you have some Information about the issue: where is the issue found, and the good thing is how to Solve it and if you want to learn more again, there’s a link to the tech way. University, the second one is the art and our so a RC toolkit. So if you run test Then you can see group pi test and, for example, and they the red column, is Errors and the yellow columns are warnings and, for example, let’s take aria Usage, the good thing of this library is that they are highlighted in the code as Well, in the interview – and you can highlight where you are inspecting so in The in this case, there is a fake button that has a role and any set in valia Invalid aria role specified and if you click, you will directly go to the Element in the HTML inspector other thing that the chrome dev tool has is Enact a new accessibility panel, where you can see the whole accessibility tree Which is the Aria tree, so you can see what the screen readers will read when When the element will be focused, I think Sergey hey, I know Sergey.

I talked about This a couple of months ago, very interesting, so really read this Talk because it’s really interested and some other manual testing the rice that I tend to do is climbing a Chrome extension called no coffee. Which is a vision simulator, so you can simulate how what what can happen when I When a person has a kind of blurry vision, so this is how your website will Appear for them or someone with protanopia.

In this case the colors are Different, so it’s very interesting how to to to simulate the disability of Someone so you can develop this application in a better way, so yeah Those are three extensions that you can use to test your application in the Browser again manual testing, the small react application that I create again is Just a test to to to use these tools, you can find it in the repository in github.

In my in my profile, and I want to leave with with the sentence that I found and I really really like it, which is it says it just it’s not just about disabled Users being able to access your website is about everyone being able to assess Your website, so when we develop and with web accessibility in mind, we just we Don’t develop for disabled people, we develop for everyone, so so we need to we Need to to think about this, and that’s it! Okay, thanks Adrian, do you want to put us On you know, stop sharing your screen there.

You go So are there any questions from anyone on the call today? I know that there was A question from from Andrew he was curious to know which Jas X, Alli air as You extend beyond the default recommended ones, yes, so the errors that I am used for this example and again you can see it in the in the repository. Are All tasks, and you can say which elements you are looking for alternatives in this Case I use image, object, area and inputs with the type image no interactive tab.

Index so I want to avoid table indexes and different numbers, so not interactive. So in a div or in a label, no static element interactions, so I don’t want to I’m looking not to have on click on mouse down on Mouse up on creep press on Key down and on key up in a non static element. So only the the elements that Could have on click should have on click, so no thieves that looked like a button. That you can click on them.

No redundant rolls label has associated control again. Labels with an input so no input without labels, interactive support, focus again. Focus focus of all elements should only be part on check box link search boards. Switch text box le Phaedra Phaedra, the anchor link is Valid – and this is very interested because you can specify here the Components do you looking for, so, if you remember at the beginning, I I create a Tiny component called fake button, which is an anchor link, so in This case, I’m extending anchor is Bartlett and I’m passing the component Fake button, so every time that in react, I’m using this fake button, the Linton Will say, look, this is not.

This is not an phallic anchor and I can look for Special link so href left and HOF right and no ATF. Obviously, anchor has content. So, for example, the most typical example for this is you have the logo of your Website and is wrapped into an anchor in the image or you have an anchor with Nothing inside so the every Ankara should have an a Content. I’m looking Forward language Aria roles, so in heaven heaven has content, so I’m looking for as Well, heading so h1: h2 that has content no empty h2 host because of we are Things you want to use it as separate or whatever things that you should not to That’s that’s the the extension.

So I’m doing but again, if you, if you want to Take a look of the of the repository you can find everything specify the labels So to one it depends on critical minor, whatever you want to do to treat them. Perfect um so certainly has a question for you. You wanted to know if you use Cypress js4 accessibility, testing and if you have yeah no, no, we don’t. We don’t Use Cypress, we use react testing library from Kent, art asked he said.

No, no. We Don’t use Cypress we evaluated to use Cypress at the beginning, I mean we use Playing just and and now we are starting to use, react testing library. If react. Used to have a react, thus I eleven, but it was deprecated Towards react, ax well done liquid, because hopefully it Will be included in further versions of react um. What do you do when the Different tools give you different reports or conflicting reports: sorry ii, What do you do when different tools give you different reports or conflicting Results: okay, it depends on first, you will.

I will check how each library Is configured so, for example, maybe in the linking that we were talking before I I was looking for Encore not valid in in my component, which is the fake button. But if I don’t specify that this, but because I’m specifying explicitly this Component is wrong, but react. Acts in the browser will not say anything because I’m not specifying to this library a this component is not a nun.

It’s not a Valid anchor so for other libraries will be a valid anchor. So first you need to See the configuration of each one if you have a conflicting one, I normally go to The w3c specification and read about like okay, what is the best practice and Why is this not correct in this library and wise is correct in this library or What is the levels? Maybe one is a warning and one is a error. I normally go.

Directly to the specification so and has a couple of questions in the chat Window, the first one he asked was: do you normally test the entire application for Violations in one assertion, as opposed to testing specific components or Screams we do we do both, so the test that you’ve seen is rendered the whole Thing and and pass it through the through the testing library text, yest Acts and we test component by component, but the problem with component by Component is that if you have a component, I receive an alternate test: For an image as a prop and the prop is not passed, you would need to have Several tests for this with the application you just know what is what Is there unless is obviously coming from database or an API The other issue for us is that we are a service site application based, so we use Next J’s, sometimes we have data render in the server that are not coming in the Client, so sometimes the data is not coming, so we sometimes we have Components that we cannot test, unfortunately, his second question in the chat was: do You ever find their violations that fail the test, suite that you need to write.

That you want to skip or ignore, and is that possible on a one-off basis or do You always need to make code changes to address accessibility, test failures and Then, following that up with, how do you account for testing the rendered app in Different states: okay, for the first question: yes, there’s violations that Fail the test suite that we need to ignore sometimes pick it again. There’s Something that I always say to my team, and not only the team that I that I Developed with, but product team in general is that we need to be the most Accessible as we can it’s very difficult to be hundred percent accessible, Unfortunately, but I can tell you an example because it happened today, we Have an image gallery and some of the thumbnails are not images – are our Background images, the good thing is that all the thumbnails are in the gallery, so All the images have their correct alternate test.

So if I look for images This thumbnail will not appear in my test, so it’s not that they failed. It is Just literally, they are not found so, for me is a failure, but I just keep it Because in favor of the other images, and sometimes we have to ignore them – the Better example for us is as our logo, the colors of our logo are not three pelican Compliant so it’s like our our corporate log, our corporate colors, it’s just blue And white: it’s something that I cannot change, because the board will kill me if I change the logo.

This is something that we need to ignore. How do you account for Testing the render up in different states mocking we normally model data That we pass to the test, so we mark some 500 errors. We mark some success. We mark Errors – and we test as much we can – is obvious that sometimes we miss Some stuff, but this is, I guess, that’s obvious and Kari asked in the chat if what you Thought of and I’m going to destroy this last name man, not you this article on Building the most inaccessible site possible with the perfect lighthouse Score, did you read that yeah absolutely absolutely, of course, actually actually He lives in Vienna and we are going to.

We are in the process of found in the Accessibility cloth, Vienna together so yeah, absolutely I mean again know all the all the libraries, the tools that we have seen. They are not perfect manual show us perfectly how to how to score 100 % with The list with with the worst website possible but yeah if you find the hole The best thing they can do or the best thing you can do – maybe it’s reported and Try to make this tool better, so yeah we need.

We need to use the tools knowing That it could happen and and again, as I said in my slide ultimate test, it’s just One part of the whole testing process, so manual test is very important as well. And I think manual said it in in a talk and Sergei as well the best testing tool That you can do is just is close your eyes and try to use your website with The screen reader, if you’re, able to perform an end-to-end action, you can log In you can purchase you can whatever Pro you’re selling or service, then then can You good yeah, there’s definitely limitations to automated testing, so this Is like a really broad question, but do you find that there’s one tool, that’s Better than another for finding certain issues, not not better but different, Situations so, for example, I use react, acts in the Browser instead of the acts, chrome, chrome, plugin, because I normally have The console log open, so for me it’s not that one is which is totally wrong.

Both Are developed by the same company, but one is better in one situation for Example, just I use testing in the terminal instead of using the CL eyes or The CI, so I prefer to write my own tests than to run PAE them in Y or or acts Directly in the terminal both are going to most likely most likely are going to give Me the same result but yeah I can run tests continuously and and the other one Is their one-time thing so yeah, it’s not it’s again, not not better than the other One, but with their situations Carrie asked well, she stated we captured A lot of the links, but do you have any other resources you want to recommend to The group for yes definitely, I mean a linear project.

Carrie, your your blog literally Scott ohhara Twitter account any any Accessibility guru, let’s say in Twitter, follow them they will. They are Continuously, sharing blog, post and and and tools to use in our development. Process you mentioned manual mitosis, which the His blog is a huge inspiration as well. He has a very cool section, which is Today I learned field section and every day is he posted he posts and heals Anything so my biggest resource is Twitter, follow probably the hashtag 11y And you will find would find a lot yeah.

It’s it’s social Media is really great for for targeting what you want to learn. The hashtag does Anybody else on the call any questions or discussion points for Adrienne Opening it up for people, no, nothing, okay, so I wanted to ask you something: About your music, playlist, Adrienne, okay, you said you liked jazz, but like what Kind of jazz, like comes you up when you’re doing the accessibility testing Like what like really gets you going like what really helps you get into that Groove, because sometimes you know I hate to say this – the testing can be a little Bit arduous and tedious, you know, so what do you recommend and I I just know, not a Specific art disorder – or I mean probably it’s more into the funk part, so Maceo Parker is my probably my favorite, Frank artist, but any just music is very Relaxing to tour with jazz doesn’t funk yeah.

I am not quite there in the jazz World you know I like the funky side of jazz, but some of the other stuff I Either are not cultured enough or I haven’t been exposed to the right. Artists to really have a deep sense of you know, love for the genre yet, but That’s okay: I can get there. Okay, I just want to give everyone else. One more Opportunity for questions or discussion points there’s a big group today, so I’m Just hoping someone has something to say it might be.

My Yeah, go on under you’re welcome to turn your microphone on to Andrew. If you want To do it that way, yeah! I can do that. Can you hear me yeah, okay, great yeah? This is Easier, so I’ve got lots of test suite questions, I’m in the middle of trying to Set up something for my team as full feature is this so when your tests are Failing that you choose to ignore, is that mean your test? Suite is literally Family, because a lot of test suites are gatekeepers for the CI to build.

So how Do you, how do you deal with that or how does that work? Yes, absolutely There’s this is something that you need to talk with your team of course, and so In our team again, this is what you just said: Our tests need to get to pass to continue building, so we have every pull Request that we make to the code, it has a test and the build breaks. We need to Look where is this the the wait break, what he braked – Oh buddy, Proctor, so Sometimes you need to analyze this this test.

We normally have our UX designer Sitting with us – and we need to talk – is this really a usability test? Failure is It an accessibility test failure is the accessibility. How is the level of Severity, so is it a warning, is in an error, is a huge error. Like okay, you have Links with with no ATF, you have images with no alt text. Okay, you need to fix That again, do you have color contrast that is not compliant with a triple-a or Not even double-a, okay, then we need to evaluate it and, if is if we need to in If we decided to ignore it either you delete the test.

Or either you make it past the test I mean, how can I say it like you: can Reduce the severity your tolerance today today to the test. So if you’re looking For errors in the industry, in in this specific test, you’re looking for Warnings so the error will go or the other around the warning will go: okay, That’s really helpful, so you’re you’re not letting a your test suite fail to be Green, you know indefinitely because that’s one Thing that’s important is that we trust the test suite.

So definitely I mean so In making a test with expect through and and passing to is just delete the test Makes no sense. Okay thanks. I really appreciate that good anybody else. Do you have anything else? You want to say Adrienne, one thing is just we need to understand that the Responsibility of web accessibility is not on the developers it has to be in The whole team so from product managers, product owners, you x-ray designers, UI Designers developers testers everyone, it’s not.

The developers needs to think Of everything, and if it’s not accessible, is part of the developments or the Developers fault, no. This is something that, if you, if your product owner Requests a new feature and if the feature is a button who perform an Action then it has to be a button and if it’s a link who goes to a place, then it Needs to be linked, ok, button against link. We talk about a different day, but But we have so HTML provides us with the correct tools.

We don’t need to download. Any polyfill we don’t need to download any library is everything. Is there to be Used so one is use with what you have correctly and again, it’s responsibility. Of everyone, so to argue, do not fight if it’s possible with your designer and say No, I’m not going to delete the outline of our focus only and an argue it Correctly, I will delete the outline only if we put something else or this this Button I want a button that looked like an icon and in small screens look way.

Way less this if it’s possible, because if it’s not possible, let’s look for a Different solution that that the UX, the user experience is not effective, so it’s Something that is is easy, because it is really really easy, because Aria, give you all the opportunity you have so you just need to do it and again Involve everyone in the process, so if you could involve the whole team, but I Prefer it for for your process, because if you have an accessible product, then Is a better product not again the same as they my last sentence, not for Disabled people, for everyone right – those are great words – well enjoy your time in Columbia, I hope you really enjoy the countryside, along with the conference.

And congratulations on your engagement. That’s really exciting! Thank you thanks for coming on today, we really appreciate everyone: who’s, not Thank you for having thank you for having me

---