We’ve been asking ourselves whether accessibility checks on code can become part of the pipeline and stop a merge request, a proposed change, before it’s merged with the rest. To find out, we built a proof of concept (PoC) with Playwright and axe-core. It’s the subject of one of our talks at DrupalCamp Italy 2026. At the end you’ll find seven questions to help you work out where to start on your own portal.
By the bmeme team
On our projects we check accessibility during design and development, with automated tools and manual checks. Once the pages have been generated, in test or in production, we check them in bulk with MAUVE++, CNR-ISTI’s validator, which the lab behind it has given us access to for our projects. CNR-ISTI is an information science and technology institute of CNR, Italy’s National Research Council. What we get is a snapshot of the portal at that moment, and it goes out of date with every release, because the code keeps changing in the meantime.
All it takes is a change that removes a button’s name or reduces a link to an icon. The problem reaches the pages, and the check only finds it afterwards, once the code has been merged and is sometimes already live.
In security, automated checks are already part of the pipeline, so we asked ourselves whether the same could be done for accessibility, agreeing the rules in advance instead of debating them at every release. We call this idea Accessibility-as-Code, knowing that for now it’s more a hypothesis than an established practice. We built the PoC to put it to the test, and in the sections that follow we explain how we built it, how the pipeline works and where it stops
The accessibility of digital services is regulated by law, with different rules for public sector bodies and for private companies that offer certain services to consumers. Below is a short summary, for orientation only and not legal advice. In both cases the obligation lies with whoever owns the service. A pipeline like the one in the PoC doesn’t shift it onto the developers, but it can give the person accountable for accessibility technical evidence to support their work. In a public body, that person is the Responsabile per la Transizione Digitale, the official in charge of digital transition.
Law 4/2004 applies, known as the Legge Stanca: Italy’s digital accessibility law, which requires an accessibility statement to be published every year. AgID (Agenzia per l’Italia Digitale), the Italian government’s digital agency, promotes its application and supervises services.
For services such as e-commerce, banking, passenger transport and electronic communications, the European Accessibility Act applies. It’s Directive (EU) 2019/882, which has applied since 28 June 2025. Here too, AgID is the supervisory authority.
The PoC runs on a basic Drupal 11 install, with the standard profile and the Olivero theme. On it we built the portal of an openly fictitious public body, the ‘Portale Servizi al Cittadino’ (Citizen Services Portal), with a login, a logged-in area and a form that ends with a confirmation dialog, to simulate a public service. The site installs from code, with synthetic content and no database dump, so every pipeline starts from exactly the same point.
We’re testing two tools, which work in sequence. Playwright opens the browser, logs in to the restricted area and brings the page to the state to be checked. Then axe-core analyses it and returns the violations, each with a severity level: critical, serious, moderate or minor. axe-core, which we use only in the PoC, does a different job from MAUVE++. It works on the code of a merge request before it’s merged, whereas MAUVE++ takes a snapshot of pages that have already been generated.
The repository contains three merge requests, each with its own pipeline that actually ran. In the first, the form’s confirmation dialog has a close button with no accessible name. The pipeline blocks it, and in the next commit adding a label to the button is enough to unblock it. In the second, the dialog’s heading drops from h2 to h4 and the pipeline lets the change through with a warning. In the third, an urgent fix reduces the ‘Paga online’ (Pay online) link to an icon alone. The pipeline blocks it, and the merge request can only be unblocked with the override procedure we describe further down.
Inside CI, the job starts DDEV, installs the site and uses Playwright to run the spec that visits the pages, saving axe-core’s report for each one in a JSON file. Its only task is to measure: the decisions are made by the next job.
Before the tests, a setup project runs just once. It opens the login page, fills in the username and password and presses the login button. The setup only treats the login as successful when the log-out link appears, then saves the session to a file that all the other tests reuse without logging in again. Credentials stay out of the repository. Locally they live in an unversioned file, while in CI they come from the project’s variables.
A Node.js script reads the reports and applies three rules written in code: the Severity Budget, the known-debt register and the Break-Glass. The measurement is the set of axe-core reports produced by the tests in the previous job; the decision is what the gate concludes from them by applying the rules. Keeping the two in separate jobs means the rules can change without touching the tests.
The gate exits with 0 if there’s nothing to report, 1 if it blocks, 3 if it tolerates and 4 if there’s an override. GitLab shows 3 and 4 as ‘passed with warnings’, so the merge request stays unblocked while the warning remains in the log.
The pipeline isn’t optimised and takes several minutes, almost all of them spent building the image with the browsers. That’s fine, because a PoC is there to test the method, not to be installed in production.
A test can pass even when the defect is there. In the repository, e2e/tests/naive.spec.ts shows how: it’s a naive spec that only runs locally. It opens the service request page and runs axe-core’s analysis straight away, before the confirmation dialog is in the DOM. axe-core doesn’t see the close button with no name, and the test comes out green. That’s what we call a false green.
The spec the pipeline uses, e2e/tests/ci-audit.spec.ts, first presses the ‘Invia richiesta’ (Send request) button, then waits for the dialog with
await expect(page.getByRole('dialog')).toBeVisible()before running the analysis. That’s the line the naive spec is missing, and it’s what makes it possible to find the defect and block the merge request.
The same principle applies to the login in e2e/tests/auth.setup.ts, which only counts as successful when
expect(page.getByRole('link', { name: 'Log out' })).toBeVisible()finds the log-out link. The test grows along with the feature and writing it well is part of the job, because a green pipeline is only as good as the tests behind it.
The Severity Budget is the set of thresholds that decide, for each severity level, whether a change goes through, goes through with a warning or is blocked. The thresholds are written in code and versioned, so they apply to everyone and every change to them is tracked. They aren’t set by developers on their own, but by whoever is accountable for accessibility, together with the team.
A critical or serious violation stops the merge request. The same happens when axe-core’s reports are missing or malformed, because then the gate has nothing to base a decision on.
If only moderate or minor violations are left, the pipeline passes with a warning, as happens in the PoC when a heading drops from h2 to h4.
A blocking violation can only be overridden with the Break-Glass, the emergency procedure we describe further down.
A defect that’s already known and registered stays in the log but doesn’t count against the budget, as long as its entry is still valid.
Not everything the pipeline finds is down to the team. One example is the defect axe-core reports on the login page: it comes from Drupal core’s Olivero theme and has already been reported on drupal.org. Blocking every release because of it would make no sense, but neither would ignoring it.
That’s why the repository has a versioned register, the a11y-known-issues.json file. Each entry states the rule, the page and the element, where the defect comes from, where it was reported upstream, who is responsible for it, why it’s there and by when it must be reviewed. Here is the entry for the login page defect.
[
{
"rule": "region",
"url": "/user/login",
"target": "#primary-tabs-title",
"origin": "Drupal core (Olivero)",
"upstream": "https://drupal.org/i/3318396",
"owner": "bmeme",
"reason": "Titolo nascosto delle schede primarie fuori da un landmark; correzione attesa in core",
"review_by": "2026-12-31"
}
]While the entry is valid, the violation appears in the log as known debt without counting against the budget. Once the review date has passed it counts again. If the defect disappears, the gate warns that the entry is obsolete, and if the register is badly written it blocks the pipeline, because a broken register is a governance error. New debt gets blocked; known debt gets managed.
The limitation is that anyone who can change the branch can also add entries. But additions show up in the merge request’s diff, which is the place to review them.
The Break-Glass is the emergency procedure for getting past a block when something is genuinely urgent. You start a pipeline on the branch by hand and fill in three fields: the override request, the reason and the fix ticket.
GitLab rejects a ticket in the wrong format straight away. Then the gate checks that the reason is at least ten characters long and that the ticket is valid. If the request is incomplete the pipeline is blocked, even when there would be no blocking violations.
If the request is valid, the gate exits with 4 and the merge request is unblocked without changing the code. The pipeline artifacts keep the Audit Trail, a record that doesn’t expire, of who overrode the block, when, why, with which ticket, on which commit and for which violations, together with the axe-core reports used to decide. It counts as supporting technical evidence, not as legal proof.
In the PoC, anyone who can start a pipeline can trigger the Break-Glass. So the procedure discourages and records, but it doesn’t decide who may override: that’s for the organisation’s own process. To strengthen it, GitLab offers protected branches even on the Free tier, while required approvals and Code Owners, which ask for a second person before merging, come with the Premium and Ultimate tiers. We’ve read about these in the documentation but haven’t tried them, because the PoC uses the Free tier.
Automated tests find only some accessibility problems, so manual audits and testing with disabled people are still needed. The pipeline takes the repetitive part out of the way, leaving more time for those.
On some contrasts in the Olivero theme, axe-core returns ‘incomplete’, because the gradient background doesn’t let it decide. In those cases the gate doesn’t count them and a human check is needed.
Keyboard behaviour inside a dialog, the so-called focus trap, has no rule in axe-core and has to be checked by hand.
The content the editorial team uploads every day, such as text, images and documents, stays outside the code pipeline, because developers can’t fix it. It needs dedicated work with its own tools, while attached PDFs are out of scope.
The pipeline stops the regressions it can recognise and produces technical evidence, but establishing whether a service conforms requires a full evaluation, which remains a separate piece of work.
The PoC is there to test the method, which we’re now bringing into our own production work, a more complex job than a PoC.
The repository is public on GitLab, with the merge requests for the cases described here and a README with instructions for repeating the tests.
Answer all the questions to see your result.