Edinburgh festivals diary: Booker Prize author's Gaelic AI fail showed investment needed for deaf audiences

Covering politics over Covid, the transcription software was great for noting down the basics of then-first minister Nicola Sturgeon’s daily pandemic press conference. Working from my kitchen table, I could let the interview play out of the speakers and record it to transcribe - letting me skip easily to the bits I wanted to quote - and listening back to them again for accuracy.

Of course, I couldn’t rely on it. Five years ago, the tech was in its infancy and Sturgeon’s accent did not translate well. I found if I was transcribing an interview with an east coast American, or someone who spoke in received pronunciation, all was fairly good: otherwise, forget it. Since then, it has improved and the tool can pick up all, but the strongest of regional accents fairly accurately. Of course, I would still never rely on it for direct quotes.

But in an Edinburgh International Book Festival event with author James Kelman this week, I was reminded of the dark Covid days of AI transcription when live captions popped up on the big screen, alongside the use of a British Sign Language interpreter.

In the text beamed onto the big screen, Kelman’s phrase “imperious bastards” was captioned as “peel his bastards”, as the AI tool struggled with his Glasgow accent, while he word “Gaelic” came up repeatedly as “garlic”.

“I listened agog” was translated into the curious name - which will hopefully pop up in one of Kelman’s next novels - of “Alison Dugog” and “made your toes curl” was captioned as “Major Toes Curl”.

Another Fringe show I attended at the weekend also had captions. These, I believed not to be live as the show was scripted - and deviated slightly from the live speech as the performers changed word order slightly as they spoke. Instead, someone on the tech team was hitting the button on each pre-loaded line as they spoke.

However, the problem was not the deviation from exact words the performers were saying, but the fact the captions had not been proofread and were littered with (albeit small) grammatical and spelling errors - two weeks into the show.

While the use of captions has to be welcomed - Deaf Action says they are “essential” for deaf people to access arts events such as the Edinburgh festivals - the standard desperately needs to be improved.

The book festival says it has increased captioning provision this year, using AI to complement the in-person captioning it has previously carried out, in a bid to ensure it has coverage across all of its events. I may well have hit a bad night, but​ that would not have helped any deaf people in the audience.

“Inaccurate, lagging or poorly proofread captions can exclude people just as surely as having none,” the charity’s chief executive, Philip Gerrard tells me, adding that skilled live captioners are scarce and need to be booked well in advance.

“What’s needed now is systemic investment and planning: grow the workforce through training; involve specialist organisations like Deaf Action early to choose the right access for each event; improve technical set-ups, rehearsal and oversight; and clearly label what audiences can expect, (live human, scripted or AI) with basic quality assurance in place,” says Mr Gerrard.

All of the festivals need to step up and take note for next year.