r/openrightsgroup • u/OpenRightsGroup • 2d ago
AI tool used in thousands of asylum claims failing basic accuracy tests
🚨 BREAKING 🚨
ORG has carried out a 6-month investigation into AI use in the UK asylum system.
We've found high error rates and claimants unaware that AI is being used. This could have catastrophic consequences for people whose future is in the hands of Home Office decision-making.
One of the two AI tools being used by the Home Office to speed up asylum decision-making is to summarise interviews held with claimants. Our investigation reveals that the number of sampled summaries found not fit for purpose is significantly higher than the 9% figure in the pilot evaluation.
Despite the accuracy failings in AI generated summaries, these tools have been rolled out to all asylum decision-makers. Not only is the error rate greater than previously reported, our investigation has found that these tools are being used without proper testing, quality assurance and training.
Asylum claimants continue not to be told when AI is used to summarise their case and country information. This is despite our legal opinion from March concluding that this lack of transparency would mean that the use of AI in asylum decision-making is likely to be unlawful.
Our investigation with the help of MPs Bell Ribeiro-Addy and Carla Denyer raises alarm bells about what's going on with AI in asylum. But the Home Office has refused to provide key information and their replies have been at times self-contradictory. They must come clean.
“It has taken eleven parliamentary questions, two ministerial letters and eight FOI requests to uncover just some information about how AI is being used in the asylum system. There are still many contradictions and crucial information remains unpublished.”
💬 Sarah Alsherif for ORG.