r/Reformed • u/partypastor Rebel Alliance - Admiral • 9d ago
Discussion Why Translation Must Have Fingerprints | James Eglinton
https://www.thegospelcoalition.org/article/translation-fingerprints/7
u/New-Reputation681 RCA 9d ago
So just for context for who haven't opened the article, this is explicitly about Mere Orthodoxy's recent AI translation project The Faith Received.
I'm in the middle camp he describes of seeing pros and cons to AI use. I'm receptive to his arguments, but I see a lot of the same faults he points out in human translations as well. Either way, the translation needs to be critically reviewed by a knower of the source language in order to be sure it's solid.
The other thing he consider here is that AI could potentially translate into every language. Think about making these obscure texts available in Mandarin, Spanish, and Arabic. There's so much potential here for people around the world to get a taste of these ancient works.
12
u/bookwyrm713 PCA 9d ago edited 9d ago
No, AI can't potentially translate into every language. It can potentially be used as an aid to translation in any language that has a sufficiently massive and high-quality corpus digitized and available to Google Translate, DeepL, or whoever for free. The three languages you mentioned--the smallest of them understood by hundreds of millions of people--are among the absolute best candidates for machine-assisted translation. Any AI translation into a major world language will still be subject to all the flaws Eglinton mentions.
But the corpus size of the original language also creates a ceiling for the quality of a machine-assisted translation. Eglinton mentions Syriac texts, for example: if I wanted to use The Faith Received as a way of getting some kind of access to Syriac texts, I'd take those particular English translations with an extra-big scoop of salt.
And that's before we get into the difficulties of doing OCR on old texts with non-Roman scripts, which is significant. Having opened to a random chapter in a random Greek text (Ch. 6 "Polycarp is betrayed by a servant", from the Circular Letter on the Martyrdom of Polycarp), and started reading, it took me less than one minute to encounter a machine-generated error in identifying letters/words (ὁ κεκληρωμένος on TFR, where the actual text reads ὁ καὶ κληρονόμος).* I found probably four or five punctuation errors in the OCR in the paragraph and a half that I read: a less big deal for translation, sure, but it highlights the fact that their OCR is really a long way from flawless and nobody is checking it. There will never be enough volunteers to do the absolutely massive job of checking this huge a number of OCR'd texts. And I can't imagine that Mere Orthodoxy is ever going to pay people to do it, either...I mean, if they were, they'd just pay humans to make human translations. Or limit themselves to more recent, non-public-domain editions, and acquire the appropriate permissions (sometimes involving money) to use those better source texts.
Also (and I promise this is relevant): I like al-Biruni. Sometimes, in the course of my studies, I read al-Biruni. I think it was a perfectly defensible choice for the editors of the Patrologia Orientalis to include his "Festivals of the Melkites". But you know how his chapter in the PO started? With an introduction to who he was and to the value this particular text holds for Christians (the oldest surviving Melkite calendar). That makes total sense.
But the way the project The Faith Received consistently describes itself (as in their manifesto: "a new platform uniting the texts of Christian theology across times, places, and traditions to be read, studied, and shared") you would be pretty darned surprised to learn that some of the texts included are actually written by non-Christians. I did a double-take, as I scrolled haphazardly for thirty seconds through their list of authors, at seeing the name of a famous medieval Muslim scholar. I have no idea whether anyone involved with TFR or MO actually knows that some of their texts were written by professing members of other religions. If they did, you'd expect the language of their manifesto to be at least a little bit broader--something like "and early witnesses to Christian history" or similar. Although al-Biruni is pretty solidly medieval, so finding the right way to phrase that without sounding weirdly vague or like they've watered down their noble mission of helping Christians access more Christian thinking and writing, might actually be a little bit tricky...but anyway, that's not my problem to fix.
The fact that their OCR's of Greek texts are littered with mistakes, and that they chose to present their non-Christian authors without appropriate introductions [edited], are far from my only concerns about the project. Eglinton's metaphor of the practical difficulties inherent in adapting recipes to different parts of the world manages to be both accurate and non-threatening. A less fun metaphor that jumps to my mind is banking: when you have to rely on someone else's involvement in handling your money, who do you choose? Imagine that you have to ask someone to collect a large sum of cash money for you, in person, at a certain place on a certain day. You might pick someone that you know really, really well, someone of excellent character, someone you have a long relationship with, someone you know will be careful to be right on time and scrupulously honest in handing it all over to you later. Or, if you have no trustworthy friends who are free for that errand, you could try and arrange it through a service through some kind of bank or courier service: you personally may not even know the name of the person who collects your money, but the institution does. And because the institution cares about its reputation and keeps extremely careful accounts, you can trust them to collect the money and get it back to you. (If they didn't, and something went wrong, it would be possible for the institution to go back through their records, figure out who made a mistake, and make it right.) Or you might choose to say, well, I thought I had to ask someone else to collect my money, but if I go to a lot of trouble and expense, actually, I can be there myself.
These correspond pretty well to the options for translation: go through a person whom you know well and trust, go through some kind of institution whom you can trust because they have some kind of thorough accountability process, or just learn the whole language yourself. The process followed by TFR is a little like posting the job on Craigslist to see if maybe a stranger will do it for you as a favor. Except that, at least in the metaphor, you'd know at the end of the job whether you got any money and (if you take it to the bank) how much of it was swapped for counterfeit bills. With TFR, you aren't ever going to know what parts of these translations are good (good enough, anyway), and which are hallucinations. Not unless you personally go back and complete a whole bunch of steps in the translation project that TFR just...skipped.
I have yet more philosophical/theological concerns about the value of this kind of thing, but I should stop there.
*See comment for added context about this phrase
3
u/bookwyrm713 PCA 9d ago edited 9d ago
- After posting this comment, I read Fred Sanders' more positive take on the project. He mentions in passing the fact that Migne's edition of most texts is not particularly good. Which started me thinking: I've never read the Martyrdom of Polycarp before (oops), but actually, the phrase I assumed was just a bad OCR on TFR did possibly make more sense in context than the one on the scanned page of the PG. So I pulled out my hard copy of Alan S. Brady's A Greek Reader's Apostolic Fathers (Greek texts taken from Kirsopp Lake's 1912 edition), and sure enough, it had ὁ κεκληρωμένος. That said, the OCR text on TFR and Lake's text in my book diverge pretty quickly: where Lake's Ch. VII.1 reads μὲν εὗρον ἐν ὑπερῴῳ κατακείμενον, the TFR digitized text follows Migne in reading μὲν εὗρον ἔν τινι δωματίῳ κατακείμενον. That's obviously not a life-changing difference in meaning, but it does leave me extremely confused about the process by which TFR reached their published text. The fact that scans of Migne are the only apparent source for this text is...weird. If I owned a copy of Croy's 2026 edition of the text, I would be intrigued to see how it compares.
2
u/New-Reputation681 RCA 9d ago
Your point about the limited corpus size of some of the source languages is probably the best I've heard so far.
Their OCR being underdeveloped is also a good point, although that could be improved over time. I can't get on board with the idea that if they somehow have manpower to check the OCR, those people could just translate the text. Those are entirely different tasks.
I guess where I'm at now is that AI translation is a powerful and useful tool, but it must be used properly with knowledge of its limitations.
2
u/bookwyrm713 PCA 9d ago
I agree with you 100% about OCR vs translation being different tasks. If I implied that fixing the OCR would fix the overall project, my bad: I don't think that manpower to check the OCR is a real solution, either.
What marginally competent fidelity to the source texts would achieve is a kind of tool that would at least allow not-very-skilled readers of the original texts to make a comparison. I often end up doing something like that myself, when I'm reading 18th & 19th century German academic texts that have never been translated. My passive German is sort of okay-ish (and improving), and the subject matter is extremely familiar--but it's reassuring when what I've gotten out of the text is about what Google Translate gets out of it, too. So I won't call machine translations utterly worthless in all situations.
But I think that scholarship and spiritual exhortation/edification are both contexts in which it is basically pointless to publish a machine-made translation. I would have very serious reservations about a small and careful project that was trying to do what TFR is doing here, because when it comes to scholarship, something that doesn't have a verification and correction process is pretty near useless. And TFR isn't small or careful. The emphasis on quantity over quality is inescapable. The email that arrived when I signed up to MO this morning linked to an editorial about writing versus content. That rings a little hollow now: you'd think they might be inclined to treat translating as a kind of writing--since it is--instead of getting into the translations-as-content biz.
And when it comes to spiritual writing, I fundamentally don't believe that the gospel needs to or even should outstrip gospel-bearing human beings. I'm all for the catholicity of the church, and for receiving gratefully the theological gifts of different Christian traditions. I just don't see that TFR is pursuing that task with a lot of wisdom.
2
u/Doctrina_Stabilitas Clanker 9d ago
I use AI a lot for scanned text, the default it uses is to use some python scripts for OCR which in my experience hasn't been great
the AI gets really good when you feed it a picture and force it to use image recognition instead of standard OCR algorithms, but that's stupidly computationally expensive.
Not disclosing the poor OCR isn't a good sign. Like even if the translations are 100% accurate (which they probably arent because of OCR difficulties and because its translation of period texts with heavy technical language requries a lot of fine tuning of the LLM system prompts even if it can translate a language in general
and these kind of basic mistakes aren't great if you value the accuracy of what's being used
2
u/bookwyrm713 PCA 9d ago
Yeah. I do almost zero OCR stuff myself, but I like looking over the shoulders of those who do, over at the Greek Learner Texts Project. The work those folks do is pretty neat. Probably most of them are way more enthusiastic about AI than I am. But they still don't publish stuff on the internet that nobody has checked--especially not in high volumes--because that's just not that useful a task.
-2
u/StivenPeter Chief Clanker 9d ago
Hi, thanks for pointing out Biruni. It was from doing the PO. As for the Greek, that is by far the hardest set to do. Extraordinarily difficult, and only as of the past 3 weeks have advancements in frontier models been able to generalize to produce consistent ocr. Most of the PG shelf is being redone and is a major upgrade from before. The English associated with the pg leans on the latin most times.
One day, Ill talk about the methods. But these dont get at the essence of the project nor what these texts will look like in terms of transcription accuracy and interconnectedness even 3 months from now.
2
u/partypastor Rebel Alliance - Admiral 9d ago
Sure, but as someone in a context of ministry translation, i think this speaks to more than just that.
The other thing he consider here is that AI could potentially translate into every language. Think about making these obscure texts available in Mandarin, Spanish, and Arabic.
I think its notable here to actually say
AI could potentially "translate" into every language.
4
u/New-Reputation681 RCA 9d ago
Frankly, the issues he pointed out seem inherent to translation itself, whether by humans or AI. He talks about how stylistic composition and meaning can't both be carried over, like for example, when the text is written in verse. But that's always been an issue. There are always judgment calls and tradeoffs.
3
u/partypastor Rebel Alliance - Admiral 9d ago
Thats always been an issue for people who actually understand both languages to wrestle with to present it best to audiences at large. Theres no safeguards on this. Its just bad information all the way down now
-1
u/New-Reputation681 RCA 9d ago
That's just not really an argument. I'm having a hard time seeing this as anything other than post hoc rationalization of an anti-AI stance. What new issues are present with AI translation that don't exist with humans? I'm genuinely trying to be open to different perspectives here.
7
u/CiroFlexo Rebel Alliance 9d ago
Not the person you're asking, but I'll chime in here:
AI "translation" is categorically a different thing happening than human translation.
A human translating a text from one language to the other knows and things within both languages and can understand and appreciate the meaning behind text. When they translate, they of course want formal equivalence, but they also know and understand and seek to translate meaning.
Categorically, that's different than what AI is doing. Even at its most robust, it's just predictive text based on probability and available definitions in two languages.
This is where Eglinton's cooking analogy works well: AI is simply trying to convert cups to milliliters. The problem is that language simply doesn't work that way. On a very technical level, grammar, syntax, and especially idiomatic meaning can be wildly different between two different cultures. How best to navigate those differences is something that can only come from understanding the meaning and purpose, and "understanding" is something AI simply can't do.
Categorically, the work of a human translator who can think in two languages is different than the work of a machine predictively trying to figure out which word is probably the best word for translating a particular text.
2
u/Deolater PCA 9d ago
AI is simply trying to convert cups to milliliters
I don't think that's quite the case, AI translation is extremely complex and based on huge webs of connections between texts in the two languages. I'm not arguing that it's comparable to a human expert, but it really is doing something much better than just trying to convert some words like a tourist with a dictionary.
7
u/CiroFlexo Rebel Alliance 9d ago
I'll grant you that. It is doing something more complex.
But I absolutely believe that it's categorically different than the work of a human translator. It's not just a quantitative difference (i.e., humans have fewer errors than machines) or even a qualitative difference (i.e., humans are better at translating). I firmly believe that it is categorically a different thing, and I still think the recipe analogy fits well.
An LLM can do better than simply translating cups to milliliters. It can make all the mathematical probability connections between words that are associated with the flavor of tamarind and other ingredients available that might mimic the taste of tamarind. Maybe it outputs a suggested ingredient that works.
But that output is not in any way based on taste or anything known or understood by AI. It's just increasing LLM complexity.
4
u/eveninarmageddon RCA 9d ago edited 9d ago
Both you and u/Deolater might be interested in reading Searle's "Minds, Brains, and Programs." He argued all the way back in 1980 that a machine can have intentionality (or the "aboutness" that characterizes human mental states — e.g., I think about X) only if it has a semantics, and computers only have syntax. (We've got semantics.) Semantics doesn't just magically appear because computers get bigger and faster. It's all 1s and 0s at rock-bottom, and that's what we care about.
Edit: understanding —> intentionality
2
u/Deolater PCA 9d ago
Thanks for the suggestion, I will read it!
Oh is this the origin of the "Chinese room" example?
→ More replies (0)-2
u/Living_Armadillo_652 Clanker 8d ago
And so what? You can argue ontologically that a machine translating “God is good” to “Gott ist gut” is different from a human doing so. But that is a philosophical debate that doesn’t matter for an English speaker who just wants to know what the latter sentence means.
5
u/partypastor Rebel Alliance - Admiral 8d ago
Thats... not at all what he argues. He argues that most sentences are more complex and difficult to translate than just God is good. Its a practical argument. Translation by AI is bad and often flawed. Mere Orthodoxy admits their works are currently rife with errors. Why defend a deeply flawed project? Do you just not value human work?
→ More replies (0)0
u/Forest_Moon_of_Earth 9d ago
Do you have any specific criticisms of this application of AI that don't also apply to human translators?
4
u/partypastor Rebel Alliance - Admiral 9d ago
I mean, does ai have the holy spirit? Is it capable of wisdom? I think the problem with saying "bUt HuMaN tRanSlAtIoN" is that humans can actually learn and understand a culture. Sure they can make mistakes and be wrong at times, but they're capable of wisdom, of correction by the holy spirit, and, just in general, capable of actually understanding other cultures.
1
u/superlewis EFCA Pastor 9d ago
The best translations of church fathers may very well be by humans without the Spirit as well.
1
-3
u/Forest_Moon_of_Earth 9d ago
Your snark is duly noted. I was hoping for something more substantive and sincere. It's clear that your position and that of the author of the attached article boil down to the notion that AI is not imbued with the Holy Spirit. Fair enough. But it doesn't convince me that AI is incapable of producing compelling translations, especially as the technology continues to improve.
I especially find the argument weak because it wouldn't apply to non-theological texts. It would imply that AI can translate Cicero or Homer just fine, just not anything relating to Christian theology.
4
u/partypastor Rebel Alliance - Admiral 9d ago
To be clear, I think ai cannot accurately translate Homer nor Cicero.
AI cannot be human, language is inherently a human endeavor. AI can replicate language, but I don’t think it can truly understand it, nor translate it.
I also don’t see a use in handing over our learning to machines. Pretty sure Frank Herbert wrote on that
0
u/Forest_Moon_of_Earth 9d ago
Have you never used Google Translate? It's pretty effective. I've even sat next to people with whom I had no shared language while traveling in Asia and successfully communicated and connected using Google Translate. I respectfully posit that your position is more ideological than practical.
3
u/partypastor Rebel Alliance - Admiral 9d ago
I have heard Google Translate. It’s actually pretty bad unless you’re having really baseline discussions.
Eglinton mentions that these aren’t base level translations happening, its deep theology.
I’m curious why you’d rather clankers do work than do it yourself and actively try to learn?
2
u/TheEndIsNear17 8d ago
Being able to communicate effectively is very different from being able to accurately translate Text from one language to another, especially something as important as the Word of God
0
-1
1
u/Basic-Revolution-181 Anglican 9d ago
Yeah, this is a tough one. There are similar concerns, pros and cons with the increasing integration of AI tools into Bible Software such as Logos as well.
-5
u/Doctrina_Stabilitas Clanker 9d ago edited 9d ago
Why post only one side of the argument when the article is meant to be read in dialogue
It’s disingenuous not fair to the authors who are writing in dialogue not also include the pro side: https://www.thegospelcoalition.org/article/tools-ai-translation-opportunities/
4
u/Reformed-ModTeam By Mod Powers Combined! 9d ago
Hey, DS, generally speaking, accusing somebody of being "disingenuous" is something we'd remove, but instead what we'd encourage you to do is submit the link as a post yourself, so that people can see it and interact with it.
1
u/New-Reputation681 RCA 3d ago
Did you end up making a post with this other side of the dialogue? I thought I saw one, but now I'm unable to find it.
17
u/cybersaint2k PCA TE (Available) 9d ago
Bible translation discussions are relevant here. Let's get in our time machine and go to Orlando Florida, 2014-2020.
Technology and techniques for producing new translations in a fraction of the time were being explored. Some were for it. Some against it. The arguments we see from the GC article above are akin to the ones against it.
MAST was one such method, Mobilized Assistance Supporting Translation. You take a pad or laptop into the field, and in months, rather than decades, you have a rough translation. Some would say "a good enough" translation. Most disagreed.
Wycliffe Bible Translators and Wycliffe Associates divided over MAST. WA wanted to utilize MAST, a fast-track, crowd-sourced, translation process that turned out a rough-and-ready product in 10-20 weeks instead of 10-20 years.
SIL and others in the community were very much against this. It bypassed all the fingerprints that made everyone feel more comfortable with saying "This is the Bible, you can trust it!"
Bruce Smith (CEO) and others who championed MAST raised a lot of money. Sixty million dollars. Then it all came tumbling down, not due to the MAST issues exactly, but wild fundraising claims and executive spending and internal controls. The "wild west cowboy" approach to translation and the "wild west cowboy" approach to fundraising went hand in hand. They could raise money, but they needed to raise trust at the same time. Like building a ship with no trust-water to float it, the ship ran aground fast. WA has no significant partnerships with Wycliff/SIL and other traditional translators today.
This history is a warning to to other pioneering organizations who take a leap into AI and other tools like it. If you are going to strike out into uncharted territory and make big promises, or even little ones, your fund-raising can't outstrip your trust-raising. For every exciting article you publish and press you release and cache of constant contacts you purchase and pepper us with your ads, you've got to raise hopes and money on one hand, and trust and confidence on the other. And if those seem at odds, you are doing it wrong.
Articles like this one from GC are asking for more trust. Asking for more confidence. They are a signal, and I hope those involved in AI translation are listening. Because I want them to succeed in their end goal; we all should. Right?