How to judge translation quality

When you send out a document to be translated, how can you know that the translation that comes back to you is a good one?

Unfortunately, this is a bit of a stumper, as it is for so many other services whose relative quality you may be unable to judge. How do you know you have picked the best dentist? The best doctor? The best teacher? A bestselling book on how to market one's professional services carries this telling title: "Selling the Invisible." In a nutshell, it says that clients who cannot judge the quality of a service will judge its quality by other factors, such as by the quality of customer service they receive. Does the doctor really listen to you? Is the dental office staff friendly? Does the teacher return your calls? These may be entirely unrelated to the quality of the professional service, but clients with no other knowledge on which to base their judgments will assume that someone who is professional and attentive towards them must be just as professional and attentive to the quality of the services he or she provides. This may be "mostly" true, but it is an indirect relationship at best. There are also clear counterexamples, such as those individuals who are absolute geniuses at what they do, but who have no people skills whatsoever.

To produce a high-quality translation requires so much knowledge, art, and skill that the people most able to judge a translation's quality are, no surprise, other translators. Hand me a translation from Spanish to English (or English to Spanish) and the original document, and I can tell you pretty quickly whether it is a good translation, a great translation, or a terrible one. Hand me a translation from a different language pair and, in most cases, I can tell you little to absolutely nothing about how well it has been done. This is a major reason that I work only with Spanish < > English translations and translators; I want to be absolutely sure of the quality of the work my company produces.

Chances are, however, that you are not a translator. How, then, can you judge the quality of the translation you are buying? It is not easy, but it is possible. There are other ways to judge translation quality.

The best way, in my opinion, would be to find a reputable translator to compare the translation to the original. In my experience, the best translators tend to be those who: 1) translate only into their native language, 2) have an extensive educational background (graduate degree or equivalent), 3) are not willing to work for average or below-average rates (the quality of their work is such they do not have to), and 4) demonstrate a commitment to the translation profession by their participation in professional organizations and by having sought, and received, some kind of professional certification.

This reputable translator can judge the quality of a translation for you, showing you where the translation leaves out information that was in the original, adds information, changes the tone or even the meaning(!!), introduces grammar or spelling errors, sounds like Tarzan (a common problem when a translator works into a language other than his or her native language), or in some other way distorts or disfigures the message in your original document. Ninety percent of translations by market-rate or below-market-rate translators will usually commit several of these errors in a single translation.

Even the best translators are human, making occasional mistakes they do not detect. This is why the best translations are the ones that begin with a high-quality translation of the original and are then carefully proofread by the original translator and thoroughly edited by a second reputable translator (in collaboration with the original translator) to polish away any imperfections that may have crept in.

The only reason I can think of to use a different method of judging translation quality would be to save money. However, given the stakes, monetary or otherwise, that are often riding on the quality of a translation, this could prove to be a false economy. If what is at stake for you is important or valuable in any way, then it should be worth a half hour to an hour of a translator's time for you to be sure you are getting it right. Myself, I am glad to offer this kind of translation evaluation service for free, knowing that most translations are poorly done and that, by helping someone see whatever lapses in quality there might be in the current translations, I may gain a new client and, if the current translations are already of good quality, then I have at least performed a public service, not just for that person, but also for the other translator, the translation profession in general and, perhaps most importantly, for everyone who must rely on those translations.

Less perfect ways of judging translation quality are:
1) Asking someone who is not a translator to judge
2) Judging translation quality by the translator's experience, education, and/or certifications

The problem with the first option is that most people will not find someone with the necessary qualifications for the task, but will instead seek out anyone they happen to know who is "bilingual" or a "native speaker." I use quotes around these terms because they can cover a whole range of language abilities. Often, in the U.S. at least, someone who is described as bilingual or as a native speaker either studied (or majored in) the language in college or is a "heritage speaker"—someone who learned the language, while in the U.S., by hearing relatives speak it but, having never studied the language formally, is often unfamiliar with native usage outside of the U.S., or even with proper spelling and grammar! While either could be described as bilingual or a native speaker, neither has, in most cases, anywhere near the background necessary to be a competent judge of translation quality. However, given that some people think that high school Spanish students are an economical way to get free "translations", you could certainly do worse. You could also do much better. If you can not afford to pay a professional, at the very least find someone who was college-educated (ideally to the graduate level) in the country where most of the readers of your translation are from, still lives in or makes frequent visits to that country, is a native speaker of the language of the translation, and is also a highly fluent speaker of the language of the original document.

For an overview of the different levels of translation ability, as defined by the U.S. Government, see: http://www.govtilr.org/AdoptedILRTranslationGuidelines.htm
If you are in a hurry, you can skip down to the "Performance Level" section at the bottom. Most professional (if not reputable) translators whose work I have seen fall into the categories "Limited Performance Level 2" or "Limited Performance Level 2+." The highest level is "Professional Performance Level 5." For a more detailed treatment, read the short preface, too.

The second option is to use the translator's experience, education, and/or certifications as a way to judge translation quality. It must be admitted that these are all rather indirect. Even ATA corporate members themselves, when surveyed (The ATA Chronicle, April 2007), said they rely on other measures (especially their company's own translation tests) more than they rely on ATA certification. It is certainly true that some of the best translators I know are not certified, and some of the certified translators whose work I have seen I would not hire. Experience, education, and certification can not be considered guarantees of translation ability.

If I had to pick, say, an English-to-Japanese translator, how would I find a good one? First, I would go to http://www.atanet.org, click on "Find a Translator or Interpreter," click on "Search ATA's online directory for individual translators and interpreters," and click on "Advanced Search." There, I would choose "English to Japanese," "Show all members," "Undergraduate," native language: "Japanese." This produces a list of 124 translators, so I would begin using additional criteria to narrow the search. Limiting the search further to translators living in Japan produces only 5 results, 2 of which are professional enough to list "English into Japanese" as their only language combination. Choosing, instead, to limit my search to ATA-certified English-to-Japanese translators with graduate degrees produces 21 results, 12 of which are either professional enough to list "English into Japanese" as their only language combination or good enough to also be certified in "Japanese into English." I would read all of these profiles carefully, request translation samples from three of them (experienced translators will be able to produce these readily), hire each of the three to help me evaluate the samples from the other two (so two independent opinions on each sample), and use that information to help me decide which one to hire for my English-to-Japanese translations. End result, I get a far better idea of the quality of work each can produce than I could ever have otherwise, each of them gets paid for an hour or two of work, I get to evaluate how professional and easy to work with each one is and, ideally, one of them gains a new client.

During this whole process, I would also keep my ears open for any mention that the translator works with an editor. It can be especially helpful if the editor is a reputable Japanese-to-English translator (the opposite language combination). This way, the native English-speaking translator can check to make sure that the native Japanese-speaking translator has correctly understood, and therefore correctly translated, everything in the original English document.

If all this begins to sound very complicated, it is just because it really is. Anything less, however, puts you at greater risk of having your translation have a different meaning than your original document. If your business, your personal life, or anything else, depends on your translation having the same meaning as your original, then you can not be too careful!