ChatGPT and Human Translators: Competition or Collaboration?
The Generative Pre-training Transformer Chat, better known as ChatGPT, is an artificial intelligence chatbot developed by the company OpenAI, which allows users to ask it questions using conversational or natural language. It was released in November 2022 with the goal of generating text in a similar way to humans, which can be used for a variety of applications, such as chatbots, automatic content generation, and language translation.
In the face of such advances in artificial intelligence, many professionals and organizations wonder: is this really a tool that has arrived to replace human translators?
The Model’s Limitations
Language models generate text based on the probability of a word occurring given the preceding words in the sequence. Trained on approximately 45 terabytes of internet text, the GPT-3 language model used by ChatGPT calculates that certain word sequences are more likely to occur than others.
The growing popularity of the software has generated criticism and distrust from various industries. Concern arises because it is challenging to distinguish human writing from text generated by ChatGPT.
It is estimated that ChatGPT reached 100 million monthly active users in January 2023, just two months after its launch, making it the fastest-growing consumer application in history.
Source: Reuters
The truth is that while the model has many strengths, it also has several weaknesses. ChatGPT can accurately add two-digit numbers, but when it comes to multiplying two-digit numbers, it produces the correct answer only about 30% of the time. Furthermore, like other extensive language models, the chatbot prototype often responds with inaccurate, misleading, or sometimes completely erroneous information, and it struggles with certain discourse issues such as consistency, referential expressions, and coherence.
In fact, when asked whether it is a reliable source of information, ChatGPT responds that “relying on ChatGPT as the sole source of objective information is not recommended. It should be used as a tool to generate text or complete language-based tasks, and any information provided by the model should be verified with reliable sources.”
A Complementary Approach
Beyond being able to provide correct or incorrect answers on various topics, when it comes to translations, ChatGPT also struggles with dealing with colloquialisms, slang, and acronyms, for example. It also cannot decipher multiple languages in a single message: although the software supports 95 different languages and 12 code languages, the platform can get confused by the use of more than one language in a single message.
Initial research has begun to address the translation capabilities and scope of ChatGPT. Some experts conclude that, as long as it is complemented with translation prompts and appropriately reviewed by a specialized human linguist, it can achieve a comparable or near-human performance for languages that have abundant resources. However, it shows less promising results for languages with fewer available references.
The truth is that ChatGPT does not have deep knowledge in specific areas of language. “When evaluating a series of benchmark datasets, we found that ChatGPT performs competitively with commercial translation products (e.g., Google Translate) in European languages with abundant resources but falls significantly behind in distant or low-resource languages”, concludes another recent research.
Although its contributions are very positive in many industries and activities, the limitations of this type of technology become evident again when it comes to tasks that involve common sense, emotional intelligence, context, deep sociocultural knowledge, relevance, and even grammatical elements.
In that regard, ChatGPT may have problems with idiomatic expressions and cultural references that are specific to a particular language. Likewise, it may not always recognize the context of a specific phrase or sentence, resulting in inaccurate translations.
Thus, accurate and high-quality translations still rely on experienced human translators or, at the very least, their involvement in the most critical parts of the translation process. Technology is a valuable and essential ally, but it does not eliminate the human factor from the equation. As with other artificial intelligence tools, a hybrid and synergistic approach appears to be the best method to tackle translation projects.