prikaz prve stranice dokumenta Testing ChatGPT’s Capabilities as an English-Croatian Machine Translation System in a Real-World Setting: eTranslation versus ChatGPT at the European Central Bank
  Download
PDF 848.19 KB
master's thesis
Testing ChatGPT’s Capabilities as an English-Croatian Machine Translation System in a Real-World Setting: eTranslation versus ChatGPT at the European Central Bank
Zagreb: University of Zagreb, Faculty of Humanities and Social Sciences, 2024. urn:nbn:hr:131:477427
Jozić, Krešimir
University of Zagreb
Faculty of Humanities and Social Sciences
Department of English language and literature

Institutional repository: ODRAZ

Cite this document

Jozić, K. (2024). Testing ChatGPT’s Capabilities as an English-Croatian Machine Translation System in a Real-World Setting: eTranslation versus ChatGPT at the European Central Bank (Master's thesis). Zagreb: University of Zagreb, Faculty of Humanities and Social Sciences. Retrieved from https://urn.nsk.hr/urn:nbn:hr:131:477427

Jozić, Krešimir. "Testing ChatGPT’s Capabilities as an English-Croatian Machine Translation System in a Real-World Setting: eTranslation versus ChatGPT at the European Central Bank." Master's thesis, University of Zagreb, Faculty of Humanities and Social Sciences, 2024. https://urn.nsk.hr/urn:nbn:hr:131:477427

Jozić, Krešimir. "Testing ChatGPT’s Capabilities as an English-Croatian Machine Translation System in a Real-World Setting: eTranslation versus ChatGPT at the European Central Bank." Master's thesis, University of Zagreb, Faculty of Humanities and Social Sciences, 2024. https://urn.nsk.hr/urn:nbn:hr:131:477427

Jozić, K. (2024). 'Testing ChatGPT’s Capabilities as an English-Croatian Machine Translation System in a Real-World Setting: eTranslation versus ChatGPT at the European Central Bank', Master's thesis, University of Zagreb, Faculty of Humanities and Social Sciences, accessed 16 June 2024, https://urn.nsk.hr/urn:nbn:hr:131:477427

Jozić K. Testing ChatGPT’s Capabilities as an English-Croatian Machine Translation System in a Real-World Setting: eTranslation versus ChatGPT at the European Central Bank [Master's thesis]. Zagreb: University of Zagreb, Faculty of Humanities and Social Sciences; 2024 [cited 2024 June 16] Available at: https://urn.nsk.hr/urn:nbn:hr:131:477427

K. Jozić, "Testing ChatGPT’s Capabilities as an English-Croatian Machine Translation System in a Real-World Setting: eTranslation versus ChatGPT at the European Central Bank", Master's thesis, University of Zagreb, Faculty of Humanities and Social Sciences, Zagreb, 2024. Available at: https://urn.nsk.hr/urn:nbn:hr:131:477427

Metadata
TitleTesting ChatGPT’s Capabilities as an English-Croatian Machine Translation System in a Real-World Setting: eTranslation versus ChatGPT at the European Central Bank
Title (croatian)Testiranje mogućnosti jezičnog modela ChatGPT kao sustava za strojno prevođenje s engleskog na hrvatski u stvarnom okružju: eTranslation i ChatGPT u Europskoj središnjoj banci
AuthorKrešimir Jozić
MentorNataša Pavlović (mentor)
Committee memberKristijan Nikolić (član povjerenstva)
Committee memberMateusz-Milan Stanojević (član povjerenstva)
GranterUniversity of Zagreb
Faculty of Humanities and Social Sciences
(Department of English language and literature)
Zagreb
Defense date and country2024-04-24, Croatia
Scientific / art field,
discipline and subdiscipline		HUMANISTIC SCIENCES
Philology
Anglistics
Abstract
Ever since the 1950s, researchers have continuously strived to improve machine translation systems, developing new, more successful approaches to the machine translation problem. The newest innovation in the field of natural language processing are transformer-based large language models, which can perform various language tasks, including machine translation. This study explores the performance of the large language model ChatGPT-3.5 as an English-Croatian machine translation system in a real-world professional setting. An excerpt from a document of the European Central Bank was translated with ChatGPT and with eTranslation, the more conventional neural machine translation system used by ECB translators. Two translators with experience in working at the ECB’s Croatian translation unit were then asked to evaluate the quality of both translations. An automatic evaluation using six different metrics and an error analysis using the MQM error typology were also performed. All this was done to determine if ChatGPT would be more useful than eTranslation for the ECB’s Croatian translation unit, and if ChatGPT better deals with problems that more conventional neural translation systems struggle with, specifically context-related issues. The findings suggest that ChatGPT would presently not be as useful as eTranslation for English-Croatian machine translation at the European Central Bank and that ChatGPT does not fare significantly better than eTranslation when it comes to context-related issues. Future research could further explore this topic, as the use of ChatGPT for English-Croatian machine translation remains underexplored.
Abstract (croatian)
Još od 1950-ih godina znanstvenici neprestano teže unaprjeđenju sustava za strojno prevođenje smišljajući nove, uspješnije pristupe problemu strojnog prevođenja. Najnovija inovacija na području obrade prirodnog jezika veliki su jezični modeli koji se temelje na transformerima. Sustavi su to koji mogu obavljati različite jezične zadaće, uključujući strojno prevođenje. U ovoj studiji istražuje se koliko je veliki jezični model ChatGPT-3.5 uspješan kao sustav za strojno prevođenje s engleskog na hrvatski u stvarnom profesionalnom okružju. Odlomak iz dokumenta Europske središnje banke preveden je sustavom ChatGPT i sustavom eTranslation, konvencionalnijim neuronskim strojnim prevoditeljem koji se upotrebljava u Europskoj središnjoj banci. Zatim su prevoditelji s iskustvom rada u hrvatskoj prevoditeljskoj jedinici ESB-a procijenili kvalitetu obaju prijevoda. Provedena je i automatska procjena s pomoću šest različitih metrika te analiza pogrešaka prema sustavu MQM. Sve je to učinjeno kako bi se utvrdilo bi li ChatGPT hrvatskim prevoditeljima u ESB-u bio korisniji od sustava eTranslation te rješava li ChatGPT uspješno probleme koje raniji sustavi nisu mogli riješiti, točnije probleme povezane s kontekstom. Rezultati istraživanja upućuju na to da ChatGPT hrvatskim prevoditeljima u ESB-u trenutačno ne bi bio korisniji od sustava eTranslation i na to da ChatGPT nije znatno uspješniji u rješavanju problema povezanih s kontekstom. S obzirom na to da je upotreba modela ChatGPT za strojno prevođenje s engleskog na hrvatski i dalje velikim dijelom neistražena, buduća istraživanja mogla bi dodatno istražiti tu temu.
Keywords
Keywords (croatian)
Languageenglish
URN:NBNurn:nbn:hr:131:477427
Study programmeTitle: English Language and Literature (double major); specializations in: Linguistics, Literature and Culture, Translation, TEFL
Course: Translation
Study programme type: university
Study level: graduate
Academic / professional title: magistar/magistra engleskog jezika i književnosti (magistar/magistra engleskog jezika i književnosti)
Type of resourceText
File originBorn digital
Access conditionsOpen access
Terms of useLicence In copyright icon
Created on2024-04-29 17:15:02