Machine Translation: Evolution, Multilingual Models, Evaluation, and Challenges

1. W. J. Hutchins, "The Georgetown-IBM experiment demonstrated in January 1954," in Machine Translation: From Real Users to Research (AMTA 2004), Lecture Notes in Computer Science, vol. 3265. Berlin, Germany: Springer, 2004, pp. 102–114.
2. R. M. K. Sinha, K. Sivaraman, A. Agrawal, R. Jain, R. C. Srivastava, A. Jain, et al., "ANGLABHARTI: A multilingual machine-aided translation project on translation from English to Indian languages," in Proc. IEEE Int. Conf. Systems, Man, and Cybernetics, vol. 2, 1995, pp. 1609–1614, doi: 10.1109/ICSMC.1995.538002.
3. P. F. Brown, J. Cocke, S. A. Della Pietra, V. J. Della Pietra, F. Jelinek, J. D. Lafferty, R. L. Mercer, and P. S. Roossin, "A statistical approach to machine translation," Computational Linguistics, vol. 16, no. 2, pp. 79–85, Jun. 1990.
4. P. Koehn, F. J. Och, and D. Marcu, "Statistical phrase-based translation," in Proc. Human Language Technology Conf. North American Chapter of the Association for Computational Linguistics (HLT-NAACL), Edmonton, Canada, 2003, pp. 127–133.
5. I. Sutskever, O. Vinyals, and Q. V. Le, "Sequence to sequence learning with neural networks," in Advances in Neural Information Processing Systems, vol. 27, 2014, pp. 3104–3112.
6. D. Bahdanau, K. Cho, and Y. Bengio, "Neural machine translation by jointly learning to align and translate," in Proc. Int. Conf. Learning Representations (ICLR), 2015, arXiv:1409.0473.
7. A. Vaswani et al., "Attention is all you need," in Advances in Neural Information Processing Systems, vol. 30, 2017, pp. 5998–6008, arXiv:1706.03762.
8. NLLB Team, "Scaling neural machine translation to 200 languages," Nature, vol. 630, pp. 841–846, Jun. 2024, doi: 10.1038/s41586-024-07335-x.
9. J. Gala et al., "IndicTrans2: Towards high-quality and accessible machine translation models for all 22 scheduled Indian languages," Trans. Machine Learning Research, 2023, arXiv:2305.16307.
10. W. Zhu, H. Liu, Q. Dong, J. Xu, S. Huang, L. Kong, J. Chen, and L. Li, "Multilingual machine translation with large language models: Empirical results and analysis," in Findings of the Association for Computational Linguistics: NAACL 2024, Mexico City, Mexico, 2024, pp. 2765–2781, doi: 10.18653/v1/2024.findings-naacl.176.
11. Automatic Language Processing Advisory Committee, Language and Machines: Computers in Translation and Linguistics, Publication 1416. Washington, DC, USA: National Academy of Sciences–National Research Council, 1966.
12. A. Hurskainen and J. Tiedemann, "Rule-based machine translation from English to Finnish," in Proc. 2nd Conf. Machine Translation (WMT), Copenhagen, Denmark, 2017, pp. 323–329, doi: 10.18653/v1/W17-4731.
13. P. F. Brown, S. A. Della Pietra, V. J. Della Pietra, and R. L. Mercer, "The mathematics of statistical machine translation: Parameter estimation," Computational Linguistics, vol. 19, no. 2, pp. 263–311, Jun. 1993.
14. F. J. Och and H. Ney, "The alignment template approach to statistical machine translation," Computational Linguistics, vol. 30, no. 4, pp. 417–449, Dec.2004, doi: 10.1162/0891201042544884.
15. P. Koehn, H. Hoang, A. Birch, C. Callison-Burch, M. Federico, N. Bertoldi, et al., "Moses: Open source toolkit for statistical machine translation," in Proc. ACL Interactive Poster and Demonstration Sessions, Prague, Czech Republic, 2007, pp. 177–180.
16. P. Koehn and H. Hoang, "Factored translation models," in Proc. Joint Conf. Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), Prague, Czech Republic, 2007, pp. 868–876.
17. G. Foster and R. Kuhn, "Mixture-model adaptation for SMT," in Proc. 2nd Workshop on Statistical Machine Translation, Prague, Czech Republic, 2007,pp. 128–135.
18. N. Kalchbrenner and P. Blunsom, "Recurrent continuous translation models," in Proc. Conf. Empirical Methods in Natural Language Processing (EMNLP), Seattle, WA, USA, 2013, pp. 1700–1709.
19. K. Cho, B. van Merriënboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio, "Learning phrase representations using RNN encoder–decoder for statistical machine translation," in Proc. EMNLP, Doha, Qatar, 2014, pp. 1724–1734, doi: 10.3115/v1/D14-1179.
20. T. Luong, H. Pham, and C. D. Manning, "Effective approaches to attention-based neural machine translation," in Proc. Conf. Empirical Methods in Natural Language Processing (EMNLP), Lisbon, Portugal, 2015, pp. 1412–1421, doi: 10.18653/v1/D15-1166.
21. Y. Wu et al., "Google's neural machine translation system: Bridging the gap between human and machine translation," arXiv preprint arXiv:1609.08144, 2016.
22. R. Sennrich, B. Haddow, and A. Birch, "Neural machine translation of rare words with subword units," in Proc. 54th Annu. Meeting Association for Computational Linguistics (ACL), Berlin, Germany, 2016, pp. 1715–1725, doi: 10.18653/v1/P16-1162.
23. R. Sennrich, B. Haddow, and A. Birch, "Improving neural machine translation models with monolingual data," in Proc. 54th Annu. Meeting ACL, Berlin, Germany, 2016, pp. 86–96, doi: 10.18653/v1/P16-1009.
24. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, "BERT: Pre-training of deep bidirectional transformers for language understanding," in Proc. Conf. North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), Minneapolis, MN, USA, 2019, pp. 4171–4186, doi: 10.18653/v1/N19-1423.
25. A. Conneau, K. Khandelwal, N. Goyal, V. Chaudhary, G. Wenzek, F. Guzmán, E. Grave, M. Ott, L. Zettlemoyer, and V. Stoyanov, "Unsupervised cross-lingual representation learning at scale," in Proc. 58th Annu. Meeting ACL, online, 2020, pp. 8440–8451, doi: 10.18653/v1/2020.acl-main.747.
26. L. Xue, N. Constant, A. Roberts, M. Kale, R. Al-Rfou, A. Siddhant, A. Barua, and C. Raffel, "mT5: A massively multilingual pre-trained text-to-text transformer," in Proc. NAACL-HLT, online, 2021, pp. 483–498, doi: 10.18653/v1/2021.naacl-main.41.
27. Y. Liu, J. Gu, N. Goyal, X. Li, S. Edunov, M. Ghazvininejad, M. Lewis, and L. Zettlemoyer, "Multilingual denoising pre-training for neural machine translation," Trans. Association for Computational Linguistics, vol. 8, pp. 726–742, 2020, doi: 10.1162/tacl_a_00343.
28. M. Johnson, M. Schuster, Q. V. Le, M. Krikun, Y. Wu, Z. Chen, N. Thorat, F. Viégas, M. Wattenberg, G. Corrado, M. Hughes, and J. Dean, "Google's multilingual neural machine translation system: Enabling zero-shot translation," Trans. Association for Computational Linguistics, vol. 5, pp. 339–351, 2017, doi: 10.1162/tacl_a_00065.
29. A. Fan, S. Bhosale, H. Schwenk, Z. Ma, A. El-Kishky, S. Goyal, et al., "Beyond English-centric multilingual machine translation," J. Machine Learning Research, vol. 22, no. 107, pp. 1–48, 2021.
30. H. Xu, Y. J. Kim, A. Sharaf, and H. H. Awadalla, "A paradigm shift in machine translation: Boosting translation performance of large language models," in Proc. Int. Conf. Learning Representations (ICLR), 2024, arXiv:2309.11674.31. T. Kocmi et al., "Findings of the WMT24 general machine translation shared task: The LLM era is here but MT is not solved yet," in Proc. 9th Conf. Machine Translation (WMT), Miami, FL, USA, 2024, pp. 1–46, doi: 10.18653/v1/2024.wmt-1.1.
32. N. Goyal, C. Gao, V. Chaudhary, P.-J. Chen, G. Wenzek, D. Ju, S. Krishnan, M. Ranzato, F. Guzmán, and A. Fan, "The Flores-101 evaluation benchmark for low-resource and multilingual machine translation," Trans. Association for Computational Linguistics, vol. 10, pp. 522–538, 2022, doi: 10.1162/tacl_a_00474.
33. NLLB Team et al., "No language left behind: Scaling human-centered machine translation," arXiv preprint arXiv:2207.04672, 2022.
34. B. Alastruey et al., “Omnilingual MT: Machine Translation for 1,600 Languages,” in Proc. Conf. Empirical Methods in Natural Language Processing (EMNLP), Budapest, Hungary, 2026, pp. 29177–29200, doi: 10.18653/2026.emnlp-main.1280.
35. S. Qian and Y. Scherrer, “Why do Large Language Models Fail in Low-resource Translation? Unraveling the Token Dynamics of Large Language Models for Machine Translation,” in Proc. 26th Annu. Conf. European Association for Machine Translation (EAMT), vol. 1, Tilburg, The Netherlands, 2026, pp. 22–47.
36. S. Kumar, P. Jyothi, and P. Bhattacharyya, “SrcMix: Mixing of related source languages benefits extremely low-resource machine translation,” in Findings of the Association for Computational Linguistics: EACL 2026, Rabat, Morocco, 2026, pp. 6306–6323, doi: 10.18653/v1/2026.findings-eacl.332.
37. P. Pakray et al., “Findings of WMT 2026 shared task on Low-resource Indic Languages Translation,” in Proc. 11th Conf. Machine Translation (WMT), Budapest, Hungary, 2026, pp. 1091–1116, doi: 10.18653/2026.wmt-main.53.
38. K. Thellmann, B. Stadler, M. Färber, and J. Lehmann, “Quantifying the Impact of Translation Errors on Multilingual LLM Evaluation,” in Proc. 64th Annu. Meeting Association for Computational Linguistics (ACL), vol. 1: Long Papers, San Diego, CA, USA, 2026, pp. 41311–41334, doi: 10.18653/v1/2026.acl-long.1916.
39. Seamless Communication et al., "SeamlessM4T: Massively multilingual and multimodal machine translation," arXiv preprint arXiv:2308.11596, 2023.
40. Seamless Communication et al., "Joint speech and text machine translation for up to 100 languages," Nature, vol. 637, pp. 587–593, Jan. 2025, doi:10.1038/s41586-024-08359-z.
41. OpenAI, "GPT-4 technical report," arXiv preprint arXiv:2303.08774, 2023.
42. A. Hendy, M. Abdelrehim, A. Sharaf, V. Raunak, M. Gabr, H. Matsushita, et al., "How good are GPT models at machine translation? A comprehensive evaluation," arXiv preprint arXiv:2302.09210, 2023.
43. C. Lyu, Z. Du, J. Xu, Y. Duan, M. Wu, T. Lynn, A. F. Aji, D. F. Wong, and L. Wang, "A paradigm shift: The future of machine translation lies with large language models," in Proc. Joint Int. Conf. Computational Linguistics, Language Resources and Evaluation (LREC-COLING), Torino, Italy, 2024, pp. 1339–1352.
44. J. Pang, F. Ye, D. F. Wong, D. Yu, S. Shi, Z. Tu, and L. Wang, "Salute the classic: Revisiting challenges of machine translation in the age of large language models," Trans. Association for Computational Linguistics, vol. 13, pp. 73–95, 2025, doi: 10.1162/tacl_a_00730.
45. D. Elliott, S. Frank, K. Sima'an, and L. Specia, "Multi30K: Multilingual English-German image descriptions," in Proc. 5th Workshop on Vision and Language, Berlin, Germany, 2016, pp. 70–74, doi: 10.18653/v1/W16-3210.
46. S. Naskar and S. Bandyopadhyay, "Use of machine translation in India: Current status," in Proc. Machine Translation Summit X: Posters, Phuket, Thailand, 2005, pp. 465–470.
47. A. Bharati, V. Chaitanya, A. P. Kulkarni, R. Sangal, and G. U. Rao, "Anusaaraka: Overcoming the language barrier in India," arXiv preprint arXiv:cs/0308018, 2003.
48. G. Ramesh, S. Doddapaneni, A. Bheemaraj, M. Jobanputra, R. AK, A. Sharma, et al., "Samanantar: The largest publicly available parallel corpora collection for 11 Indic languages," Trans. Association for Computational Linguistics, vol. 10, pp. 145–162, 2022, doi: 10.1162/tacl_a_00452.
49. S. Doddapaneni, R. Aralikatte, G. Ramesh, S. Goyal, M. M. Khapra, A. Kunchukuttan, and P. Kumar, "Towards leaving no Indic language behind: Building monolingual corpora, benchmark and models for Indic languages," in Proc. 61st Annu. Meeting ACL, Toronto, Canada, 2023, pp. 12402–12426, doi: 10.18653/v1/2023.acl-long.693.
50. R. Dabre, H. Shrotriya, A. Kunchukuttan, R. Puduppully, M. Khapra, and P. Kumar, "IndicBART: A pre-trained model for Indic natural language generation," in Findings of the Association for Computational Linguistics: ACL 2022, Dublin, Ireland, 2022, pp. 1849–1863, doi: 10.18653/v1/2022.findings-acl.145.
51. Y. Madhani, S. Parthan, P. Bedekar, G. Nc, R. Khapra, A. Kunchukuttan, P. Kumar, and M. Khapra, "Aksharantar: Open Indic-language transliteration datasets and models for the next billion users," in Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, 2023, pp. 40–57, doi: 10.18653/v1/2023.findings-emnlp.4.
52. H. Singh, N. Gupta, S. Bharadwaj, D. Tewari, and P. Talukdar, "IndicGenBench: A multilingual benchmark to evaluate generation capabilities of LLMs on Indic languages," in Proc. 62nd Annu. Meeting ACL, Bangkok, Thailand, 2024, pp. 11047–11073, doi: 10.18653/v1/2024.acl-long.595.
53. A. Sai B, T. Dixit, V. Nagarajan, A. Kunchukuttan, P. Kumar, M. M. Khapra, and R. Dabre, "IndicMT Eval: A dataset to meta-evaluate machine translation metrics for Indian languages," in Proc. 61st Annu. Meeting ACL, Toronto, Canada, 2023, pp. 14210–14228, doi: 10.18653/v1/2023.acl-long.795.
54. T. Kocmi et al., "Findings of the WMT25 general machine translation shared task: Time to stop evaluating on easy test sets," in Proc. 10th Conf. Machine Translation (WMT), Suzhou, China, 2025, pp. 355–413, doi: 10.18653/v1/2025.wmt-1.22.
55. K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, "BLEU: A method for automatic evaluation of machine translation," in Proc. 40th Annu. Meeting ACL, Philadelphia, PA, USA, 2002, pp. 311–318, doi: 10.3115/1073083.1073135.
56. M. Post, "A call for clarity in reporting BLEU scores," in Proc. 3rd Conf. Machine Translation (WMT), Brussels, Belgium, 2018, pp. 186–191, doi: 10.18653/v1/W18-6319.
57. S. Banerjee and A. Lavie, "METEOR: An automatic metric for MT evaluation with improved correlation with human judgments," in Proc. ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, Ann Arbor, MI, USA, 2005, pp. 65–72.
58. M. Snover, B. Dorr, R. Schwartz, L. Micciulla, and J. Makhoul, "A study of translation edit rate with targeted human annotation," in Proc. 7th Conf.Association for Machine Translation in the Americas (AMTA), Cambridge, MA, USA, 2006, pp. 223–231.
59. M. Popović, "chrF: Character n-gram F-score for automatic MT evaluation," in Proc. 10th Workshop on Statistical Machine Translation (WMT), Lisbon, Portugal, 2015, pp. 392–395, doi: 10.18653/v1/W15-3049.
60. M. Popović, "chrF++: Words helping character n-grams," in Proc. 2nd Conf. Machine Translation (WMT), Copenhagen, Denmark, 2017, pp. 612–618, doi: 10.18653/v1/W17-4770.
61. T. Zhang, V. Kishore, F. Wu, K. Q. Weinberger, and Y. Artzi, "BERTScore: Evaluating text generation with BERT," in Proc. Int. Conf.Learning Representations (ICLR), 2020, arXiv:1904.09675.
62. T. Sellam, D. Das, and A. Parikh, "BLEURT: Learning robust metrics for text generation," in Proc. 58th Annu. Meeting ACL, online, 2020, pp. 7881–7892, doi: 10.18653/v1/2020.acl-main.704.
63. R. Rei, C. Stewart, A. C. Farinha, and A. Lavie, "COMET: A neural framework for MT evaluation," in Proc. Conf. Empirical Methods in Natural Language Processing (EMNLP), online, 2020, pp. 2685–2702, doi: 10.18653/v1/2020.emnlp-main.213.
64. N. M. Guerreiro, R. Rei, D. van Stigt, L. Coheur, P. Colombo, and A. F. T. Martins, "xCOMET: Transparent machine translation evaluation through fine-grained error detection," Trans. Association for Computational Linguistics, vol. 12, pp. 979–995, 2024, doi: 10.1162/tacl_a_00683.
65. M. Freitag, G. Foster, D. Grangier, V. Ratnakar, Q. Tan, and W. Macherey, "Experts, errors, and context: A large-scale study of human evaluation for machine translation," Trans. Association for Computational Linguistics, vol. 9, pp. 1460–1474, 2021, doi: 10.1162/tacl_a_00437.
66. T. Kocmi and C. Federmann, "Large language models are state-of-the-art evaluators of translation quality," in Proc. 24th Annu. Conf. European Association for Machine Translation (EAMT), Tampere, Finland, 2023, pp. 193–203.
67. N. M. Guerreiro, D. M. Alves, J. Waldendorf, B. Haddow, A. Birch, P. Colombo, and A. F. T. Martins, "Hallucinations in large multilingual translation models," Trans. Association for Computational Linguistics, vol. 11, pp. 1500–1517, 2023, doi: 10.1162/tacl_a_00615.
68. G. Stanovsky, N. A. Smith, and L. Zettlemoyer, "Evaluating gender bias in machine translation," in Proc. 57th Annu. Meeting ACL, Florence, Italy, 2019,pp. 1679–1684, doi: 10.18653/v1/P19-1164.