Findings of the AmericasNLP 2023 Shared Task on Machine Translation into Indigenous Languages

Abteen Ebrahimi; Manuel Mager; Shruti Rijhwani; Enora Rice; Arturo Oncevay; Claudia Garcia Baltazar; María Elena Méndez Cortés; Cynthia Montaño; John E. Ortega; Rolando Coto-Solano; Hilaria Cruz; Alexis Palmer; Katharina Kann

Findings of the AmericasNLP 2023 Shared Task on Machine Translation into Indigenous Languages

Abteen Ebrahimi
, Manuel Mager
, Shruti Rijhwani
, Enora Rice
, Arturo Oncevay
, Claudia Garcia Baltazar
, María Elena Méndez Cortés
, Cynthia Montaño
, John E. Ortega
, Rolando Coto-Solano
, Hilaria Cruz
, Alexis Palmer
, Katharina Kann

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

21 Scopus citations

Abstract

In this work, we present the results of the AmericasNLP 2023 Shared Task on Machine Translation into Indigenous Languages. This edition of the shared task features eleven language pairs, one of which – Chatino–Spanish – uses a newly collected evaluation dataset, consisting of professionally translated text from the legal domain. Seven teams participated in the shared task, with a total of 181 submissions. Additionally, we conduct a human evaluation of the best system outputs and compare them to the best submissions from the 2021 shared task. We find that this analysis agrees with the quantitative measure we use to rank submissions, ChrF, which itself shows an improvement of 9.64 points on average across all languages, compared to the prior winning system.

Original language	English
Title of host publication	ACL 2023 - Proceedings of the Workshop on Natural Language Processing for Indigenous Languages of the Americas, AmericasNLP 2023
Editors	Mager Manuel, Abteen Ebrahimi, Arturo Oncevay, Enora Rice, Shruti Rijhwani, Alexis Palmer, Katharina Kann
Publisher	Association for Computational Linguistics (ACL)
Pages	206-219
Number of pages	14
ISBN (Electronic)	9781959429913
State	Published - 2023
Externally published	Yes
Event	3rd Workshop on Natural Language Processing for Indigenous Languages of the Americas, AmericasNLP 2023, co-located with ACL 2023 - Toronto, Canada Duration: 14 Jul 2023 → …

Publication series

Name	Proceedings of the Annual Meeting of the Association for Computational Linguistics
ISSN (Print)	0736-587X

Conference

Conference	3rd Workshop on Natural Language Processing for Indigenous Languages of the Americas, AmericasNLP 2023, co-located with ACL 2023
Country/Territory	Canada
City	Toronto
Period	14/07/23 → …

Cite this

Ebrahimi, A., Mager, M., Rijhwani, S., Rice, E., Oncevay, A., Baltazar, C. G., Méndez Cortés, M. E., Montaño, C., Ortega, J. E., Coto-Solano, R., Cruz, H., Palmer, A., & Kann, K. (2023). Findings of the AmericasNLP 2023 Shared Task on Machine Translation into Indigenous Languages. In M. Manuel, A. Ebrahimi, A. Oncevay, E. Rice, S. Rijhwani, A. Palmer, & K. Kann (Eds.), ACL 2023 - Proceedings of the Workshop on Natural Language Processing for Indigenous Languages of the Americas, AmericasNLP 2023 (pp. 206-219). (Proceedings of the Annual Meeting of the Association for Computational Linguistics). Association for Computational Linguistics (ACL).

Ebrahimi, Abteen ; Mager, Manuel ; Rijhwani, Shruti et al. / Findings of the AmericasNLP 2023 Shared Task on Machine Translation into Indigenous Languages. ACL 2023 - Proceedings of the Workshop on Natural Language Processing for Indigenous Languages of the Americas, AmericasNLP 2023. editor / Mager Manuel ; Abteen Ebrahimi ; Arturo Oncevay ; Enora Rice ; Shruti Rijhwani ; Alexis Palmer ; Katharina Kann. Association for Computational Linguistics (ACL), 2023. pp. 206-219 (Proceedings of the Annual Meeting of the Association for Computational Linguistics).

@inproceedings{556d3b5b50ee497088755b7af6275a58,

title = "Findings of the AmericasNLP 2023 Shared Task on Machine Translation into Indigenous Languages",

abstract = "In this work, we present the results of the AmericasNLP 2023 Shared Task on Machine Translation into Indigenous Languages. This edition of the shared task features eleven language pairs, one of which – Chatino–Spanish – uses a newly collected evaluation dataset, consisting of professionally translated text from the legal domain. Seven teams participated in the shared task, with a total of 181 submissions. Additionally, we conduct a human evaluation of the best system outputs and compare them to the best submissions from the 2021 shared task. We find that this analysis agrees with the quantitative measure we use to rank submissions, ChrF, which itself shows an improvement of 9.64 points on average across all languages, compared to the prior winning system.",

author = "Abteen Ebrahimi and Manuel Mager and Shruti Rijhwani and Enora Rice and Arturo Oncevay and Baltazar, \{Claudia Garcia\} and \{M{\'e}ndez Cort{\'e}s\}, \{Mar{\'i}a Elena\} and Cynthia Monta{\~n}o and Ortega, \{John E.\} and Rolando Coto-Solano and Hilaria Cruz and Alexis Palmer and Katharina Kann",

note = "Publisher Copyright: {\textcopyright} 2023 Association for Computational Linguistics.; 3rd Workshop on Natural Language Processing for Indigenous Languages of the Americas, AmericasNLP 2023, co-located with ACL 2023 ; Conference date: 14-07-2023",

year = "2023",

language = "English",

series = "Proceedings of the Annual Meeting of the Association for Computational Linguistics",

publisher = "Association for Computational Linguistics (ACL)",

pages = "206--219",

editor = "Mager Manuel and Abteen Ebrahimi and Arturo Oncevay and Enora Rice and Shruti Rijhwani and Alexis Palmer and Katharina Kann",

booktitle = "ACL 2023 - Proceedings of the Workshop on Natural Language Processing for Indigenous Languages of the Americas, AmericasNLP 2023",

}

Ebrahimi, A, Mager, M, Rijhwani, S, Rice, E, Oncevay, A, Baltazar, CG, Méndez Cortés, ME, Montaño, C, Ortega, JE, Coto-Solano, R, Cruz, H, Palmer, A & Kann, K 2023, Findings of the AmericasNLP 2023 Shared Task on Machine Translation into Indigenous Languages. in M Manuel, A Ebrahimi, A Oncevay, E Rice, S Rijhwani, A Palmer & K Kann (eds), ACL 2023 - Proceedings of the Workshop on Natural Language Processing for Indigenous Languages of the Americas, AmericasNLP 2023. Proceedings of the Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics (ACL), pp. 206-219, 3rd Workshop on Natural Language Processing for Indigenous Languages of the Americas, AmericasNLP 2023, co-located with ACL 2023, Toronto, Canada, 14/07/23.

Findings of the AmericasNLP 2023 Shared Task on Machine Translation into Indigenous Languages. / Ebrahimi, Abteen; Mager, Manuel; Rijhwani, Shruti et al.
ACL 2023 - Proceedings of the Workshop on Natural Language Processing for Indigenous Languages of the Americas, AmericasNLP 2023. ed. / Mager Manuel; Abteen Ebrahimi; Arturo Oncevay; Enora Rice; Shruti Rijhwani; Alexis Palmer; Katharina Kann. Association for Computational Linguistics (ACL), 2023. p. 206-219 (Proceedings of the Annual Meeting of the Association for Computational Linguistics).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Findings of the AmericasNLP 2023 Shared Task on Machine Translation into Indigenous Languages

AU - Ebrahimi, Abteen

AU - Mager, Manuel

AU - Rijhwani, Shruti

AU - Rice, Enora

AU - Oncevay, Arturo

AU - Baltazar, Claudia Garcia

AU - Méndez Cortés, María Elena

AU - Montaño, Cynthia

AU - Ortega, John E.

AU - Coto-Solano, Rolando

AU - Cruz, Hilaria

AU - Palmer, Alexis

AU - Kann, Katharina

PY - 2023

Y1 - 2023

N2 - In this work, we present the results of the AmericasNLP 2023 Shared Task on Machine Translation into Indigenous Languages. This edition of the shared task features eleven language pairs, one of which – Chatino–Spanish – uses a newly collected evaluation dataset, consisting of professionally translated text from the legal domain. Seven teams participated in the shared task, with a total of 181 submissions. Additionally, we conduct a human evaluation of the best system outputs and compare them to the best submissions from the 2021 shared task. We find that this analysis agrees with the quantitative measure we use to rank submissions, ChrF, which itself shows an improvement of 9.64 points on average across all languages, compared to the prior winning system.

AB - In this work, we present the results of the AmericasNLP 2023 Shared Task on Machine Translation into Indigenous Languages. This edition of the shared task features eleven language pairs, one of which – Chatino–Spanish – uses a newly collected evaluation dataset, consisting of professionally translated text from the legal domain. Seven teams participated in the shared task, with a total of 181 submissions. Additionally, we conduct a human evaluation of the best system outputs and compare them to the best submissions from the 2021 shared task. We find that this analysis agrees with the quantitative measure we use to rank submissions, ChrF, which itself shows an improvement of 9.64 points on average across all languages, compared to the prior winning system.

UR - https://www.scopus.com/pages/publications/85175616115

M3 - Conference contribution

AN - SCOPUS:85175616115

T3 - Proceedings of the Annual Meeting of the Association for Computational Linguistics

SP - 206

EP - 219

BT - ACL 2023 - Proceedings of the Workshop on Natural Language Processing for Indigenous Languages of the Americas, AmericasNLP 2023

A2 - Manuel, Mager

A2 - Ebrahimi, Abteen

A2 - Oncevay, Arturo

A2 - Rice, Enora

A2 - Rijhwani, Shruti

A2 - Palmer, Alexis

A2 - Kann, Katharina

PB - Association for Computational Linguistics (ACL)

T2 - 3rd Workshop on Natural Language Processing for Indigenous Languages of the Americas, AmericasNLP 2023, co-located with ACL 2023

Y2 - 14 July 2023

ER -

Ebrahimi A, Mager M, Rijhwani S, Rice E, Oncevay A, Baltazar CG et al. Findings of the AmericasNLP 2023 Shared Task on Machine Translation into Indigenous Languages. In Manuel M, Ebrahimi A, Oncevay A, Rice E, Rijhwani S, Palmer A, Kann K, editors, ACL 2023 - Proceedings of the Workshop on Natural Language Processing for Indigenous Languages of the Americas, AmericasNLP 2023. Association for Computational Linguistics (ACL). 2023. p. 206-219. (Proceedings of the Annual Meeting of the Association for Computational Linguistics).

Findings of the AmericasNLP 2023 Shared Task on Machine Translation into Indigenous Languages

Abstract

Publication series

Conference

Other files and links

Fingerprint

Cite this