Ship-lemmatagger: Building an nlp toolkit for a peruvian native language

José Pereira-Noriega; Rodolfo Mercado-Gonzales; Andrés Melgar; Marco Sobrevilla-Cabezudo; Arturo Oncevay-Marcos

doi:10.1007/978-3-319-64206-2_53

Ship-lemmatagger: Building an nlp toolkit for a peruvian native language

José Pereira-Noriega, Rodolfo Mercado-Gonzales, Andrés Melgar, Marco Sobrevilla-Cabezudo, Arturo Oncevay-Marcos

Grupo de Inteligencia Artificial PUCP

Pontifical Catholic Univ. of Peru

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

5 Scopus citations

Abstract

Natural Language Processing deals with the understanding and generation of texts through computer programs. There are many different functionalities used in this area, but among them there are some functions that are the support of the remaining ones. These methods are related to the core processing of the morphology of the language (such as lemmatization) and automatic identification of the part-of-speech tag. Thereby, this paper describes the implementation of a basic NLP toolkit for a new language, focusing in the features mentioned before, and testing them in an own corpus built for the occasion. The obtained results exceeded the expected results and could be used for more complex tasks such as machine translation.

Original language	English
Title of host publication	Text, Speech, and Dialogue - 20th International Conference, TSD 2017, Proceedings
Editors	Kamil Ekstein, Vaclav Matousek
Publisher	Springer Verlag
Pages	473-481
Number of pages	9
ISBN (Print)	9783319642055
DOIs	https://doi.org/10.1007/978-3-319-64206-2_53
State	Published - 2017
Event	20th International Conference on Text, Speech and Dialogue, TSD 2017 - Prague, Czech Republic Duration: 27 Aug 2017 → 31 Aug 2017

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	10415 LNAI
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	20th International Conference on Text, Speech and Dialogue, TSD 2017
Country/Territory	Czech Republic
City	Prague
Period	27/08/17 → 31/08/17

Keywords

Lemmatization
Low resource language
Part-of-speech tagging
Shipibo-konibo

Access to Document

10.1007/978-3-319-64206-2_53

Cite this

Pereira-Noriega, J., Mercado-Gonzales, R., Melgar, A., Sobrevilla-Cabezudo, M., & Oncevay-Marcos, A. (2017). Ship-lemmatagger: Building an nlp toolkit for a peruvian native language. In K. Ekstein, & V. Matousek (Eds.), Text, Speech, and Dialogue - 20th International Conference, TSD 2017, Proceedings (pp. 473-481). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10415 LNAI). Springer Verlag. https://doi.org/10.1007/978-3-319-64206-2_53

Pereira-Noriega, José ; Mercado-Gonzales, Rodolfo ; Melgar, Andrés et al. / Ship-lemmatagger : Building an nlp toolkit for a peruvian native language. Text, Speech, and Dialogue - 20th International Conference, TSD 2017, Proceedings. editor / Kamil Ekstein ; Vaclav Matousek. Springer Verlag, 2017. pp. 473-481 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{475f1553e05e4e2bb1bb5cf82592c8cb,

title = "Ship-lemmatagger: Building an nlp toolkit for a peruvian native language",

abstract = "Natural Language Processing deals with the understanding and generation of texts through computer programs. There are many different functionalities used in this area, but among them there are some functions that are the support of the remaining ones. These methods are related to the core processing of the morphology of the language (such as lemmatization) and automatic identification of the part-of-speech tag. Thereby, this paper describes the implementation of a basic NLP toolkit for a new language, focusing in the features mentioned before, and testing them in an own corpus built for the occasion. The obtained results exceeded the expected results and could be used for more complex tasks such as machine translation.",

keywords = "Lemmatization, Low resource language, Part-of-speech tagging, Shipibo-konibo",

author = "Jos{\'e} Pereira-Noriega and Rodolfo Mercado-Gonzales and Andr{\'e}s Melgar and Marco Sobrevilla-Cabezudo and Arturo Oncevay-Marcos",

note = "Publisher Copyright: {\textcopyright} Springer International Publishing AG 2017.; 20th International Conference on Text, Speech and Dialogue, TSD 2017 ; Conference date: 27-08-2017 Through 31-08-2017",

year = "2017",

doi = "10.1007/978-3-319-64206-2_53",

language = "English",

isbn = "9783319642055",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "473--481",

editor = "Kamil Ekstein and Vaclav Matousek",

booktitle = "Text, Speech, and Dialogue - 20th International Conference, TSD 2017, Proceedings",

}

Pereira-Noriega, J, Mercado-Gonzales, R, Melgar, A, Sobrevilla-Cabezudo, M & Oncevay-Marcos, A 2017, Ship-lemmatagger: Building an nlp toolkit for a peruvian native language. in K Ekstein & V Matousek (eds), Text, Speech, and Dialogue - 20th International Conference, TSD 2017, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 10415 LNAI, Springer Verlag, pp. 473-481, 20th International Conference on Text, Speech and Dialogue, TSD 2017, Prague, Czech Republic, 27/08/17. https://doi.org/10.1007/978-3-319-64206-2_53

Ship-lemmatagger: Building an nlp toolkit for a peruvian native language. / Pereira-Noriega, José; Mercado-Gonzales, Rodolfo; Melgar, Andrés et al.
Text, Speech, and Dialogue - 20th International Conference, TSD 2017, Proceedings. ed. / Kamil Ekstein; Vaclav Matousek. Springer Verlag, 2017. p. 473-481 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10415 LNAI).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Ship-lemmatagger

T2 - 20th International Conference on Text, Speech and Dialogue, TSD 2017

AU - Pereira-Noriega, José

AU - Mercado-Gonzales, Rodolfo

AU - Melgar, Andrés

AU - Sobrevilla-Cabezudo, Marco

AU - Oncevay-Marcos, Arturo

N1 - Publisher Copyright: © Springer International Publishing AG 2017.

PY - 2017

Y1 - 2017

N2 - Natural Language Processing deals with the understanding and generation of texts through computer programs. There are many different functionalities used in this area, but among them there are some functions that are the support of the remaining ones. These methods are related to the core processing of the morphology of the language (such as lemmatization) and automatic identification of the part-of-speech tag. Thereby, this paper describes the implementation of a basic NLP toolkit for a new language, focusing in the features mentioned before, and testing them in an own corpus built for the occasion. The obtained results exceeded the expected results and could be used for more complex tasks such as machine translation.

AB - Natural Language Processing deals with the understanding and generation of texts through computer programs. There are many different functionalities used in this area, but among them there are some functions that are the support of the remaining ones. These methods are related to the core processing of the morphology of the language (such as lemmatization) and automatic identification of the part-of-speech tag. Thereby, this paper describes the implementation of a basic NLP toolkit for a new language, focusing in the features mentioned before, and testing them in an own corpus built for the occasion. The obtained results exceeded the expected results and could be used for more complex tasks such as machine translation.

KW - Lemmatization

KW - Low resource language

KW - Part-of-speech tagging

KW - Shipibo-konibo

UR - http://www.scopus.com/inward/record.url?scp=85028645758&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-64206-2_53

DO - 10.1007/978-3-319-64206-2_53

M3 - Conference contribution

AN - SCOPUS:85028645758

SN - 9783319642055

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 473

EP - 481

BT - Text, Speech, and Dialogue - 20th International Conference, TSD 2017, Proceedings

A2 - Ekstein, Kamil

A2 - Matousek, Vaclav

PB - Springer Verlag

Y2 - 27 August 2017 through 31 August 2017

ER -

Pereira-Noriega J, Mercado-Gonzales R, Melgar A, Sobrevilla-Cabezudo M, Oncevay-Marcos A. Ship-lemmatagger: Building an nlp toolkit for a peruvian native language. In Ekstein K, Matousek V, editors, Text, Speech, and Dialogue - 20th International Conference, TSD 2017, Proceedings. Springer Verlag. 2017. p. 473-481. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-64206-2_53

Ship-lemmatagger: Building an nlp toolkit for a peruvian native language

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this