AIAC-TR-2026-01: Baseline Evaluation of Tokenization and Morphological Parsing for Kannada Corpora

Darshan K., Aditi S., Bhavana V.

AIAC Technical Report Series · 2026-02-15

Abstract

Comprehensive empirical analysis comparing byte-pair encoding (BPE) vs sentencepiece subword segmentation on low-resource regional text with vocabulary boundary metrics. Evaluates agglutinative morphology preservation across 10,000 sentences from regional public administrative records.

Cite this Work

@techreport{aiac_tr_2026_01,
  title = {AIAC-TR-2026-01: Baseline Evaluation of Tokenization and Morphological Parsing for Kannada Corpora},
  author = {Darshan K., Aditi S., Bhavana V.},
  institution = {AI & Analytics Club Research Division},
  year = {2026},
  url = {https://aiac.siddharthgupta.com.np/outputs/tr-2026-01}
}

License & Open Access

Distribution: This work is released under CC BY 4.0 / Apache-2.0. You are free to share and adapt the material provided appropriate attribution is credited to AIAC and original student authors.