SciELO - Scientific Electronic Library Online

 
vol.21 issue4Named Entity Recognition on Code-Mixed Cross-Script Social Media ContentA Supervised Method to Predict the Popularity of News Articles author indexsubject indexsearch form
Home Pagealphabetic serial listing  

Services on Demand

Journal

Article

Indicators

Related links

  • Have no similar articlesSimilars in SciELO

Share


Computación y Sistemas

Print version ISSN 1405-5546

Abstract

GHOSH, Souvick; GHOSH, Satanu  and  DAS, Dipankar. Complexity Metric for Code-Mixed Social Media Text. Comp. y Sist. [online]. 2017, vol.21, n.4, pp.693-701. ISSN 1405-5546.  http://dx.doi.org/10.13053/cys-21-4-2852.

An evaluation metric is an absolute necessity for measuring the performance of any system and complexity of any data. In this paper, we have discussed how to determine the level of complexity of code-mixed social media texts that are growing rapidly due to multilingual interference. In general, texts written in multiple languages are often hard to comprehend and analyze. At the same time, in order to meet the demands of analysis, it is also necessary to determine the complexity of a particular document or a text segment. Thus, in the present paper, we have discussed the existing metrics for determining the code-mixing complexity of a corpus, their advantages and shortcomings as well as proposed several improvements on the existing metrics. The new index better reflects the variety and complexity of a multilingual document. Also, the index can be applied to a sentence and seamlessly extended to a paragraph or an entire document. We have employed two existing code-mixed corpora to suit the requirements of our study.

Keywords : Natural language processing; computational linguistics; human language technologies; scientometrics; code mixing; code switching; transliteration; social media text; complexity index; code mixing index.

        · text in English     · English ( pdf )