藏汉跨语言话题模型构建及对齐方法研究

发布时间：2018-11-14 08:54

【摘要】：如何获取藏文话题在其他语种中的相关信息,对于促进少数民族地区的社会管理科学化水平、维护民族团结和国家统一、构建和谐社会具有重要意义。目前大多数研究集中在英汉跨语言信息处理方面,针对藏汉跨语言研究较少。如何根据藏语、汉语的特点,并结合目前藏语信息处理的研究现状,实现藏汉多角度的社会网络关系关联,同步发现关注话题并进行数据比较,是迫切需要解决的问题。该文在藏汉可比语料的基础上,利用词向量对文本词语进行语义扩展,进而构建LDA话题模型,并利用Gibbs sampling进行模型参数的估计,抽取出藏语和汉语话题。在LDA话题模型生成的文档-话题分布的基础上,提出一种基于余弦相似度、欧氏距离、Hellinger距离和KL距离四种相似度算法的投票方法,来实现藏汉话题的对齐。
[Abstract]:How to obtain the relevant information of Tibetan topics in other languages is of great significance to promote the scientific level of social management in minority areas, maintain national unity and national unity, and build a harmonious society. At present, most researches focus on English and Chinese cross-language information processing, but few on Tibetan and Chinese. According to the characteristics of Tibetan and Chinese and the current research situation of Tibetan information processing, it is urgent to solve the problem that how to realize the social network relation of Tibetan and Chinese, find the topic of concern and compare the data simultaneously. On the basis of Tibetan and Chinese comparable corpus, this paper uses word vector to extend the semantic of text words, and then constructs LDA topic model, and estimates the parameters of the model by using Gibbs sampling to extract Tibetan and Chinese topics. On the basis of document-topic distribution generated by LDA topic model, a voting method based on four similarity algorithms, cosine similarity, Euclidean distance, Hellinger distance and KL distance, is proposed to achieve the alignment of Tibetan and Chinese topics.
【作者单位】：中央民族大学信息工程学院;国家语言资源监测与研究中心少数民族语言分中心;
【基金】：国家自然科学基金(61501529,61331013) 国家语委项目(ZDI125-36,YB125-139)
【分类号】：H214;H136;TP391.1

【相似文献】