基于视觉行为的情感计算研究

发布时间：2018-01-25 22:25

本文关键词： 表情识别交互行为识别多模态融合情感计算人机交互　出处：《天津大学》2016年博士论文　论文类型：学位论文

【摘要】：随着计算机科学的不断发展,以及对个性化人机交互需求的不断增强,情感计算在人机交互中的重要性日益显著,基于情感理解与表达的人机交互研究受到了各领域的广泛关注。情感识别及理解作为情感计算的基础,是使计算机拥有情感感知能力的重要研究内容。而面部表情和肢体行为作为人类表达情感的两种主要方式,也就成为了情感识别研究的重要手段。因此,本文基于对面部表情及肢体行为的分析,对情感识别理解中的特征表示、分类方法及多模态融合等问题进行了深入的研究,并完成了如下几个方面的内容及创新工作:1.对基于面部表情的情感识别进行了深入的研究,提出了基于改进决策森林的表情识别方法。通过对人的面部表情进行分析,获得了对表情辨别中变化显著的多个面部区域,并基于该分析结果提出了一个基于决策森林的表情分类模型——隐马尔可夫决策森林模型(Hidden Markov Model Decision Forest,HMMDF)。该模型以决策树作为其基本拓扑结构。模型中每个节点被定义为一个具有可分辨性的强分类器,并利用多个隐马尔可夫链的组合进行构造,从而实现对表情的视频数据的分析识别。其中,每个隐马尔可夫链描述了唯一的面部区域,且针对不同的面部区域,其在模型中的作用价值通过权重的形式加以表示。由此,面部区域在表情识别中所表现出的差异性将在模型中得到充分的发挥,从而实现了对多个面部区域特征的融合。实验验证了面部区域对不同表情分辨能力的差异性特点,并通过实验对其进行了量化。实验结果表明,所提出的表情识别模型HMMDF在动态表情识别中能够取得较好的识别效果。2.对基于肢体行为的情感识别进行了研究,分析了交互行为中的情感交流,并针对交互行为特征描述问题,提出了与情感相关的交互行为的特征表示方法。基于交互行为的肢体运动特点,提出了两层特征表示模型。该模型不仅描述了行为过程中的运动特征,还融合了交互行为过程中的上下文信息。在该特征表示模型中,低层表示采用了局部时空轨迹的编码方法,并提出了层次时空轨迹编码模型对特征进行学习,从而实现了对运动特征的表示。此外,基于对交互行为中的时空关系特点的分析,还提出了交互行为的上下文信息的描述方法。该方法从行为序列中提取了蕴含交互行为中时空上下文信息的局部交互姿态轮廓,并以此作为交互上下文特征引入到低层表示中。模型的高层表示则基于对局部特征的编码,提出了基于时空序列特性的聚类方法,从行为的整体性角度进一步描述行为特征。3.基于以上研究,对人脸表情及肢体动作所表达情感的理解及意图的识别进行了深入探讨,提出了基于面部表情和肢体行为的多模态融合模型,解决了相似交互行为在不同环境中的情感理解及意图识别问题。该模型基于多层隐马尔可夫链实现,将交互者的表情分别作为彼此情感意图表达的上下文信息,并融合了交互中双方的行为姿态的变化。模型中不仅通过隐马尔可夫链定义了每种模态的行为状态转换关系,还描述了不同模态之间状态的相互联系,因此能够更有效地分辨相似的交互行为,进而实现了对情感的理解及意图的识别。4.提出并构建了基于多模态情感表达的行为数据集。为了对以上基于面部表情与肢体行为融合的情感意图识别方法进行验证,提出并建立了多模态的情感数据集。该数据集包含了来自32部电影的283个视频片段,共表达了4类情感意图,分别为庆祝、问候、安慰和感谢。此外,还对该数据集中所包含的面部表情类型、视频片段中行为的相关特性等进行了标注。综上所述,本文分别基于多种模态对情感识别中的特征表示、分类算法进行了深入的研究,并基于以上工作采用了多模态融合的方法对交互行为中复杂情感加以理解,进而实现了更深层的情感意图的识别。以上研究使计算机在具有情感识别能力的基础上,进一步具有理解情感、识别情感意图的能力,从而在人机交互中表现得更加灵活、自然、和谐。
[Abstract]:With the development of computer science, and constantly enhance the personalized human-computer interaction, the importance of affective computing in human-computer interaction is significant, human-computer interaction research based on emotional understanding and expression has attracted extensive attention in various fields. Emotion recognition and understanding as the basis of affective computing, is an important research content of the computer with emotion perception ability. And facial expressions and body behavior as the two main ways of human emotion, it has become an important means of emotion recognition. Therefore, the analysis of facial expressions and body behavior based on the characteristics of emotion recognition expressed understanding, in-depth study of the problem of classification and multimodal fusion etc., and complete the contents and innovations are as follows: 1. aspects of emotional facial expression recognition based on in-depth research, put forward A facial expression recognition method based on improved decision forest. Based on the analysis of human facial expressions, the change of expression to identify significant multiple facial region, and based on the results of the analysis put forward a classification model based on decision forest hidden Markov decision model (Hidden Markov Model Decision forest Forest, HMMDF). In this model, the decision tree as the basic topological structure. Each node in the model is defined as a strong classifier resolution, and constructed by using a combination of hidden Markov chain, so as to realize the recognition of video data on facial expression. Among them, each hidden Markov chain to describe the facial region only and, for different facial regions, the model of value represented by the form of weight. Thus, the face area is in in face recognition The difference will be brought into full play in the model, so as to realize the fusion of multiple facial region features. Experimental results show that the characteristics of facial region differences expression resolution, and experiments are carried out for the quantification. The experimental results show that the proposed face recognition model HMMDF can achieve.2. recognition effect the better studied emotion recognition based on nonverbal behavior in dynamic facial expression recognition, analysis of the emotional interaction behavior, and describe the problem for interactive behavior, a feature representation is proposed interactive behavior associated with emotion. Limb movement characteristics based on interactive behavior, puts forward two layer representation model. The model not only describes the motion characteristics of behavior in the process, but also the integration of the context information interaction process. In the feature model, the lower the Bureau said Department of spatio-temporal trajectory encoding method, and puts forward the hierarchical encoding model for learning spatio-temporal trajectory characteristics, so as to realize the representation of motion features. In addition, space-time relationship between the characteristics of interactive behavior based on the proposed method, describe the context information interaction behavior. The method of extracting the local interactive gesture contour the spatio-temporal context contains interactive behavior from the behavior sequence, and as interactive context features into the low layer. Said model is based on the high-level expression of local feature encoding, proposes clustering method based on the characteristics of the empty sequence, study the behavior characteristics of.3. based on the above described further from the behaviour of the whole angle. The expression of emotion understanding of facial expressions and body movements and intention recognition are discussed and the proposed model of facial expressions and body based on behavior State fusion model to solve the similar interaction behavior in different environments and emotional understanding of intention recognition problem. The model of multi hidden Markov chain based on the interaction of expression as the context information of each other emotional intent, change and integration of the interaction in the behavior of both attitude. Not only through the hidden Markov chain model the definition of transformation between each modal behavior, the contact state between different modes is also described, so it can effectively differentiate the interactive behavior of similar, so as to realize the emotional understanding and intention recognition.4. propose and construct the behavioral data set based on multi modal emotional expression. Based on the above in order to the fusion of facial expressions and body emotion behavior intention recognition method is verified, and the establishment of the proposed multimodal emotion data set. The data set contains from 283 video clips of 32 movies, 4 kinds of co expression of emotion intention, respectively, to celebrate, greetings, comfort and gratitude. In addition, facial expression of the data type also contains the behavior in video clips related characteristics were marked. In summary, based on the characteristics of multimodal emotion recognition the said classification algorithm is studied, and based on the above work by using a multimodal fusion approach to understanding complex emotional interaction behavior, so as to realize the recognition of emotion intention deeper. Based on above research so that the computer has the ability of emotion recognition, with further understanding of emotion, emotion recognition ability the intention, thus in human-computer interaction is more flexible, natural and harmonious.

【学位授予单位】：天津大学
【学位级别】：博士
【学位授予年份】：2016
【分类号】：TP391.41

【相似文献】