基于要素的词汇量可扩展的手语手势动作识别方法研究

发布时间：2018-12-08 20:04

【摘要】：手语手势识别旨在将手势动作翻译成文本或者语音形式,以增进聋哑人群体和正常人之间的交流。同时,手势识别技术在人机交互系统的应用中也扮演了不可或缺的角色。从实用角度出发,针对聋哑人交流需求设计的手语翻译装置应满足便携性、一定规模的可识别词汇量、较高的识别效率、低成本等基本要求。基于表面肌电、加速计和陀螺仪等运动传感器的手势识别技术具有实现实用型手语翻译系统的潜力。尽管基于表面肌电和运动传感器的手势识别探索已取得了一定的进展,这种技术距离实际应用仍有较大距离：一方面,当前基于该技术的手语手势识别研究涉及的手语词数量有限,仅为几十种,不能满足日常交流的需求。另一方面,当前的手势识别算法大多在用户有关条件下实现,大词汇量识别下的用户训练负担会限制该技术的实用性。针对基于表面肌电和运动传感器的手语手势识别面临的问题,本文前提出了一种融合表面肌电和加速计、陀螺仪传感器信息的词汇量可扩展手势识别方法,该方法采用手势动作可拆分为多种基本要素的思想,通过较小规模的手势要素识别实现较大词汇量的手势词识别。研究结果显示,本文提出的方案可大大降低大词汇量识别下的用户训练负担,从而可推动实用型手语翻译技术的发展。本文主要工作可概括如下：(1)从手型、朝向、轴向、转向和轨迹五种可有效描述手语手势动作执行特点的手势要素出发,综合分析手势动作执行过程中各要素的变化,提出了一种分段考察手势要素的手势编码方案。(2)在基于要素的手势编码基础上,提出了一种词汇量可扩展的手势识别方案。该方案首先由参考受试者数据确定手势要素的子类,对目标手势集各手势进行要素编码,构建目标手势集编码表。然后,基于表面肌电、加速计和陀螺仪信号特点,对手势要素进行特征提取和分类器设计,并通过手势要素的识别和编码匹配实现较少训练负担下的较大词汇量手势识别。(3)以110个常见中国手语词为目标手势集,在五个受试者(其中Sub3和Sub5分别为参考受试者)数据上验证了本文提出的词汇量可扩展的手势识别方案的可行性。实验结果显示,当训练集规模为目标手势集1/3左右时,目标手势集平均识别率可达(82.6±13.2)%(以Sub3为参考受试)和(79.7-t-13.4)%(以Sub5为参考受试),当训练集规模为目标手势集1/2左右时,目标手势集平均识别率达到了(88+13.7)%(以Sub3为参考受试)和(86.3+13.7)%(以Sub5为参考受试)。
[Abstract]:Sign language gesture recognition is designed to translate gesture action into text or speech form to enhance communication between deaf and normal people. At the same time, gesture recognition technology also plays an indispensable role in the application of human-computer interaction system. From the practical point of view, the sign-language translation device designed to meet the needs of the deaf and mute should meet the basic requirements of portability, recognizable vocabulary of a certain scale, high recognition efficiency and low cost. Gesture recognition technology based on surface electromyography accelerometer and gyroscope has the potential to realize practical sign language translation system. Although some progress has been made in the exploration of gesture recognition based on surface electromyography and motion sensors, this technique is still far from being applied: on the one hand, The current research on sign language gesture recognition based on this technology is limited in the number of hand words, only a few dozen, which can not meet the needs of daily communication. On the other hand, most of the current gesture recognition algorithms are implemented under user related conditions, and the user training burden under large vocabulary recognition will limit the practicability of the technique. In order to solve the problem of sign language gesture recognition based on surface electromyography and motion sensor, this paper proposes a new method of hand gesture recognition based on the fusion of surface electromyography and accelerometer, and the vocabulary of gyroscope sensor. The method uses the idea that gesture action can be divided into many basic elements, and realizes gesture word recognition with large vocabulary by small scale gesture element recognition. The results show that the proposed scheme can greatly reduce the user training burden under the condition of large vocabulary recognition and thus promote the development of practical sign language translation technology. The main work of this paper can be summarized as follows: (1) from the hand type, orientation, axial direction, direction and trajectory, which can effectively describe the characteristics of sign language gesture execution, this paper comprehensively analyzes the changes of each element in the process of hand gesture execution. In this paper, a gesture coding scheme is proposed to examine gesture elements in segments. (2) based on the feature based gesture coding, an extended vocabulary gesture recognition scheme is proposed. Firstly, the subclass of gesture elements is determined by referring to the subject data, and the target gesture set is encoded by each gesture element, and the target gesture set coding table is constructed. Then, based on the characteristics of surface electromyography, accelerometer and gyroscope signal, the feature extraction and classifier design of gesture elements are carried out. Furthermore, gesture recognition with large vocabulary under less training burden is realized by the recognition and coding matching of gesture elements. (3) 110 common Chinese hand words are taken as the target gesture set. The feasibility of the extended vocabulary gesture recognition scheme is verified on the data of five subjects (Sub3 and Sub5 respectively as reference subjects). The experimental results show that when the training set size is about one third of the target gesture set, The average recognition rate of target gesture set was (82.6 卤13.2)% (with Sub3 as reference) and (79.7-t-13.4)% (with Sub5 as reference). When the training set size was about 1 / 2 of target gesture set, The average recognition rate of target gesture set was (88 13.7)% (with Sub3 as reference) and (86.3 13. 7)% (with Sub5 as reference).
【学位授予单位】：中国科学技术大学
【学位级别】：硕士
【学位授予年份】：2016
【分类号】：H026.3

【参考文献】