面向审计领域的聚类离群点检测研究

发布时间：2018-06-04 22:18

本文选题：聚类分析 + DBSCAN　；参考：《哈尔滨工程大学》2011年硕士论文

【摘要】：离群点检测又称小事件检测、偏差检测等。在某些应用中,那些小概率发生的事件往往比经常发生的事件更有趣、更有研究价值。聚类算法实质是将数据集中的数据进行分组,使得同一组内的数据尽可能相似而不同组内的数据尽可能不同。以往的有些聚类算法,只是将离群点检测作为聚类过程的副产品,使一些重要信息丢失；而在离群点挖掘时,只注重离群点检测,不关注数据的分布情况,使得离群点分析难度加大,甚至使一些离群点失去了其实际的应用价值。如将离群点挖掘与聚类分析算法有机的结合,可以更准确的了解数据分布情况。审计方法的好坏直接影响着审计结果的质量,动态监测指标直接影响着审计的时效。传统的审计方法往往是由审计专家的经验和政策法规构建的,这存在着多处不足。利用数据挖掘技术在海量的审计数据中挖出对构建审计方法和提炼动态监测指标提供决策依据的数据,此项工作具有理论和实际意义。本文提出DBSCAN_LOF算法,将DBSCAN的核心对象判定融合到LOF算法中,并重新定义核心对象、增加了k-邻域半径的概念。该算法不但以k-近邻这个概念将聚类算法和离群点算法有机的结合起来,还打破了传统基于聚类的离群点检测算法离群点检测结果受聚类结果的影响、降低了DBSCAN对参数的敏感性及数据空间分布不均匀对聚类结果的影响,并且可以在快速聚类的同时检测离群点。通过多种数据集,比较DBSCAN_LOF算法与原有一些算法在聚类效果和时效上的差异。然后以社会保障审计数据作为实验数据,对审计数据中复合数据类型数据、不同意义的数值型数据的预处理研究；并利用DBSCAN_LOF进行实验验证,实现对审计方法构建提供决策依据数据的挖掘功能。
[Abstract]:Outlier detection is also called small event detection, deviation detection and so on. In some applications, events with small probabilities tend to be more interesting and valuable than those that often occur. The essence of clustering algorithm is to group the data in the data set so that the data in the same group is as similar as possible and the data in the different group is as different as possible. In the past, some clustering algorithms only used outlier detection as a by-product of the clustering process and lost some important information. However, when mining outliers, they only pay attention to outlier detection, and do not pay attention to the distribution of data. It makes the analysis of outliers more difficult, and even makes some outliers lose their practical application value. If outlier mining and clustering analysis are combined organically, the data distribution can be more accurately understood. The quality of audit method directly affects the quality of audit results, and the dynamic monitoring index directly affects the timeliness of audit. The traditional audit methods are often constructed by the experience of audit experts and policies and regulations, which has many shortcomings. The data mining technology is used to dig out the data which provides the decision basis for constructing audit method and refining dynamic monitoring index in the massive audit data. This work has theoretical and practical significance. In this paper, the DBSCAN_LOF algorithm is proposed, which integrates the core object decision of DBSCAN into the LOF algorithm, redefines the core object, and adds the concept of k- neighborhood radius. This algorithm not only combines clustering algorithm with outlier algorithm organically, but also breaks the traditional outlier detection algorithm based on clustering. The sensitivity of DBSCAN to parameters and the influence of data spatial distribution on clustering results are reduced, and outliers can be detected at the same time as fast clustering. The difference of clustering effect and time-effect between DBSCAN_LOF algorithm and some old algorithms is compared by means of various data sets. Then, taking the social security audit data as the experimental data, the preprocessing of the composite data type data and the numerical data of different meanings in the audit data is studied, and the experimental results are verified by DBSCAN_LOF. The function of mining the data of decision-making basis for the construction of audit method is realized.
【学位授予单位】：哈尔滨工程大学
【学位级别】：硕士
【学位授予年份】：2011
【分类号】：F239.4;TP311.13

【相似文献】