An Empirical Study on Document Similarity Comparison Evaluation Between Machine Learning Techniques and Human Experts - Evaluation metric and vision propagation measurement (VPMS) - Machine Learning vs humans

Aug 3, 2026

Upload by Liam

Focuses on document similarity comparison beyond pure accuracy in machine learning. Examines how machine-learning decision behavior aligns with human judgment across different domains, and evaluates whether organizational vision is effectively propagated to lower organizational levels. Uses word representation methods including count-based models (Bag of Words, TF-IDF), ANN models (Word2Vec, GloVe), and a VPMS approach that combines two methods. Compares model outputs and expert-group results from multiple perspectives, proposing a VPMS model that outperforms existing methods. Results support evaluation of vision dissemination and can inform text-based algorithms like customer segmentation for target marketing.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages12

Reading Time30 min

AccessFree to download

Introduction

Background and motivation in the 4IR era

Document similarity as an evaluation indicator

Human-AI cooperation and the limits of accuracy-only training

Methodology (models and representations)

Count-based representations: Bag of Words and TF-IDF

ANN representations: Word2Vec and GloVe

VPMS model combining methods

Evaluation

Expert-group comparison

Multi-perspective assessment of similarity decisions

Results and contributions

Empirical comparison across ML techniques vs experts

VPMS model performance and implications

Applications

Vision propagation evaluation

Text-based targeting and customer segmentation

ISSN 1330-3651 (Print), ISSN 1848-6339 (Online) [https://doi.org/10.17559/TV-20231011001013](https://doi.org/10.17559/TV-20231011001013) Original scientific paper An Empirical Study on Document Similarity Comparison Evaluation Between Machine Learning Techniques and Human Experts Won-Jung JANG Abstract: Current machine-learning training focuses solely on accuracy. In this study, the weights of other dimensions were examined rather than measuring only the accuracy of machine learning. By comparatively analyzing the decision-making of machine learning and humans in various fields, this study examines how well organizational vision is propagated to lower levels of the organization. Also, the results evaluated by humans and machine learning models were comparatively analyzed from multiple perspectives. As numerical representation methods of words, count-based models (Bag of Words, TF-IDF), artificial neural network (ANN) models (Word2Vec, GloVe), anda vision propagation measurement (VPMS) model combining two methods were used to calculate the similarity between documents, which are comparatively analyzed with the actual results measured by an expert group. The findings of this study can be used as an evaluation metric for how effectively the vision of the upper organization is being disseminated to the lower-level organizations. Additionally, it could be utilized in developing algorithms such as customer segmentation for target marketing using textdata. The study makes two key contributions- (i) providing an extensive empirical comparison of document similarity analysis by different ML techniques versus human experts, and (ii) proposing a new VPMS model that outperforms existing methods. Keywords: ANN model; count-based model; document similarity; ensemble learning model; machine learning 1 INTRODUCTION In the era of the 4IR (Fourth Industrial Revolution), personalized customer experience through big data-based value enhancement can have a significant impact on corporate profits. Data generation is expected to increase significantly owing to advancements in communication technology, such as the Internet and smartphones. According to an IDC report, 16.1 ZB (zettabyte) of digital data generated in 2016 will increase by 10 times in less than 10 years, while the rate of increase will also go up [9] . The Ministry of Science and ICT of South Korea reported that the data market size was 14 trillion won in 2018 andwill expand to 30 trillion won by 2023 based on relevant policies and programs that foster global competitiveness in related industries, such as data and artificial intelligence (AI) . The enormous net worth of Facebook is attributable to the large and unique data assets of the company [29] . Most companies are focusing on strengthening their AI-based products and service competitiveness, and securing big data based on AI has become crucial to improving competitiveness. In particular, increasing the competitiveness of a company using unstructured data is important because of the large volumes of data generated. From the technological advances in personal computers (PCs) in the 1990s to those in the Internet (2000) and smartphones (2007), digitized information and objects have become interconnected. In the digital era, the success of a company is determined by the efficiency of these connections. Companies such as Yahoo and Google are typical examples of these success stories. Therefore, searching for accurate information easily is critical for the survival of companies in the digital era. Finding an entity that quickly and easily provides information with a high level of similarity is an important issue. Because individuals communicate and connect with each other using documents, which are typical text data and may also be used for important decision-making, research based on documents is emerging as a significant research topic. Studies on developing algorithms to quickly and easily find accurate documents have

FAQs

What problem does the study address regarding current machine learning training?

It argues that many machine-learning efforts optimize only accuracy, without examining other dimensions that shape decision behavior and alignment with human judgment.

Which document similarity representations and models are used in the study?

The study uses count-based models (Bag of Words, TF-IDF), ANN-based models (Word2Vec, GloVe), and a VPMS model that combines two methods to compute document similarity.

How are the machine-learning results validated in the paper?

Model outputs are comparatively analyzed against results measured by an expert group, and the comparison is evaluated from multiple perspectives.

Aug 3, 2026

Upload by Liam