Dialog System Using Real-Time Crowdsourcing and Twitter Large-Scale Corpus

Sep 11, 2026

A dialog system is proposed that generates responses by retrieving the most similar utterance-reply pair from a large-scale dialog corpus built from Twitter data and real-time crowdsourcing. Instead of complex dialog management, the system directly replies with database utterances whose similarity matches user input. When no adequate response exists, a realtime crowdsourcing framework delegates the query to other users, addressing low-similarity cases through crowd-based handling.

LanguageEnglish

CategoryPresentations

FormatPDF

Pages5

Reading Time2 min

AccessFree to download

Abstract

Introduction

Method

Overview

Real-Time Crowdsourcing

Similar Utterance-Pairs Retrieval Method

Dialog System Using Real-Time Crowdsourcing and Twitter Large-Scale Corpus Fumihiro Bessho, Tatsuya Harada, Yasuo Kuniyoshi The University of Tokyo Department of Mechano-Informatics 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-8656, Japan {bessho, harada, [kuniyosh](kuniyosh}@isi.imi.i.u-tokyo.ac.jp)[}](kuniyosh}@isi.imi.i.u-tokyo.ac.jp)[@isi.imi.i.u-tokyo.ac.jp](kuniyosh}@isi.imi.i.u-tokyo.ac.jp) Abstract We propose a dialog system that creates responses based on a large-scale dialog corpus retrieved from Twitter and real-time crowdsourcing. Instead of using complex dialog management, our system replies with the utterance from the database that is most similar to the user input. We also propose a realtime crowdsourcing framework for handling the case in which there is no adequate response in the database. 1 Introduction There is a lot of language data on the Internet. Twitter offers many APIs to retrieve or search post status data, and this data is frequently used in research, such as in stock market prediction (Bollen et al., 2011), the spread of information through social media (Bakshy and Hofman, 2011), and representations of textual content(Ramage et al., 2010) . Several models for conversation using Twitter data (Ritter et al., 2010; Higashinaka et al., 2011) have been proposed because of the data’s vast size and conversational nature. Kelly (2009) previously showed that 37% of English tweets are Conversational, of which 69% are two-length (one status post and a reply) . In our analysis of over 2.5 million tweets, 37.5% of all Japanese tweets are Conversational, which matches Kelly’s data. However, less than 58 .3% of these are twolength tweets. Many chat bots are rule-based, which requires alot of human effort to create or add new rules. For example, A.L.I.C.E (Wallace, 2009), which won the U User Utterance Figure 1: Utterance pair. Loebner Prize three times, creates responses based on a dialog strategy database written in a markup language named AIML. Recently, some other chat bots based on a large-scale dialog corpus have been proposed 1,2 . In this paper, we propose a novel dialog system (chat bot) that uses real-time crowdsourcing and Twitter large-scale corpus. We evaluate response selection methods based on positive/negative example to judge if each feature could be exploited to judge similarity between uterrances. 2 Method 2.1 Overview We create an ”Utterance Pair” database as shown in Figure 1 . Each pair is composed of an utterance (Figure 1, A) and a reply to the utterance (Figure 1, B) . Our approach for creating responses is simple and is illustrated in Figure 2 . For each user input, the system searches the utterance-pair database for the pair of which the tweet (Figure 1, A) is most similar to that input. The reply contained in this pair (Figure 1, B) forms the system’s response to the user’s input. 1Jabberwacky: [http://www.jabberwacky.com](http://www.jabberwacky.com) 2Cleverbot: [http://www.cleverbot.com](http://www.cleverbot.com) 227 Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL) , pages 227–231, Seoul, South Korea, 5-6 July 2012 . 􀀍c2012 Association for Computational Linguistics Figure 2: System overview. If the system cannot ﬁnd a post that is sufﬁciently similar to the user’s input, then it ”outsources” the response to another user. To build the conversation database, we collected 1.2 million utterance-pairs from the microblogging service, Twitter. We fetched public timeline data using the Streaming API 3 , and then looked for tweets which were written in Japanese 4 and had an in-reply-to ﬁeld. We followed the replies using the REST API 5. Raw posts from Twitter included mentions (the symbol @ followed by a user name), quotes (written with letters ”RT”), hashtags (a word preceded by the symbol \#), and URLs. We ﬁltered away this information using regular expressions. Unlike English tweets, in Japanese users

FAQs

How does the dialog system generate responses?

For each user input, it searches an utterance-pair database and selects the reply associated with the utterance most similar to the input.

What happens when the system cannot find a sufficiently similar response?

It outsources the response to other users via a real-time crowdsourcing mechanism when the similarity score falls below a threshold.

How is similarity between user input and database utterances computed?

Tweets are preprocessed and analyzed for parts of speech, converted into document vectors using term frequency, and compared using the inner product similarity (xTa x b).

Sep 11, 2026

Uploaded by Caleb Sterling