Towards Quality Assessment of Crowdworker Output Based on Behavioral Data
| 言語 | 英語 |
|---|---|
| 種類 | 国際会議等の予稿 |
| 著者 | Shigeaki Yuasa，Takumi Nakai，Takanori Maruichi，Manuel Landsmann，Koichi Kise，Masaki Matsubara，Atsuyuki Morishima |
| 論文名 | Towards Quality Assessment of Crowdworker Output Based on Behavioral Data |
| 論文誌名 | Proc. of The Third Human-in-the-loop Methods and Human-Machine Collaboration in BigData, IEEE Big Data |
| ページ数 | 3 |
| 査読の有無 | 有 |
| 発表の種類 | 口頭発表 |
| 年月 | 2019年12月 |
| 分野 | クラウドソーシング, 行動認識, センシング |
| キー | Yuasa2019 (id=1706) |
要約
In this paper, we show preliminary results on the quality assessment of crowdworker output based on the movements of the mouse and the eyes while the task is performed. We assume that the mouse and the eyes stop longer if the quality is lower due to the lack of knowledge, or confidence, etc. Because the mouse- and eye-stopping duration follows log-normal distribution, we estimate its parameters (mean and standard deviation) to evaluate the quality. Results of preliminary experiments with 10 participants show that the parameters of correct outputs are different from those of incorrect ones. As compared to the task duration, which is often used as a feature for assessment, we have found that the mouse- and the eye-stopping duration is advantageous and complementary for the assessment.
関連ファイル
- …
この文献を出力
書式の定義
書式の定義の書き方
{authors} のように波括弧でフィールドを挿入します．[ ... ] で囲んだ部分は，その中のフィールドが空のとき丸ごと省略されます．使えるフィールド：
{authors}{authors_and}{authors_ja}{authors_ja_tight}{authors_en}{authors_en_and}{authors_tight}{first_author}{title}{title_ja}{title_en}{journal}{journal_ja}{journal_en}{book_title}{volume}{number}{series}{presen_id}{pages}{pp}{num_pages}{year}{month}{month_en}{month_short}{ym}{date}{date_ja}{date_en}{publisher}{address}{location}{review}{review_ja}{review_en}{reviewed_mark}{presen_type}{type}{type_label}{doi}{url}{key}{awards}