《Research note: An evaluation of the Elo algorithm for pairwise visual assessment surveys》

打印
作者
来源
LANDSCAPE AND URBAN PLANNING,Vol.157,P.131-137
语言
英文
关键字
Visual assessment methods; Google street view; Elo algorithm; QScore; Win Ratio; Pairwise survey; COMPARATIVE JUDGMENT; LANDSCAPE; PREFERENCES; LAW
作者单位
[Goodspeed, Robert] Univ Michigan, Taubman Coll Architecture & Urban Planning, 2000 Bonisteel Blvd, Ann Arbor, MI 48109 USA. Goodspeed, R (reprint author), Univ Michigan, Taubman Coll Architecture & Urban Planning, 2000 Bonisteel Blvd, Ann Arbor, MI 48109 USA. E-Mail: rgoodspe@umich.edu
摘要
A growing number of online visual preference surveys are conducted which ask respondents to make pairwise comparisons between visual stimuli, raising questions about how to best analyze the resulting data. This research note directly compares three potential methods for analyzing such data, two which have been previously described (Win Ratio and Q Score) and one newly applied to this context (Elo Score). Drawing on a dataset of 103,200 votes among 362 randomly selected streets in Philadelphia, the note finds that although the three metrics are highly correlated overall, the Elo Score results in more dispersed standardized scores and a somewhat different rank order than the Win Ratio (Kendall's tau b 0.864). Although the Win Ratio may be adequate for some research designs, the Elo Score provides a more discriminating measure for images that elicit strong responses. The note discusses how studies using pairwise comparisons might incorporate respondent demographics, and recommends further investigation of respondent demographics for online preference surveys. (C) 2016 Elsevier B.V. All rights reserved.