Open Access: Pairwise comparisons as a scale development tool for composite measures

Each week, we select a recently published Open Access article to feature. This week’s article comes from the Journal of the Royal Statistical Society: Series A and proposes a method for creating linear indices for aggregate concepts based on pairwise comparisons.

The article’s abstract is given below, with the full article available to read here.

Floridi, G. & Lauderdale, B.E.(2022) . J R Stat Soc Series A1– 24. Available from:
Composite scales are widely used for measuring aggregate social science concepts. These often consist of linear indices obtained as the weighted sum of a set of relevant indicators. However, selecting coefficients (or weights) that reflect the substantive importance of each indicator towards the concept of interest is a difficult task. We propose a method for the generation of linear indices for aggregate concepts based on pairwise comparisons. Specifically, we ask a group of subject-matter experts to perform a series of pairwise comparisons, with respect to the concept of interest, between profiles displaying different combinations of indicators. This allows us to estimate coefficients for each indicator that provide a linear approximation to how experts make the pairwise evaluations. As we show, the method makes it straightforward to assess intercoder reliability, while being a more accessible task than directly asking experts for coefficients. We demonstrate our method with an application to the concept of ‘productive ageing’, including a cross-cultural comparison of weighting schemes derived from a group of Italian and a group of South Korean experts on this concept.
More Details