The Agreement Measure Gamma-cat, a complement to Gamma focused on Categorization of a Continuum

Abstract

Agreement on unitizing, where several annotators freely put units of various sizes and categories on a continuum, is difficult to assess because of the simultaneaous discrepancies in positioning and categorizing. The recent agreement measure γ offers an overall solution which simultaneously takes into account positions and categories. In this paper, we propose the additional coefficient γcat which complements γ by assessing the agreement on categorization of a continuum, putting aside positional discrepancies. When applied to pure categorization (with predefined units), γcat behaves the same way as the famous dedicated measure α, even with missing values, which proves its consistency. A variation of γcat is also proposed which provides an in-depth assessment of categorizing for each individual category. All the family of γ coefficients is implemented in a free software.