Tests Of Agreement

Krippendorffs alpha[16][17] is a versatile statistic that evaluates the concordance between observers who classify, evaluate, or measure a certain amount of objects relative to the values of a variable. It generalizes several specialized conformity coefficients by accepting any number of observers, applicable to nominal, ordinal, interval and proportional levels, capable of processing missing data and being corrected for small sample sizes. Statistics κ can take values ranging from – 1 to 1 and are arbitrarily interpreted as follows: 0 = concordance equivalent to chance; 0.10-0.20 = light match; 0.21-0.40 = fair agreement; 0.41-0.60 = moderate concordance; 0.61-0.80 = essential correspondence; 0.81-0.99 = almost perfect match; and 1.00 = full compliance. Negative values indicate that the agreement respected is worse than what is expected by chance…