This Article Full Text (OnlineFirst[PDF]) All Versions of this Article: 1525822X08317166v1 20/3/272    most recent Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Google Scholar Articles by De Vries, H. Articles by Teleki, S. S. Social Bookmarking                   What's this?

Using Pooled Kappa to Summarize Interrater Agreement across Many Items

^* To whom correspondence should be addressed. E-mail: hdevries{at}rand.org.

	Abstract

We propose the pooled estimator of kappa, an efficient estimator when summarizing the interrater agreement for qualitative data with many items but few subjects. We evaluate this estimator through a simulation of proposed and alternative (average kappa) estimators and subsequently apply our method to calculate pooled and average kappas over 2,176 rated items from six semistructured interviews with sponsors of the CAHPS. The proposed pooled kappa estimator efficiently summarizes interrater agreement by domain. It is more widely applicable and makes better use of scarce subjects than simply averaging item-level kappas.

CiteULike    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?