Free Cohen's Kappa Calculator
Use this online Cohen's kappa calculator to measure inter-rater reliability for two raters classifying the same items into nominal categories.
Enter counts for each cell in the confusion/agreement matrix. Rows = Rater A, Columns = Rater B.
What Is Cohen's Kappa?
Cohen's kappa (κ) is a statistical measure of agreement between two raters who classify the same set of items. Unlike simple percent agreement, kappa adjusts for chance agreement. This makes it a much better indicator of true consistency in coding, diagnosis, scoring, annotation, or labeling tasks.
If two raters agree often only because one category is very common, percent agreement can be misleadingly high. Kappa helps correct that bias by comparing observed agreement with expected agreement by chance.
How This Online Kappa Calculator Works
This tool uses the standard unweighted Cohen's kappa formula for nominal categories:
where:
Po = observed agreement (diagonal total / grand total)
Pe = expected agreement by chance (sum of row-total × column-total across categories, divided by N2)
- Observed Agreement (Po): How often raters actually agree.
- Expected Agreement (Pe): How often they would agree by chance based on marginal totals.
- Kappa (κ): Chance-corrected agreement coefficient.
How to Use the Calculator
Step-by-step
- Select the number of categories.
- Click Build Matrix.
- Enter counts in each matrix cell (rows for Rater A, columns for Rater B).
- Click Calculate Cohen's Kappa.
You will get observed agreement, expected agreement, kappa value, and an interpretation label.
Interpreting Cohen's Kappa Values
A common interpretation framework (Landis & Koch) is:
- < 0.00: Less than chance agreement
- 0.00 - 0.20: Slight agreement
- 0.21 - 0.40: Fair agreement
- 0.41 - 0.60: Moderate agreement
- 0.61 - 0.80: Substantial agreement
- 0.81 - 1.00: Almost perfect agreement
Interpretation should always be considered in context (sample size, category imbalance, and domain consequences).
Worked Example
Suppose two reviewers classify 100 articles into three categories. After entering the matrix, the calculator may produce values like:
- Po = 0.81
- Pe = 0.3344
- κ = 0.7145
This indicates substantial agreement beyond chance. In practical terms, the raters are fairly consistent and the coding scheme is likely reliable.
Common Mistakes to Avoid
- Using kappa when categories are ordinal but requiring weighted kappa instead.
- Entering percentages instead of raw counts.
- Combining categories inconsistently across raters.
- Interpreting kappa without checking class imbalance.
FAQ
Is Cohen's kappa the same as percent agreement?
No. Percent agreement does not account for chance agreement. Cohen's kappa does.
Can kappa be negative?
Yes. A negative value means agreement is worse than expected by chance.
Can I use this for more than two raters?
No. Cohen's kappa is specifically for two raters. For more raters, consider Fleiss' kappa or Krippendorff's alpha.
Final Thoughts
If your work depends on reliable human judgment—research coding, clinical ratings, QA review, content labeling, or annotation pipelines—Cohen's kappa is one of the most important agreement statistics you can track. Use this calculator to quickly test consistency and strengthen your decision-making process.