Advanced Analytics · TOOL GUIDE

Inter-Rater Reliability

Open Qantly ↗

Understand the method. Use it with confidence.

Measure agreement between 2 raters with Cohen's kappa or 3+ raters with Fleiss' kappa. Qantly runs the method against your active analysis context and returns structured results for review and interpretation.

01

When to use it

Use this tool when the question requires multivariate structure, segmentation, simulation, causal-design assumptions, or model explainability.
Use it only when the selected variables and study design match the method's requirements.

02

How to use it

  1. Open the tool from the Qantly Tools panel.
  2. Confirm the active dataset and analysis session.
  3. Set Rater Columns (Required): Select the categorical rating columns to compare.
  4. Set Use Weighted Kappa (Optional): Apply weighted Cohen's kappa for 2-rater ordinal ratings.
  5. Set Weighting Scheme (Optional): Choose the weight function for weighted Cohen's kappa.
  6. Review optional or advanced settings and keep defaults unless your design requires a change.
  7. Run the tool, inspect warnings and diagnostics, then save or export the result with its assumptions.
03

How to interpret results

Interpret results in the context of preprocessing, tuning choices, stability, validation, and domain knowledge. Complex outputs are sensitive to assumptions and should be checked with alternative specifications.