BBitMonkey AI
Home / Case studies / P10
P10Applied Prediction and Automation

Expanded Package

Complaint classification

Sign in to open this case study.

Sign in

The case

Nagrik Seva, a city corporation's grievance cell, receives 11,795 complaints a month and a clerk reads every one to decide which of eight departments it belongs to. Two clerks given the same complaints agree 80.8% of the time. Can a classifier do this, and what does 'right' mean when the humans disagree?

What is in it

  • 13 teaching steps, one animation each
  • 26 labs, checked live against the case data
  • 65 step questions, every option explained
  • 20 final paper questions, marked on submission

The steps

  1. 1Eight departments, one clerk
  2. 2The clerks disagree
  3. 3How the classifier decides
  4. 4One number is not enough
  5. 5The confusion matrix
  6. 6Some errors are not administrative
  7. 7Where the errors actually are
  8. 8Knowing when to stay silent
  9. 9One threshold, or eight?
  10. 10Worse than a clerk, and worth deploying
  11. 11The Other category
  12. 12Watching it in a changing city
  13. 13What the corporation owes the citizen