"Variation and Imprecision of Clerkship Grading in US Medical Schools” is the understated title of the paper (full text here) in the August 2012 issue of the journal Academic Medicine. The authors, from the department of medicine at Brigham and Women’s Hospital, analyzed 2009-2010 third-year clerkship grades from 119 (97%) of the 123 US medical schools. They found many different grading systems ranging from two levels (pass/fail) to 11 levels of grades. Skeptical Scalpel is a recently retired surgeon and was a surgical department chairman and residency program director for many years. He is board-certified in general surgery and a surgical sub-specialty and has re-certified in both several times. For the last two years, he has been blogging at SkepticalScalpel.blogspot.com and tweeting as @SkepticScalpel. His blog averages over 900 page views per day, and he has over 5600 followers on Twitter.
The terminology used by the schools to describe the different grades is positively comical. To borrow an analogy I’ve used in a previous blog about dean’s letters, the citizens of Lake Wobegon would be proud because no student is “average.”
Here are some examples:
High honors, honors, pass, fail (In some schools “honors” is not the highest possible grade).
Honors, satisfactory plus, satisfactory, fail.
Honors, satisfactory, low satisfactory, fail.
Honors, high satisfactory, satisfactory, low satisfactory, unsatisfactory. (Does “unsatisfactory” mean, dare I say it, “fail”?)
Honors, near honors, pass, fail.
Excellent, good, fail.
Honors, advanced, proficient, fail.
Honors, letter of commendation, fail.
The highest grade attainable was awarded to 23% of those students in schools with three-tiered systems (range 5-51%), to 34% (range 2-84%) in four-tiered systems and to 33% (7-93%) in schools with five grade levels.
It gets worse. The authors noted that 97% of all medical students were given one of the top three grades regardless of whether the schools used 4, 5, or 6 levels of grading.
From the paper, “Less than 1% of all US medical students fail required clerkships, regardless of the grading system used.” This raises the question of whether the grade “fail” is even necessary.
Focusing on surgery, an average of about 30% of all students got the highest grade possible in their surgical clerkship, but the percentage of the class receiving the top grade ranged from 7% to 67%. This may account for the paradox found in a paper on surgical resident performance: A significant predictor of the need for remediation was that the resident had received honors in his surgical third-year clerkship. It appears that a grade of honors is virtually meaningless.
This is an excellent example of what I call the “T-ball culture”: No one keeps score. All games end in a tie. Everyone gets a trophy.
The authors of the paper recommended that schools consider creating a more consistent, transparent and reliable system of grading. As a former surgical residency program director who grappled with the difficulty in interpreting the meaning of applicant grades from different schools, this seems remarkably clear to me.
An editorial in the same issue of the journal agreed that grade terminology should be standardized but cautioned that normative grading (establishing a set distribution or “curve” of grades) may not be the answer. The editorialists offered some other possibilities such as criterion-based grading or emphasizing the mastery of a subject as a goal rather than the achieving of a specific grade.
I do not have the background in educational theory to say what is right or wrong. I do know that a grading system with so many variables and such a skewed distribution is of no help whatsoever in evaluating the desirability of an individual applicant to a residency program.