The Mathematics of Residency Interviews

This year I have just completed my first season of interviewing applicants to the internal medicine residency program at my University hospital-based program. It's something I always wanted to do, and after living through the interview process on the other side of the coin I'm inclined to help out in the years to come. My guess is that it's more fun than medical school interviews, where the stakes are higher and the candidates more nervous by virtue of the fact that everyone applying for residency is going to get in somewhere, it's just a question of where. In internal medicine especially (and doubly so since I don't work at a Hospital whose first name is "Massachusetts General"), we're trying to vie for their love as much as the other way around. So I put on a little salesman schtick at the same time I'm trying to appraise the person before me. And most of the time it's quite fun.

That said, my impressions count for about two percent of where soon-to-be-Doctor so-and-so is going to end up on the rank list. Bob Seger may have complained that he felt like a number, but we'll assume that he wasn't singing about the process of becoming a doctor, however accurate his assessment would have been. In order to become a doctor in this country, one must become a number. The only question is: how high is it? The answer to that determines whether you will become a dermatologist or family practice resident, or whether you will cruise the halls in your snappy white coat at Johns Hopkins, or at Southern Podunk Community Health Center.

The Number for our residency program is, I suspect, calculated in a manner similar to other programs of its ilk around the country. For example, you get a certain number of points for having certain kinds of grades. In most medical schools there are three grades for the "clinical rotations," that is, when you actually see patients: "honors," which is like an "A"; "high pass," which is like a "B"; and "Pass," which is like a "C." (It is rare to fail once you've gotten that far in med school; for more on this feel free to read Blind Eye by James Stewart, but have a bottle of booze at the ready.) So, if you had, say, honors for your internal medicine grade, you'd get a 3, but if you got a high pass, you'd get a 2, and so on. Since our program is internal medicine, we have a category for the general internal medicine rotation, a category for the more intensive internal medicine rotation known as a sub-internship, and a category for all the other clinical rotations put together. Thus, if you had honors in both internal medicine and in the sub-I, but only a spotty showing in all your other classes, you'd end up with an 8...not too shabby, in fact.

Anyway, we have other scoring categories for things like research and leadership positions, how well you did on the nationalized standardized medical examinations, and so on, and the total number of points one can have, if they were a perfect candidate in our system, is somewhere in the high thirties. The interview accounts for four of these points, and each candidate interviews with two different faculty members. Thus, you might make a killer impression, but it won't save you if you didn't do so well in those other categories; your number is more or less fixed before you've even shaken my hand. (On the other hand, if you interview badly it will count against you in a most unpleasant way: say or do something offensive and there's a special category marked "Red Flags," reserved for people who do something socially inappropriate during the interview day. Red Flags will pretty much put you at the bottom of the rank list, or remove you from it entirely. Don't make ethnic jokes, students!)

Much could be said about this process but in the interests of time I will say that, in general, it appears that it works for the majority of candidates in determining who's going to be a decent match for a program. It won't really help you learn who is going to be very good, nor will it help you identify who is going to drive the medicine faculty bananas, but it will do just fine for the average, solid resident. (The truly stellar, as well as the truly awful, are pretty easy to spot; I'm talking about the residents who come in with the same grades as their classmates and turn out to be future Chief Residents. These guys & gals are hard to find in this kind of application process, and I'm not sure how you'd spot them in a time-efficient manner regardless.)

But grades can translate to interesting numbers, and the interpretation of the former can very much influence the assignation of the latter, which in turn can make the difference between whether a given candidate will end up at Man's Greatest Hospital or McGrungy State City Hospital. Let's take, for example, the grades of this (anonymous) student: high passes in most clinical rotations (internal medicine, surgery, psychiatry, and family medicine) and passes in two (OB/GYN and pediatrics). The student's "overall" recommendation by the medical school is "strongly recommend," which means that they regard this student as being in the second highest of four categories ("enthusiastically recommend," "strongly recommend," "recommend with confidence," and finally, with what approximates a straight face in medical school bureaucratise, "recommend").

Let's take a closer look at this student, though. When you consider this student's grade in relation to those of the student's classmates (I would like to show the chart, but have privacy concerns for the student, and have altered the classes to maintain that privacy), you discover that in some cases the numbers are damning, and in others they provide no useful information at all. For instance, this student got a "high pass" in psychiatry, but roughly two-thirds of the students got a high pass, and virtually nobody got a mere "pass." Thus an "honors" is useful information (top third of class), as is a "pass" (this student stinks), but not so much for the middle category. All of this students high pass grades have a similar flavor; they tell you that the student was competent, but don't really indicate if he or she was a cut above--the original intent of the high pass designation. The distribution of the OB/GYN grades is about what it should be: top 10-ish percent got honors, a bit less than 30 percent got high pass (a bit too large, but closer to the mark), and the rest got pass...so this student is average, what you'd expect of a "pass" grade. The pediatric grade, by contrast, indicates something more concerning: due to the grade inflation only twenty percent got a pass--that is, the bottom twenty percent, more like a "D" than a "C". Not so hot! Now that overall "strongly recommend" status looks a touch fishy to me and seems inflated. And sure enough, almost half of this class is given this label. If I were a better than average but not outstanding med student, I'd feel ripped off by this school.

Balancing these variables is part of the art of ranking, but no matter how you slice the bread, someone's going to end up higher on a given program's rank list due to some medical school's grade inflation, while others will inadvertently get dinged. I see no easy way around this except to give greater weight to national standardized tests. Unfortunately, they only speak to how well a person knows how to take tests, and perhaps how much "book medicine" they know. It won't tell you a lick about how good a doc they'll be.

The rank list meeting, where faculty try to jockey position for certain pet candidates (though my understanding is that the bumps are very minimal and most people's rank order is pretty well fixed), is taking place in a few weeks. Maybe sometime in July or August, after I've had a chance to make a survey of the new bunch, I'll drop a line about how things turned out.
--br