Each year, US News and World Report publishes its list of the top 50 hospitals in various specialties (example here). Now, an article has been published suggesting that one aspect of the methodology used by the magazine is flawed.
"The Role of Reputation in U.S. News & World Report’s Rankings of the Top 50 American Hospitals," by Ashwini R. Sehgal, MD is in the current edition of the Annals of Internal Medicine. (You can find an abstract here, and you can obtain a single copy for review from Dr. Sehgal by sending an email to axs81 [at] cwru [dot] edu.)
Dr. Sehgal finds that the portion of the U.S. News ranking based on reputation is problematic because reputation does not correlate with established indicators of quality:
The relative standings of the top 50 hospitals largely reflect the subjective reputations of those hospitals. Moreover, little relationship exists between subjective reputation and objective measures of hospital quality among the top 50 hospitals.
More detail is provided in the article:
The predominant role of reputation is caused by an extremely high variation in reputation score compared with objective quality measures among the 50 top-ranked hospitals in each specialty. As a result, reputation score contributes disproportionately to variation in total U.S. News score and therefore to the relative standings of the top 50 hospitals.
Because reputation score is determined by asking approximately 250 specialists to identify the 5 best hospitals in their specialty, only nationally recognized hospitals are likely to be named frequently. High rankings also may enhance reputation, which in turn sustains or enhances rankings in subsequent years.
Given the importance attributed to the U.S. News ranking, this article is bound to raise concerns. I know that the folks at the magazine have worked hard over the years to make their rankings as objective as possible, and it will be interesting to see their response to Dr. Sehgal's critique.