Wednesday, October 23, 2013

Not leaping ahead

It is so striking that hospitals are keen to accept and publicize the results of the fairly meaningless US News and World Report hospital rankings--rankings that have no statistical validity and are based in part on rumors about the quality of care delivered--and yet complain bitterly when the Leapfrog Group posts scores based on data about preventable medical errors and injuries.  The scores revealed “little improvement in safety overall” since the last report.

In American life, the three great lies are (1) "The check is in the mail;" (2) "I'll still respect you in the morning;" and (3) "I'm from the government and I'm here to help you."

In the hospital world, the two great lies are (1) "Your data are flawed" and (2) "Our patients are sicker."

This article, though, contains a new rebuttal approach:

Jeff Dye, president of the New Mexico Hospital Association, fired back at the data, saying many of the state's hospitals have stopped participating in the Leapfrog survey because they “see it as extortion to obtain a higher score.”

Perhaps someone can explain what that actually means.  Heaven forbid that a hospital's score would improve.


Leah Binder said...

Just for Mr. Dye's information: Typically you can't stop doing something you never did in the first place. Most New Mexico hospitals have never participated in Leapfrog, which is too bad, because many hospitals find it jump-starts quality improvement to do so. NM needs that.

But the data from CMS speaks for itself: NM hospitals lag behind the rest of the country on safety, and residents of NM deserve better.

Paul Levy said...

Well put!

Anonymous said...

I've heard people in hospitals say they resent paying Leapfrog Group to be included in their survey.

Well, somebody needs to be overseeing hospitals (arguably, The Joint Commission has done a poor job). I guess it would be better if Leapfrog Group had no financial connections with the hospitals, ala Consumer Reports?

My concern is that the "A" hospitals are probably still hurting and killing patients. Is the grading on a curve?

Paul Levy said...

Hospitals pay The Joint Commission to do their surveys.

Hospitals pay US News for advertising space in their "best of" edition. I think they may also have to pay for use of the US News logo in their own advertisements, but I am not sure.

Erica Mobley said...

Hi Anonymous,

Just FYI- hospitals don't pay anything to Leapfrog to participate in the Leapfrog Hospital Survey or receive a Hospital Safety Score. The Survey and Score evaluation are entirely free to hospitals.

Leapfrog has to protect its brand, so it does charge a small licensing fee to hospitals that wish to use the Hospital Safety Score or A grade logos in commercial advertising. Hospitals can still announce their grade and issue a press release about the designation without paying anything, and hundreds of hospitals have done so.

Paul Levy said...

Thanks for setting that straight, Erica.

Anonymous said...

Ok, I guess that's what they were referring to, thanks.

I know they resent having to pay TJC for what they do, for that "help."

Anonymous said...

Paul - Expanding on the "our patients are sicker" line of thought.... there is a MASSIVE game of upcoding CPT and ICD-9 codes on billing documents by providers and provider networks in an effort to profile their patients as "sicker" which results in higher top line reimbursements from health plans. So a network will bill and code a diabetic with complications as opposed to a "regular" diabetic... the patients may be similar in health status but the coding with complications results in a LOT more money in their medical expense budget going forward. There are whole companies that work with provider networks to play this game to get more money for a population of patients. It leaves networks that are not playing these coding games with a LOT less money in their budgets and really, at the end of the day, does not act to allocate more money to networks with "sicker" patients but more to networks that code their sick patients better than the other guy. A dirty little secret that results in the higher capitalized networks getting even more money into their systems as this coding game is expensive in terms of technology, personnel and consultants.

Anonymous said...

I tend to believe that some leapfrog group scoring isn't based on results but their opinion on resource inputs which is not necessarily a good measure in my view.

For instance, ICU staffing. What should be evaluated is ICU mortality, infections, complication rates etc etc, adjusted for case mix severity.

Whether someone has X nurses per patient isn't important if the desired outcomes can be achieved without meeting leapfrogs opinion about resources available in the ICU.

This argument is similar to Nurses unions trying to achieve certain staffing levels, saying you need X nurses per patient for safety. But if a hospital can reduce scut work for nurses by have more orderlies, and other help and thereby help nurses focus their time on "medicine" and not "cleaning/busy work"....that should be a desirable outcome not a negative.

Leah Binder said...

I think we'd all prefer to have good data on mortality, complication rates, and infections. But there are real limits to what is publicly available to us, so we use the best measures our expert panel recommend in the public domain. We also work hard to advocate for more transparency and better measures, so we hope you will join us in that effort.

We look at some measures of whether the hospital has in place the structures and systems that are known to prevent errors. That's why one of the measures we use in the safety score concerns intensivist coverage in the ICU. Studies suggest this can reduce mortality by upwards of 40%. It would be better to have the actual adjusted ICU mortality rate, but we aren't waiting for the perfect measure when we have a good measure. There's too much at stake for patients choosing a hospital now to wait for something later.

We also look at other evidence-based safe practices, including nursing workforce as you mention. Our nursing workforce standard does not include staffing levels, but looks at nursing leadership and other issues. Only hospitals that report to Leapfrog can have this considered for their Safety Score (otherwise it has no impact plus or minus on their score). I believe that nurses are critical to a safe hospital and wish we had more measures and public reporting on nursing engagement.

Anonymous said...

Leah Binder

In my view the focus should be on results -- not resources available or particular systems in place.

Because what some might think is necessary, can be shown to not be. A good example, "repetitions of particular procedures". For instance, how many gall bladders are being taken out by, or how many hematology patients are seen at a particular local hospital.

For a long time it was believed "high volume" hospitals for a particular procedure always achieved the best results.

Admittedly I am not sure if Leapfrog had this as a standard or if it was another similar organization.

But it has since been shown that there are other means for doctors and organizations achieve that same quality, while having limited "repetitions" in a particular local community hospital.

A good example of this is Commonwealth Hematology & Oncology (CHO) a practice now affiliated with Paul's old employer BID. CHO has only limited volume in many sites, but doctors practice in multiple sites and CHO also have organizational learning encompassing the whole network. It's high quality results made it a suitable partner for BID.

So where in the past it was believed that certain types of care could only be done in Academic Medical (read - high cost) or other high volume settings, now it is clear that many more procudures can safely be kept in community settings.

When resource availability is the standard - it partially stifles innovative new approaches (especially efforts to lower cost) because organizations don't want to be labelled "low quality" by reputable organizations like Leapfrog.

For these reasons Leapfrog should keep the focus on results not inputs.

Anonymous said...

Leah Binder

Another area receiving lots of attention is hospital readmissions.

But is the focus on readmissions a good one?

Many researchers at Harvard School of Public Health and Tufts Medical Center - two of the four major contractors for ECRI have said focusing on readmissions can be misleading especially for tertiary and quaternary care.

The sickest patients with multiple simultaneous health issues are often referred to "hospitals of last resort". Someone might be admitted for cancer treatment but also have other distinct health issues, lets say heart disease, high blood pressure and diabetes (and the side effects of these conditions, eg, eye problems from diabetes etc). No one wants to treat a patient like this. They are a "walking readmission".

The best hospitals of last resort will happily accept this patient but then be penalized both financially by CMS and by quality ratings systems like leapfrog. A system that penalizes hospitals who accept these patients in a situation like this is doing a disservice to these patients and "hospitals of last resort". Something needs to be changed both by CMS and by statistical organizations like Leapfrog.

It is too easy to "game" the system for those inclined to do so.

Leah Binder said...

Anonymous, why art thou Anonymous?
You have important points to make and not sure why you are hiding your identity.

You can read the entire Leapfrog Survey online
You will see that our standard on surgical outcomes is more nuanced than the old days when we had to rely solely on volume. We also rely on outcomes, and the standard has been peer-reviewed.

How would you advise us to rate hospitals on readmissions? Because it's a big problem, and some hospitals are not doing the best for their patients--so how do we get at that? Many measures are imperfect, but some level of transparency is critical to those of us who entrust our lives to hospitals.

Anonymous said...

Leah Binder -

I am not a health care professional or administrator - so you wouldn't know me. Call me a knowledgeable amateur with a background in business [MIT Sloan]. Worked in the electronics industry when applying operational methods like "lean and six sigma" was recent there.

My interest:

-- I Believe that controlling & reducing health care costs (while greatly improving quality) is critical to future health of the U.S. economy, and governmental institutions at all levels - or we slowly bankrupt ourselves.

For improving readmissions stats, I would defer to Harvard School of Public Health and Tufts Medical Center experts. I do not know enough about the specifics of the situation to make a detailed recommendation.

But, obviously more adjustment must be made for "high case mix" and tertiary/quaternary hospitals according to press reports I have seen from organizations like MGH, BID and Tufts Med.

I am not saying improvements haven't been made by Leapfrog (and your peer organizations). Quite the contrary. But more can be done.

I am commenting because the information you provide can and should influence consumers of health care and health professionals. I want to be sure the "signals" being sent result in a more market based approach that controls and lowers cost.

We don't want consumers selecting hospitals just because they spend more if there isn't a measurable improvement in health outcomes.

We also want to incentivise. hospitals executives to experiment to find ways to improve (principles of six sigma etc) . If we provide the answer - only measure inputs, that doesn't happen. If we focus on outputs there is much more room for creativity and breakthroughs.

This is as opposed by a race for more resources and higher costs for consumers and payers if the focus is more inputs.

I commend you personally for taking the time to see what is said in blogs like this.

As I said toward the beginning, my interest is in finding ways to control and lower health care costs (while improving quality). By focusing quality measures on "inputs" you continually stack the deck in favor of institutions with large asset bases (like rich academic medical centers) and make it a race for who can "spend MORE" - and thereby drive up health costs. That road eventually leads to oligopoly or monopoly.

What we want is a race to "spend BETTER" - and move towards real market competition and a focus on the best product at the lowest possible cost, as we see in all real markets. By focusing on outcomes you have a better chance of fostering this.

Wish I could have said it better, but I hope you know what I mean.

Because you asked...I can be reached at

Anonymous said...

The criticisms directed at Leapfrog by some of the poorly rated hospitals - pointing out awards and high ratings from other organizations - reminds me of the old parable about the blind men and the elephant. Certainly a limited set of measures can lead to a distorted result, and certainly there is a movement from process measures more toward outcome measures. But the fact remains, if you wait for perfect measures then many years and many lives will be lost. One must trust that Leapfrog and the other measuring organizations are devoting extensive resources to researching and evolving better measures on a constant basis. I wonder if Ms. Binder could elaborate on that (or perhaps I missed it in the comments above)

nonlocal MD

Leah Binder said...

Dear Nonlocal MD,
Excellent point. While Leapfrog is not a research organization, we partner with organizations such as the Johns Hopkins Armstrong Institute (run by Peter Pronovost) and other leading researchers in safety, and we are strong advocates for better measures, especially more outcomes measures.Here in Washington, we devote a good deal of resources to National Quality Forum, CMS, and other policy efforts to improve measurement and public reporting.

One issue Leapfrog focuses on that we think hospital and provider lobbies should support us on, is for more frequent updating of data from CMS. In an era of Big Data, it's silly that CMS updates most of their measures once a year--and then it's already a year or two old.

Anonymous said...

I agree 100%. More frequent updates of data from CMS (and others) is necessary for real price competition in healthcare to develop.

We need good outcomes data as close to real time as possible combined with pricing data (as is beginning to happen in Massachusetts).

When health purchasers and consumers can make good cost benefit analysis decisions, "around the kitchen table", then we will begin to see the power of markets to control and reduce health costs.


Anonymous said...

Leah Binder,

You mention that your statistical partner is Johns Hopkins. Let me admit that I know little about Hopkins approach to quality and cost in Medicine. And nothing about Peter Pronovost.

But I have read that Maryland has long had cost controls for Medicine in their state. Massachusetts in contrast tracks health cost increases but doesn't actually control rates.

I believe I can comfortably say that Governor Romney's health proposal was designed to try to foster competitive market forces in contrast to Maryland's more bureaucratic top down approach. President Obama chose to use Romney's plan as his template. If the Obama administration is truly trying to foster price competition (I believe they are), they you might keep that in mind when developing your statistics.

Again, I reiterate, I don't know what Mr Provnost, nor Johns Hopkins approach to controlling costs might be. Maybe they also want to foster real price competition as was the original goal in Massachusetts.

If Mr Provnost's group takes a non-market view, as per Maryland's historical price controls you might want to reexamine what incentives your quality statistics are fostering.

Again, I think the focus should be outcomes for the reasons I have stated earlier.


Leah Binder said...

Johns Hopkins/Peter Pronovost advise Leapfrog on the science behind our Leapfrog Hospital Survey and the Hospital Safety Score. Our board (mostly of employer purchasers) handles the political issues. We don't take a position on state-specifi issues, including Maryland's system, except for one major issue: we are strong advocates that Maryland should report the same patient data publicly that every other state is required to report. Thanks to their price-control waiver they have been exempted. Maryland residents (including yours truly) deserve the same information about their hospitals as everyone else gets. The good news is that it looks like CMS will stop giving Maryland that waiver of their transparency.

Anonymous said...

Leah Binder,

Thank you for taking the time to respond these these posts.

I now understand that you and Leapfrog must be above most state specific and political issues.

I just hope your board can apply the business principles I am sure they know in a Medical setting. What gets measured receives the attention of health consumers and administrators.

We don't want consumers choosing providers or plans for the wrong reasons - for instance just because a provider has more resources (and higher prices). We want consumers to choose the provider getting the best "outcomes".

Our end goal should be a system of competitive accountable care organizations that care for populations. This includes moving health organizations to systematize, but also to find better methodologies in provision of healthcare. Focusing on outcomes rather than "inputs" is the best opportunity to foster the experimentation necessary to improve. It is the heart of Adam Smith's "invisible hand".

I hope you receive support to get faster and more frequent release of health data from CMS. It is important. Part of the reluctance of some "weaker" health organizations I am sure is poor results. But part of the reluctance by "high quality organizations" could also be "incomplete" data that is over publicized that provides that wrong incentives to consumers and providers.

I know there has been a big effort here in Massachusetts to improve the health statistics being collected, though I don't know the specifics. Those specifics could be of interest to your organization or your advisors.

I am sure you know all about this, but just trying to be helpful.

Thanks again for responding.

All the best,