Tuesday, October 18, 2011

Modeling readmissions

The current intent to judge hospital performance and modify hospital payments based on relative rates of readmissions is not wise.  Contrary to President Obama's characterization that readmitting a patient to the hospital is equivalent to bringing a car back to the mechanic after a repair, rates of readmissions are based on a number of factors, of which a significant portion are services not provided by the hospitals and environmental conditions not controlled by the hospitals.

But let's put my objections aside and determine how we would model an "appropriate" rate of readmissions.  Well, a new article in JAMA* explores existing models, noting that robust models are needed "to identify which patients would benefit most from care transition interventions, as well as to risk-adjust readmission rates for the purposes of hospital comparison."  The article concludes that the capability for doing these things does not yet exist.

In "Risk Prediction Models for Hospital Readmission," the authors state as their objective:  "To summarize validated readmission risk prediction models, describe their performance, and assess suitability for clinical or administrative use."  Their conclusion, after reviewing two dozen such models, was that "Most current readmission risk prediction models that were designed for either comparative or clinical purposes perform poorly."

For the technically inclined among you, here's more of the abstract:

Fourteen models that relied on retrospective administrative data could be potentially used to risk-adjust readmission rates for hospital comparison; of these, 9 were tested in large US populations and had poor discriminative ability (c statistic range: 0.55-0.65). 

Although most models incorporated variables for medical comorbidity and use of prior medical services, few examined variables associated with overall health and function, illness severity, or social determinants of health. 

So, even if the readmission rate is the right metric to use for comparison purposes, we don't have a model that would accurately compare one hospital to the others.  This suggests that the time is not ripe to use this measure for financial incentives or penalties.  It might give the impression of precision, but it is not, in fact, analytically rigorous enough for regulatory purposes.

---
*2011;306(15):1688-1698. doi: 10.1001/jama.2011.1515

17 comments:

Christine said...

From Facebook:

How does this compare with the tendency to hold teachers/school accountable for student failure? Should hospitals/physicians be mainly exempt from "environmental" factors outside their control, but not schools/teachers?

Paul Levy said...

Applying a failed model from another field to health care does not necessarily make either one better. Let's measure the things that matter the most in both settings, or you just get a "teach to the test" mentality.

There are so many other things that deserve attention in hospitals. Perhaps those deserve more attention.

That is inherent in my main point. If we cannot accurately measure this particular metric, why make it the subject of penalties? What resources do we take away from other measures when we do so?

e-Patient Dave said...

Paul, having participated early in 2010 in BIDMC's workshop on bounceback, all I can say is, what do YOU suggest we do about it?

I won't accept "Do nothing because there are no good proposals" - we need a solution. The human and economic cost is too great, and as far as I can tell it's all because of mismanaged transitions.

I'm not asking for a quick fix, I'm asking for an actual fix.

Anonymous said...

Hmm, Paul; this sounds a bit ironic given your posts in previous years decrying physicians for quibbling with the data presented in order to avoid having to change. Don't get me wrong; I agree with your oft used metaphor of hammer and nail for government intervention. But it's kind of like saying that because we have no evidence what treatment works best for a particular condition, we won't treat anyone till we do the studies. The sad fact is that it has become imperative to move forward in several arenas in health care, including patient safety and cost control, without sufficient evidence that what is being tried is effective. What is your suggested alternative?

nonlocal

Barry Carol said...

Paul – I agree that the risk adjustment state of the art isn’t up to the task of measuring differences in hospital readmission rates for the purpose of imposing financial penalties or awarding bonuses. I wonder about the following, though: How good is hospital discharge planning generally? How much variance is there among hospitals? Should insurers’ case managers be even more involved than they are now in discharge planning and execution since they will save money if readmissions are minimized? Hospitals, by contrast, will lose revenue if inpatient bed days decline whether due to fewer readmissions or for other reasons unless they are paid on a capitated (global payment) basis instead of fee for service. Bundled payments for surgical procedures would also reward fewer readmissions.

Paul Levy said...

Folks, I am not arguing about a need to improve. I am suggesting that using a financial penalty approach, generally, is not the best way to accomplish things in this sector. Here, too, we now have analysis that suggests that the basis upon which financial penalties are to be imposed is not accurate.

Please think about what you get when you reward or punish behavior based on poor data.

To Dave's point, an awful lot of improvement could occur just by having effective patient-family advisory councils helping hospital folks learn about the holes in their process and that of SNFs and others to whom patients are sent. That's how you get patient-driven care, not by administrative fiat.

So, think about how you get sustainable change, not just a regulatory quick hit that has unintended consequences.

Russ said...

From Facebook:

Individual patients make individual choices, many times based on socio-economic factors. These are people and the elderly have multiple co-morbidities that don't fit into neat and tidy Excel spreadsheets. Do you penalize the hospital because Mr. X has goes to his brother's barbeque and has 4 sodium laced hot dogs after being discharged with CHF? After he met with the Dietician, the MD, the Social Worker and the Case Manager about what he needs to be careful of? What about the woman who decides that she doesn't want to spend $60 on co-payments and fails to fill her prescriptions for ordered meds?

Paul Levy said...

Barry,

Look for a future post about the pricing question. . .

Anonymous said...

So you are saying that if the data were good enough, financial penalties would be acceptable?
And sorry, although your suggestion could make improvements along the margins, I don't see patients ever having enough power to inspire administrators to do the right thing if it will cost them money and others are not doing it. There is a distinct herd mentality among those who run and practice in hospitals (present company excluded) - except, it seems, in the case of making revolutionary improvements in care.

nonlocal

Paul Levy said...

I am talking about how to get sustainable change and not send people off in the wrong direction and distract them from working together, for the right reasons, to improve the design of work.

For dollars to work, the penalty has to affect each individual participant's bottom line. Hospital-based penalties are very unlikely to do that not do that in that they are highly unlikely to be passed through to individuals. So there is no impact on the salary of the nurse, the MD, the case manager. Also, even if penalties were passed through -- given the dozens of people involved in each patient's care in the hospital -- there is no way to design a financial penalty system that is of a magnitude that would meaningful to all those people.

But more fundamentally, all research shows that people in organizations become less motivated, not more, when money is introduced into their personal value equation.

Imagine you a are pathologist. How would you have react if you were told that you would be penalized for each mistaken analysis. How would it affect your attitude and motivation. What would it do to your productivity? How likely would you want to continue to review the really interesting and difficult cases?

clsmt said...

But more fundamentally, all research shows that people in organizations become less motivated, not more, when money is introduced into their personal value equation.

"People" might become less motivated but administrators and CFOs find money absolutely inspiring in my experience. As a manager, you can't go to your staff and say, "this will save the hospital money so let's do this" but you can go to them and say, "if we make this change, we'll have more resources to provide care to more patients and the quality of that care will be better". Both are driven by the same math but the motivation is different. It's up to management to translate "the feds aren't going to cough up any more cash" into something palatable for staff.

Anonymous said...

But Paul, we are not talking about how to motivate individuals, or even groups. The issue you raise is how should the government, (as steward of our taxpayer dollars paying for health care as well as public protector of health, like food safety, travel safety, etc) - how should it motivate corporations to behave in a safe and cost-effective manner? One potent method is the tax code - but most hospitals are non profit. The only other ways are also monetary - laws, and regulations, with fines/incarceration for violations.

We need to make delivering safe and cost-effective health care resemble paying your taxes - the public expectation is to do it, and the minority that don't are penalized. Instead, here the small minority do it voluntarily and the rest don't and are not penalized.
Perhaps only the development of overwhelming public pressure, with the accompanying universal expectation of doing the right thing for the right reasons, will eventually prevail. Until then, we have only imperfect measures imperfectly applied to reluctant providers.

Paul Levy said...

RE: "we have only imperfect measures". Not so. Look at many of the recent advancements in quality and safety improvement, e.g., Pronovost's work on central line infections in Michigan which has spread widely. There were no financial incentives and penalties applied there. The industry can move things along when there is a persuasive case about the importance of things.

Admittedly, there are also cases of slow movement: Many wrong-site surgeries persist. But even there, the use of financial penalties for "never events" has been unsuccessful.

I'm just saying don't use the financial hammer on every "nail" in sight, especially when the data don't support a fair application of the penalties.

Anonymous said...

I forgot to sign my previous comment to which you replied, but it was me. I'm not going away. :)

The advances you describe are piecemeal, as is, mostly, the readmission initiative. And I agree about the financial hammer; just pointing out that the government has no other hammer available to it.

I guess I have to fall back on the public pressure idea. But that will be fortuitous, if it happens - based on some celebrity's death, or something. Sad that things have to happen that way.

nonlocal

Brad F said...

I hope a useful post to add to this dialog:
http://blogs.hospitalmedicine.org/SHMPracticeManagementBlog/?p=3445

Brad

The Happy Hospitalist said...

Here's an idea. If doctors and hospitals are to blame for providing bad care to patients, that cause them to be readmitted, why aren't patient's suing to recover damages for pain and suffering related to their readmission?

There's got to be a few lawyers out there willing to take that cause on. Perhaps the AARP?

e-Patient Dave said...

Hospitalist, I can hardly imagine that improving anything... not that anyone asked... I'm tired of the old "punishment works best" approach.