I have been searching for meaningful and effective ideas to present our central line infection rate that might supplement the one we use. We use the ratio of cases per thousand ICU patient-days. This is a good and accurate metric, but the problem that arises when you have a consistently low figure like 0, 1, or 2, is that there will inevitably be variation around it that may not be helpful in analyzing or explaining how you are really doing over time. Plus, is there another metric that gives just a bit more incentive to improve?
I am not talking about what our goal is. Our goal is "zero." Whether expressed as a rate or a simple number, the virtue of "zero" is that it is indeed "zero" in both cases. As Paul O'Neill has often noted, "Setting zero errors as a goal encourages breakthrough thinking, orients work cultures towards continuous improvement, and keeps people pushing toward the goal."
In factories, you often see a sign saying "x days since our last accident" that motivates people to pay attention to safety procedures and practices. We could do that for our hospital, i.e., "X days since our last central line infection," but I am not sure if it would be as effective. For one thing, we have several ICUs dealing with different kinds of patients and different degrees of difficulty in avoiding central line infections.
For example, we have heard an excellent report from folks in Pittsburgh championing a year without a line infection, but this was for their cardiac care unit only. In our CCU, they are past the 300 day mark without a line infection, but CCUs are lower risk than other ICUs.
Of course, this problem already exists for our composite ICU rate, too. And people will point out that factories have lots of different manufacturing sections with variation in risk. A company-wide figure creates both an overall sense of pride and community and internal peer pressure among the various corporate divisions to not let the whole group down by being the site of an accident.
Would that work within the setting of academic medicine? What's the verdict from those of you out there? Have any of you done this? Did it make an appreciable difference in how people behaved? In public perception of your institution?
Would it matter to you as a prospective patient? If you read a website saying "60 days since our last infection", would you say to yourself, "What an excellent hospital" or would you say "That's a long time -- there is bound to be an infection soon, and maybe it will be me"? Does it work better or worse than posting an infection rate of "1.2 cases per thousand ICU patient-days"?
I'm just a random lurker without any medical training but I thought I would weigh in on this one. I'm an optimist so I'd rather hear nothing about the infection rate. Maybe you could do something like an ebay score? 73 positives and 1 negative, where every time you have a patient with no infection the whole time it's a positive, and every time you do have an infection it's a negative.
ReplyDeleteI know it's weird but that's how I'd like to see it. Then I could rate which hospital to go to like I choose an ebay seller or hotel. Is that too simple? Because I'm thinking about health rather than a purchase?
When I see those highway signs all I think is "how morbid!"
Anyway take it or leave it. That's the weirdness of me... :)
I also have no medical training, but I would like to make a couple of comments. First, on the concept of scoring and risk adjusting generally, I think it would be helpful if there were some consensus definition among senior medical people and managers of academic medical centers as to what constitutes excellent vs very good, average or sub-standard performance. If there were a risk adjusted ranking for the outcomes of ten surgeons, for example, it could well be that the performance of the top five amount to the college equivalent of an A or A+ while the next five might be the equivalent of an A-. If, as a practical matter, any grade of B+ or better suggests more than adequate competence to do the job such as, say, remove a gall bladder without complications, the raw rankings would not really be all that meaningful.
ReplyDeleteSecond, I work for a subsidiary of a company in heavy manufacturing. We have many jobs that are both physically demanding and dangerous. Work environments can be hot and hazardous with lots of heavy equipment including overhead cranes constantly moving around. Over the last couple of years, our CEO has embraced workplace safety as one of his top priorities. He pounds on it constantly, and it has made a positive difference. Training has been increased and improved while metrics have been developed that can affect compensation. The very fact that you are bringing CEO level attention to central line infections, ventilator associated pneumonia and other safety related issues is, in itself, a good thing.
I know you asked your peers to post their numbers in a timely fashion ... but regardless of timely numbers, I'm assuming you have access to the general "industry average" infection rate. Is a sustainable infection rate of zero an acheiveable goal? If it is achieveable, at what cost? Would it better to bring the infection rate down to somewhere between say 0.5 and 1.5 infection rates and treat those patients aggressively (rather than a potentially unrealistic goal of zero?)
ReplyDeleteOf course it would be great to snap your fingers and instantly reduce the risk to zero. However, since that undoubtedly comes at some cost - where would a regular economist draw the line? Where would the Freakonomics guys draw the line?
Thanks for your insight ...
As a potential patient with no medical training, I'd like to see the first, 60 days and no infections. It's both a mark that whatever the risks, the administration and the line workers are striving towards the same goal. And, unlike many of the college ranking metrics, this is not something that can be spun through marketing.
ReplyDeleteAs a medical student, I do not think patients would ever want to see "2 weeks since our last infection" or any other small number.
ReplyDeleteI'd like to echo the concern that zero maybe is not the best goal. It would be wrong for you to become so obsessed with this one metric that it promotes undesired behavior. Investing some of this marginal energy in promoting your efforts to other hospitals or working other issues could have a bigger payoff in patient health.
ReplyDeleteThe measure I would like to see is the ratio of discharges w/out infection (not including death by other causes)/total number of lines placed (not including death by other causes). It's kind of clunky when put that way, but it answers the patient's question about the risk of complication. The second measure would be deaths after infection/number of lines placed with successful discharges which answers the question as to what are the chances I might die.
It might seem unfair to exclude credit for any central line placed where there was no infection but the patient died, but including that data is not meaningful for non-medical people and I'm pretty sure that if that set of data (with those placements inluded) were to move in the opposite direction than the ratios above, that it would be of signifiant concernn to you and your staff.
We used the "___ days since our last infection" in our ICU's and generally, the staff didn't like it. I kind of liked the first commentators suggestion of something like eBay---66 positive feedbacks (patients without an infection) and 2 negatives (2 patients with). You could do the rankings for UTI's, VAP's, and bloodstream infections. That then let's folks see in "real" numbers how often an infection happens.
ReplyDeleteI think the problem with the way medical complications are portrayed these days is the assumption that every complication, in this case line infection, represents "error" on the part of staff. If a patient has a pneumonia and bacteria get into the blood and a central line is in place, the line can get infected. That has nothing to do with the way the line was inserted. Similarly, if a patient has postoperative bleeding, a known complication of the procedure, it cannot be assumed the surgeon made a mistake.
ReplyDeleteTo create a culture of safety is wonderful. To imply that every complication that occurs in medicine is due to human error is wrong, misleading, creates unnecessary fear for patients, and contributes to an antagonistic relationship between patients and care providers.
Although you didn't offer this as an option, my vote would be "neither." :)
ReplyDeleteThe "x number of days/months" since the "last whatever" have always struck me as more negative than positive, because they shine a light on something bad, even though the intent is to highlight something good. This information might be inspirational for employees, but I've always wondered why companies want to post it where customers can see it.
For me, this would be even more of an issue with hospital infection rates/incidents. If I'm being admitted for surgery (or a loved one is), I don't really want to be reminded of all the things that can go wrong. I'm aware of infection risks, medication mistakes, etc., but there's a difference between knowing something and having it staring you in the face. Rather than being impressed that the staff of General Hospital has made it through 60 days without an infection, I would be thinking something along the lines of, "Infection rates are enough of a problem (or have been in the past) that they need to both monitor them and publicize their improvement."
I applaud what you're trying to do, and I don't mean to sound negative or critical. Just wanted to throw out another point of view that you might not have thought about.
Mary
I disagree about using zero as a goal. There should be no other goal than having a patient survive their hospital stay with no problems they didn't come in with. We do track how many days since our last central line infection in our ICU, but what our staff prefers to hear about is how we're doing with our bundle compliance (IHI http://www.ihi.org). The major obstacle we were having was physicians refusing to wear the hat as part of the maximal barrier precautions dictated by IHI. It went so far that we had to empower the nurses to refuse to assist with a line insertion if the MDs didn't comply. As a result, we've had a fairly consistent 100% rate for bundle compliance. Not sure how you could use that for the general public but it certainly works internally.
ReplyDeleteThere is a potential cost to your goal of 0. That would be modification of practice of your physicians--ie. if you dont put in the central line you will not be at risk of being held accountable for a central line infection. Risk of infection should modify the process of catheter insertion and care of the catheter and patient, however, goals for infection or other complication rates may have an impact on the decision to place the catheter if they are given excessive importance. If the audience for the metric is the patient a simple "We are #1 in . . . " using conventional metrics is probably enough. If the audience is the clinician detailed feedback about each event is much higher yield than a simple overall running count, however formulated.
ReplyDeleteDefinitely do not use # of days since an infection. Everytime you have one infection (and you will have some)you would have to post "1 day since an infection". How awful would that be for a newly admitted patient/family.
ReplyDelete> Would it matter to you as a prospective patient?
ReplyDelete> If you read a website saying "60 days since our last infection", would you say to yourself,...
Context is everything in such displays. In a factor, "135 work days since our last accident" is clearly understood by all viewers, who live with the number all the time and who probably have been trained about it.
Newcomers to a similar situation are not so sure to have the background.
I personally like the approach you're using. Say "Our goal is zero" and display a graph (or something) with the evidence.