More in our continuing series on central lines infections. As always, these are presented as cases per thousand ICU patient days. Every single case undergoes a multidisciplinary review with department leadership present, after a review by the attending of record and primary nurse, as well as the Central Line Work Group which is overseeing this effort.The chart above shows that the overall quarterly trend is in the right direction, but as you can see below, there is troublesome variation from time to time. The up's and down's, I guess, are normal, but we all wish they stay down.
Month ----- Infection Rate
Oct 05 ----- 1.67
Nov 05 ----- 1.28
Dec 05 ----- 2.43
Jan 06 ----- 3.07
Feb 06 ----- 1.40
Mar 06 ----- 1.07
Apr 06 ----- 0.00
May 06 ----- 0.59
Jun 06 ----- 1.15
Jul 06 ----- 0.57
Aug 06 ----- 3.03
Sep 06 ----- 2.50
Oct 06 ----- 0.00
Nov 06 ----- 2.38
Dec 06 ----- 1.87
Jan 07 ----- 0.00
Feb 07 ----- 1.15
Mar 07 ----- 3.17
Apr 07 ----- 1.22
Paul,
ReplyDeleteThe question you need to ask is are these infection rates the result of a stable process. If so the only way to lower the rates is to improve the process that leads to these rates. However if these rates are not the result of a stable process you need to first establish a stable process and THEN work on improving it. Just showing numbers to people and saying LOWER IT doesn't do anything. They make work harder temporarily but there is no real improvement. That is simply management by objective, but that does nothing to treat the root cause of the infections.
Yes, of course we did that first. A full-fledged analysis of prior problems, development of a standard protocol for line insertion and maintenance, training of all clinicians who would be involved in this activity, and so on. Now the root cause analyses are conducted to see if we missed anything that should be rolled into the program.
ReplyDeleteBTW, sometimes the numbers reflect the result of a blood test that indicates an organism that can be associated with a CLI, but there are no symptoms of an actual infection. The CDC guidelines require that this be counted as a CLI nonetheless. Also, sometimes the patient has been transfered from another hospital, where the infection may have originated. We count all of these in our numbers.
dont worry about those numbers. every day u drive to work some days its 20 mins some days its 22 mins some days its 18 min. variation is life baby.
ReplyDeleteHas there been any change in the percentage of CLI's that result in death and can it be definitively determined that any deaths that occur were attributable to the CLI as opposed to some other cause?
ReplyDeleteDo you mean for us or more generally? The percentage that I have cited before is a general number for the industry. I haven't heard about any changes in that overall.
ReplyDeleteI meant for BIDMC. I didn't realize that your prior reference was a general number for the industry.
ReplyDeleteDon't have CLI-associated mortality rate here handy right now. I'll get back to you.
ReplyDeleteYou guys do statistical process control charts to analyze data?
ReplyDeleteCould you talk about how the IHI 100K Lives Campaign affected your practice?
Sorry, I don't know what you mean by that, i.e, statistical process control charts. I'm sure someone here does, but I have never heard the term.
ReplyDeleteWe work closely with IHI on lots of things. 100K Lives program overlaps with some many things we do.
scary that a major hospital ceo doesn't know what a control chart is. it helps you to differentiate between common cause and special cause variation. if you have a stable process, i.e. no special cause variation, only then can you begin process improvement initiatives.
ReplyDeleteEase up on the jargon, buddy, and speak English. I am actually trained in statistics, and I have never heard the term. But I appreciate the lecture.
ReplyDeleteI am sure, though, that the doctors and nurses here who handle our quality control programs know how to do the kind of analysis you are describing.
i doubt it! docs and nurses receive no training on this; it is management's responsability to spread the system of profound knowledge.
ReplyDeletePaul, 2 MUST READ books for u:
W. Edwards Deming "The New Economics for Industry and Government"
Joiner "Fourth Generation Management"
Yes, they do get training in this.
ReplyDeleteok fine read the books anyways *muah*
ReplyDeletePaul;
ReplyDeleteYou sound as bewildered as I was some years ago(as medical director of a hospital laboratory) when I first heard these terms; statistical process control (spc charts for short) and stable vs. unstable process. They are part of the way the performance improvement dept. decides if results are significant or not, and therefore they can just continue to monitor the indicators, or need to take some sort of action. The methods they use are really very interesting and I have the feeling you would get the same "eureka" moment I did once I saw the light. One of the PI people could give you a quick "course" in all of this - even though I have already deduced that one of your best skills as CEO is the ability to delegate and trust your people.
However, it will also help you present these numbers on this blog in a way that's more understandable to people reading them and theoretically trying to use them in deciding where to obtain care. That's the whole purpose of public outcome reporting, right? Then you could offer a summary as to the significance of your results each time you report them.
Like some of the anon's above, I had questions about your monthly variability; whether it's significant or not. (For instance, special cause variation might be that resident X is always working when the rate is higher, or whatever.)
I do have to agree the CEO should be conversant in this stuff, if for no other reason than to impress your staff and the medical staff that you know more than they think. (: No, seriously, because it affects your personal ability to evaluate BIDMC's quality of care.
Guys and/or gals, please relax. I understand this stuff. Really, truly. I just did not know the words you used. Let me say again: My major interest and training in college was statistical analysis and inference.
ReplyDeleteI disagree on this point, "However, it will also help you present these numbers on this blog in a way that's more understandable to people reading them and theoretically trying to use them in deciding where to obtain care. ... Then you could offer a summary as to the significance of your results each time you report them." Public reporting will not be at this level of detail. The general public will not be able to interpet things to this degree.
Everyone is currently working through the question of what information would be most helpful to the public. We have some ideas. Stay tuned on that.
I'm sorry, I didn't meant to start a war by mentioning SPC charts...
ReplyDeleteI think your site's great and your position affords you a lot of insight, and it's so great to read your blog because I've so rarely heard senior management talk candidly about issues.
Which brought me to my second question about IHI and their work. I've heard Don Berwick speak and he's great, and I've read tons and tons of stuff about his work in papers and such. Commitment to quality is personal, as he, Paul Bataldan, and so many others have said, and obviously you think that too since you boast about your numbers on your personal blog.
It's really fascinating how management/leadership can drive better patient care. Your work at BIDMC speaks for itself.
-SWC
Well, then, perhaps you could do it (discuss significance) for readers of this blog, if not the public; otherwise why publish it? Is your process considered to be in control, or not? (I am guessing yes from your numbers) Also, just my opinion, it seems that treating every central line infection as a sentinel event with the formal process of root cause analysis consumes a lot of resources. I know sentinel events are defined as having caused harm, or with high potential to cause harm, to a patient, but I think most hospitals reserve them for really severe incidents like wrong medication or wrong patient ID, etc. Or am I out of date? You all must perform a huge number of RCA;s, or have a truncated process for doing so with less severe incidents?
ReplyDeleteBTW, don't worry; I don't think any of your readers doubt your training or brainpower.
Dear SWC,
ReplyDeleteDon't worry. Not even close to a war. Thanks for your kind words.
But, I am not boasting about these numbers. Pls go back to several other posts as to the reason I am posting these and others. For example, http://runningahospital.blogspot.com/2007/02/we-saved-one-persons-life-can-we-keep.html.
But, especially read this one. http://runningahospital.blogspot.com/2007/03/these-things-happen.html.
Here's the pertinent excerpt: "One way to encourage organizational improvement is to publicize the results of your program. I have done that below for our hospital, and I have made the suggestion that others in the city could do the same. As I noted, I did not make the suggestion for competitive purposes -- after all, I don't know if our numbers are better or worse than those of other hospitals -- but because public exposure of all our efforts will drive all of us to do better. Also, it will build, rather than erode, public confidence in the academic medical centers in our city."
Anon,
We are intensely focusing on these cases because we want to make sure we are understanding what is systemic and what is not. There are so few cases that it is worth spending time on each one. But, btw, a central line infection is serious. A very large percentage of them result in death. That is why we are shooting for a "zero" standard.
hey anonymous do u run a hospital? i want to work for you. Love, ananymous. We'll make it a 100% deming hospital.
ReplyDeleteHey anon 8:47;
ReplyDeleteNo, I don't run a hospital; I just have fun sitting home and giving everyone else my advice! (: But yeah, Deming was part of my PI "education" and he was great. I did run a hospital lab once; arriving when it was in a state of total collapse due to the wrong chem analyzers, a failed LIS implementation, failed administrative leadership, angry clinicians, state threatening to withdraw Medicare reimbursement, etc. One learns PI fast under those circumstances. Must have been similar to when they appointed Paul CEO of a failed merger, although I wasn't aware of that one at the time.
Love, anon. (:
Lets start our own deming pi blog!
ReplyDeleteI work in quality control in a large hospital. We use statistical process control to look at our data. Basically, it is a way to draw confidence intervals around a data trend and decide whether the trend is meaningful or a random fluke. SPC works great for large number, but most things that we track -- like your central line cases -- are not that amenable to making conclusions on the basis of SPC. You would have to wait forever before you did anything about anything.
ReplyDeleteTo reply to the statistical geeks out there, a note from our Quality Control chief:
ReplyDelete"We analyzed the data and established an “upper control limit” as defined by statistical process control methodology. There is no lower confidence limit shown because it would be zero. What this basically says is that if our rate during any one quarter in FY06 jumped above 5.7 per thousand patient days, we should not attribute it to a “special cause” but is likely due to random variation. Likewise, we cannot declare any one quarter decrease as significant improvement because getting even to zero for any one quarter would also be within the realm of random variation. For this type of data, SPC is more likely to be helpful for analysis over an extended period (i.e. a year or more) to say whether the gradual decrease in rates we are seeing is because we made a significant impact, or just good luck."
So does this mean we still should not strive for zero? No way. Does it mean we should not investigate each infection for root causes? No. Does it mean that I still view variation as troublesome? Yes, managerially, even if no, statistically.
you lack understanding of the system of profound knowledge (not an insult) just read the two books I noted above and you'll join our side. but your QC chief is the man good job.
ReplyDeleteThanks, Paul, for your helpful followup. I know a lot of administrators who would not have bothered to follow up, so my respect for you is increasing even more!
ReplyDeleteAs for using root cause analysis, my comments above against it presupposed that you did RCA's on all such rare, but "predictable" events (by predictable, I mean that a certain incidence of complications of any procedure is predictable). If you did so, you would be doing a huge number of RCA's. But if you have specifically selected out central line infections for RCA's, to send the message that you really, really want this to be zero, then that is managerially appropriate. Sorry I misunderstood what you were doing.
If you will accept a voice from manufacturing, meticulous attention to quality metrics is sensible but failure to incorporate best practices into the strategy reduces the likelihood of signifiant progress. Process controls are only helpful if the process is a good one. I don't see any mention of best practices, including technology (antibiotic catheters, chlorhexidine skin scrub, etc.), that actually reduce infection. The exercise can be as academic as you choose to make it but ultimately, the best outcome will probably result from selecting the right tools then scrupulously applying them.
ReplyDeleteAbsolutely on target. We do just as you say.
ReplyDeleteWow, months later. I stumbled on this because I wanted to see why someone was using an upper control limit that I thought was way to "small" for the job.
ReplyDeleteI'm in healthcare as an analyst. We use control charts all the time.
Control charts are a way to apply bell curve statistics to events that happen over time.
With central line infections, first you establish your 'best practices' to prevent illnesses. And I try to find a statistic of how many of these infections are found in the non-hospitalized population (as a rate).
In your case, since your ideal is zero, you will show an Upper Control Limit (UCL) on your chart but no lower. You would probably also want to show your 1s and 2s lines as well. In a rough way, the 1s is one standard deviation above your average. Your 2s is two STDs, and UCL is 3s. Approximately. To have anything outside one standard deviation is slightly odd, outside two is odder, and three is becoming improbable. So, if something jumps three at once, it's a shock. On the other hand, something sneaking up with seven events through two levels is also something to check on. That's why it's best to buy a book, because there will be probably a dozen rules of activities that you might want to watch out for. And because you can "tighten" up your limits in certain ways so that they are not so classically a bell curve.
Here's the wikipedia address for control charts. http://en.wikipedia.org/wiki/Control_chart
and then I have a book (at home) called "Variations" which covers the subject well.
Also, a CEO should have an executive consultant, the highest level of analyst in a medical organization. Very likely the charts you are seeing from your doctors and nurses are being made by lower level analyst.
Why do people who are good at statistics keep trying to provide advice on this? And on whom I should hire? This is not a statistics issue. We view every infection as a potential learning event and investigate its root cause. We don't need to calculate control limits for that.
ReplyDeleteNo, the control limits just tell you "when" your process rendered an odd effect. You still need to look at the real situation to find out what might have caused it. If your control limits show you a periodicity, then you look for a periodic activity. If there is no periodicity, then you look for the singular event cause. Sometimes you can have periodicity with singular events imposed.
ReplyDelete