Saturday, July 25, 2015

Everyone likes to think they are doing better

Tissue plasminogen activator
I recall a wonderful story from Amitai Ziv, the director of MSR, the Israel Center for Medical Simulation at Sheba Medical Center on the outskirts of Tel Aviv.  He relates how Israeli fighter pilots would return from their missions and debrief how things went.  The self-reported reviews of performance were very good.  Then, the air force installed recording devices on the planes, and it turns out that the actual performance was not nearly as good as had previously been reported.  The conclusion: It's not that people are poorly intentioned or attempt to mislead about their performance. It's just that we tend to think we are doing better than we actually are.

Let's turn to health care.  Here's a recent story by Lisa Rapaport at Reuters that portrays a problem and--as in the Israeli example above--demonstrates the importance of transparency--providing staff in a hospital with actual data about their clinical performance. The lede:

Many hospitals overestimate how quickly they give stroke patients a clot-busting treatment designed to help minimize damage, a U.S. study suggests.

Researchers asked hospital staff how fast they administered an intravenous (IV) therapy known as thrombolysis to dissolve clots and compared the answers to stroke registry data with the actual times.

Only 29 percent of hospitals had an accurate sense of their own speed.

“Everyone likes to think that they are doing better,” senior study author Dr. Bimal Shah, a researcher at Duke University School of Medicine.

The slowest hospitals were also the ones most likely to be inaccurate about their results.

The gap between perception and reality was far bigger for hospitals that were generally slower.

Among the lowest-performing hospitals, staff surveyed generally thought that at least 20 percent of treated patients got the therapy within an hour. In reality, none did.

Despite their lack of speed, 85 percent of the low-performing hospitals reported their performance as average or above, with almost 5 percent of them ranking themselves as superior in comparison with other hospitals nationwide.

Those of us who are Lean adherents believe in the idea of visual cues, providing data about an organization's performance in real time to those working in an area.  That information helps a place monitor its performance and look for ways to improve and sustain improvement.

It looks like stroke centers and their patients could benefit from such real-time reporting.  As Dr. Shah notes, “Not acting quickly makes the prognosis for stroke patients worse.”


Mark Graban said...

I'd agree that it's important to look at data and our performance. It's important for Lean leaders to understand the real reality.

Instead of pretending that we're above average or top decile, we need data.

That said, having charts or metrics is "visual management," I guess. The problem with metrics is that they tend to lag, by a day or a month.

The core part of visual management is to help "detect abnormal from normal... right now!" How do you know at a glance if your process is performing well? Hanging signs and metrics is not really the truest form of visual management.

That said, you bring up an important point that organizations shouldn't fool themselves about their performance or how they compare to peers.


Anonymous said...

Reminds me of working with hospices in late 90s. They were sure they were doing the work of angels. No need to measure or report. Then we asked them to see things,such as how long it took to get pain meds delivered,how patients felt, and so on. To their surprise (not mine) they wound up with failing scores...and an imperative to change.

Neville Sarkari MD, FACP said...

Physicians as a group are the worst. We all think we are "above average"....!