Sunday, November 02, 2008

Is unilateral public disclosure really necessary?

Regular readers will know that BIDMC is remarkably open in publication of clinical outcomes, taking transparency to a place seldom seen in American hospitals. Our governing boards are comfortable with this and are strongly supportive even though it occasionally leads to publicity of the sort that can sometimes get them nervous.

That happened a few weeks ago when one of our Trustees asked the question explicitly, saying "I understand the power of transparency within the organization, but is it wise to be so open about clinical outcomes with the public? Can we be sure that the incremental value, in terms of staff performance within the hospital, merits the exposure of our warts and flaws to the broad public?"

This is, of course, a legitimate question, in that we live in a very competitive health care environment here in Boston, and we certainly do not want to engage in behavior that would undermine the reputation of the hospital and perhaps hurt its financial performance. My answer had three parts: First, an acknowledgement that what we are doing is an experiment; second, that there has been no evidence at all that it has adversely affected our clinical volume or our standing in the marketplace; and third, that studies of organizational change suggest that public disclosure has extra motivational value in encouraging people to engage in continuous process improvement.

Now frankly, I had no empirical evidence of the last point but was relying on presentations I had heard from MIT's Steven Spear and IHI's Jim Conway on this topic. I trust both of them as experts in this field of process improvement, and both have been extremely helpful to our hospital as we proceed with this adventure.

Then, this weekend, I read a somewhat old article that supports this proposition. It is from Health Affairs and is entitled "Hospital Performance Reports: Impact on Quality, Market Share, and Reputation," by Judith H. Hibbard, Jean Stockard and Martin Tusler. You can read it here. The article concludes, based on several hospitals' actual outcomes, that "making performance data public results in improvements in the clinical areas reported upon."

Intuitively, this feels correct for lots of reasons, but it was interesting to see research supporting the conclusion. I wonder if people reading this know of other studies that reinforce or undermine that result. Please comment.

As a final point, I also want to note that in today's electronic environment, it is virtually impossible to keep data "private" if it is sufficiently distributed to the hospital's staff. So, if you don't want the public to know, don't even tell your own people!


Anonymous said...

A comment I just picked up from Steve Spear:

Delivering care requires coordinating harmoniously an extraordinary number of individual disciplines. This means anticipating myriad interactions of patient, provider, place, and circumstance, and anticipating perfectly all circumstances is impossible. However, by responding when things go wrong, those working in and responsible for care delivery processes can see their vulnerabilities, identify their causes, and rectify weaknesses, leading to ever improving efficacy, efficiency, and responsiveness. This is not a hypothetical assertion: Order of magnitude improvements in care have been recorded in Pittsburgh hospitals, at Ascension Healthcare and Virginia Mason Medical Center, and elsewhere. Those hospitals not pursuing the same degree of openness are not any less dangerous. They are simply not admitting the reality to themselves, their staff, and their patients.


Anonymous said...

I would venture to believe that those hospitals who lack external transparency also lack strong mechanisms of internal transparency, and operate under 'old schools' of management that are more vulnerable to argument by authority and force of personality in clinical practice and resource allocation. Hospitals that are open about harm have a significant new obligation, however, to create internal systems to support and spread change that traditional medical institutions do not need bother with.

Anonymous said...

This citation is somewhat tangential to your specific point, but there is an article/editorial in the New England Journal within the last 2-3 years concerning the improvement in cardiac surgery results in New York following the publication of report cards, by institution and by surgeon, of 30 day mortality following CABG. Over a period of years the performance did improve substantially, with additional information that the persistent surgeon outliers eventually moved out of state.

Also, I was on the medical staff of a hospital which temporarily lost its JC accreditation due to an aggregation of patient care errors, which came out in the local newspaper after someone on the medical staff "blew the whistle" when internal attempts to remedy the situation were rebuffed by the administration. Of course it all came out in public in the worst possible way, and the hospital lost a huge amount of trust (and business). The subsequent administration scrupulously reported every single error to the state and to its staff, which initially resulted in " _____ hospital has mistakes again" headlines, but over time restored trust in the hospital. I believe your approach is the only way to handle error reporting - and you are dead right about it coming out anyway if the staff knows about it. Then it comes out in a way which may not be totally accurate and which you don't control.


Anonymous said...

ps I fortuitiously ran across another citation which may or may not be relevant:

Fung CH et al. "Systematic review: the evidence that publishing patient care performance data improves quality of care." Annals of Internal Medicine 2008; 148:111-23.

I have not read this but it may be worth checking out.


Anonymous said...

Paul, agree wholeheartedly with the comments that link internal and external transparency. You might also be interested in an article by Mark Keroack et al "Organizational Factors Associated with High Performance in Quality and Safety in Academic Medical Centers" in Academic Medicine, December 2007. Mark notes that the leaders in the top performing organizations expressed a strong sense of dissatisfaction with the current state of quality in their institutions and focused on the gaps (something you can't do without transparency...both internal and external) and the leaders in the poorer performing places tended to be more complacent or removed from how far they were from benchmark data. So it is tough, but I believe the right thing to be open.
Maureen Bisognano

Anonymous said...

Although I see the benefits of transparency, I think there should be concern that there may be unintended victims from it. That would be your staff. Being on the front page of the paper must have an impact on staff morale (not to mention the corporate campaign they are subjected to by that vile union). The staff see these negative stories on the front page they in a way become the second victim and feel like they are giving inferior care as other hospitals conceal their dirty laundry. For example- the wrong site surgery- it was your hospital's first in many years. This is not a rare occurrence as it happens multiple times around the state each year. However it seems to get more attention when it happens at your place. Another article covered surgical residents being over there duty hours and that the program is under probation. I know for a fact that the Brighams program needed an emergency review by the resident governing body for even worse violations but that does not make the press. There were multiple articles on a substance using surgeon. There are many disciplinary actions related to substance abuse by the Board that never make the press. A maternal death article has your picture next to the victim on the front page...never recall seeing an article like that and there are several of those in the state each year. There have been many similar examples over the years even before the era of transparency. It seems like the media has had their cross hairs on your institution for quite some time while other institutions are "protected" and their bad news is suppressed. Like I said, I believe that transparency in the end will lead to improved care but besides potential loss of market share, there can be other downsides namely staff morale.

Anonymous said...

Dear Paul,

This is great idea - ascertain something that would really helpful if it were true - and ask the web if it is - well .the citation posted above is really worth a read with the editorial.

tFung CH et al. "Systematic review: the evidence that publishing patient care performance data improves quality of care." Annals of Internal Medicine 2008; 148:111-23

Especially as it concludes:

Given this limited evidence that public reporting motivates
efforts to improve quality, one must ask whether the
theories and assumptions that underlie the policy approach
are faulty. Alternatively, was the theory right but the implementation
faulty? Do we need to rethink the value of measurement and reporting efforts, or do we need to figure
out how to do them better? Fung and colleagues’ analysis does not shed any light on these key questions, which I discuss further here.

Guess what they conclude - it is more likely to be an execution problem:

We should not interpret the results of Fung and colleagues’
systematic review as an indication that we should
give up on consumers as important actors in the quality
improvement equation and move on to a new paradigm.
Rather, they suggest that we should improve the execution
of public reporting efforts and only then reevaluate the
effect of public reporting on quality. Although Fung and
colleagues’ findings do not uphold the theory that drives
public reporting, they are consistent with the conventional
wisdom that inconsistent execution yields variable results.

Paul you are definitely doing your bit - keep going please!!