Let's give credit, not throw brickbats

A recent story in the Washington Post makes a common error that causes the casual reader to reach an inappropriate conclusion.  Here's the background.  The Agency for Healthcare Research and Quality offers to help hospitals conduct periodic surveys of their staff on the topic of patient safety culture.  As designed by AHRQ, the survey assessment tools are designed to help hospitals:

• Raise staff awareness about patient safety.
• Diagnose and assess the current status of patient safety culture.
• Identify strengths and areas for patient safety culture improvement.
• Examine trends in patient safety culture change over time.
• Evaluate the cultural impact of patient safety initiatives and interventions.
• Conduct internal and external comparisons.

Hospitals are not required to conduct the surveys.  According to the Post, "About 650 of the country’s roughly 5,000 hospitals took part this year."  Staff who participate in the surveys are promised that their responses will be held confidential.

As the result of  a petition by the National Nurses United to the National Labor Relations Board, the results of the 2012 survey at MedStar Washington Hospital Center were released for public view.  The hospital then decided to also release the 2014 survey results.

Here's the mistake made by the Post in its story:  It compares the Washington Hospital Center results with the national average and concludes that WHC has "low marks." Notwithstanding the language on the AHRQ site about external comparisons, this is a mistake because such results across multiple hospitals cannot be assumed to be comparable.  Indeed, it is well known that some hospitals with higher grades in the survey achieve those results because there is a lack of awareness of the depth of patient safety issues in their institutions: People therefore have false confidence that they are doing well.  Likewise, some hospitals that have excellent safety programs get lower grades because the people in those hospitals have done enough work in the field to believe that they need to do even better to reach the standard to which they hold themselves accountable.  (Indeed, I have found that the one of the best way to judge the commitment of a hospital to quality and safety improvements is to ask people how it's going and have them respond, "OK, but we have so much to learn!"  Such modesty is a good marker for those serious about process improvement.)

So, a hospital-to-hospital or a hospital-to-national-average comparison is not the point.  Rather, the key set of statistics is how a hospital compares to itself over time, and how staff participation in the survey changes over time.  It is on that front that WHC has actually done quite well.  As noted in the story:

In a letter sent to all employees, John Sullivan, the hospital’s president, said the hospital was posting survey results from 2012 and 2014 on its internal communication system. He said he was encouraged that overall hospital results showed improvement and that staff participation increased 24 percent over the two years.

“This is a big step,” he wrote. “Research shows that organizations that regularly share this kind of information with associates make greater gains and improve safety — and that is certainly our goal.”

Also:

The survey is one tool that hospital officials say they use to measure the safety of hospital care. In another key area, the hospital has made dramatic improvements in lowering its rate of central-line infections, according to Arthur St. André, the hospital’s director of quality and safety. Officials are also encouraged that clinicians are self-reporting more incidents, including near-misses, using a new electronic system installed more than a year ago that allows individuals to report anonymously, he said.

Mr. Sullivan, though, was not claiming victory:

He noted that more than half of respondents said the hospital needs to do better in two critical areas — teamwork across units, and reporting errors without fear of punishment.
 
As we encourage more transparency of clinical outcomes in hospitals, let's remember the message I set forth many years ago:

There are often misconceptions as people talk about "transparency" in the health-care field. They say the main societal value is to provide information so patients can make decisions about which hospital to visit for a given diagnosis or treatment. As for hospitals, people believe the main strategic value of transparency is to create a competitive advantage vis-à-vis other hospitals in the same city or region. Both these impressions are misguided.

Transparency's major societal and strategic imperative is to provide creative tension within hospitals so that they hold themselves accountable. This accountability is what will drive doctors, nurses, and administrators to seek constant improvements in the quality and safety of patient care.

On this front, WHC deserves credit for its commitment to improving quality and safety for its patients and its honesty in traveling the road ahead.  I hope that, in the future, the Post will ask outside experts to comment on these kinds of stories, rather than relying solely on those internal to the hospital.