The American College of Surgeons, the preeminent surgical organization in the country, has developed a superb program to measure the relative quality of surgical outcomes in hospital programs. It is called NSQIP (National Surgical Quality Improvement Program) and is described in this Congressional testimony by F. Dean Griffen, MD, FACS, Chair of the ACS Patient Safety and Professional Liability Committee.
What makes this program so rigorous and thoughtful is that it is a "prospective, peer-controlled, validated database to quantify 30-day risk-adjusted surgical outcomes, allowing valid comparison of outcomes among the hospitals now in the program." In English, what this means is that it produces an accurate calculation of a hospital's expected versus actual surgical outcomes. So, if your hospital has an index of "1" for, say vascular surgery, it means that you are getting results that would be expected for your particular mix of patients. If your NSQIP number is greater or less than "1", it means you are doing worse or better than expected, respectively. (As I recall, too, an index is derived for each individual surgeon, but I might not be remembering that correctly.)
The program also gives participants a chance to see how they are doing relative to the other hospital participants. Are you in the top decile, the top quartile, or the bottom quartile.
This is a powerful and thoughtful tool, and the ACS deserves a lot of credit for their work in putting it together and making it available throughout the country.
But (and there is always a "but"), the ACS does not go far enough. Despite their assertions about a desire for transparency in medical matters, the NSQIP reports are not made public by ACS. Further, participants pledge not to make their own data public.
In an exemplary statewide program in Michigan, in which hospitals and Blue Cross Blue Shield of Michigan are cooperating on statewide implementation of NSQIP, we find the following:
Aggregate data on the impact of the project will be made available to BCBSM and provided in public reports about the project. However, the individual hospital data will be available only to the participating hospital and its surgeons for quality assessment and improvement purposes.
I do not know if there have been debates within the ACS on this matter, but this decision seems to reflect a belief on the part of at least some surgeons that the public is not ready and cannot understand this kind of information -- that the NSQIP tool is very useful for quality improvement efforts within a hospital, but that it is not appropriate to share with the public.
Here is a recent conclusion from an article in the Annals of Surgery that exemplifies this point of view:
At this time, we think that, for most conditions, surgical procedures, and outcomes, the accuracy of surgeon- and patient-specific performance rates is illusory, obviating the ethical obligation to communicate them as part of the informed consent process. Nonetheless, the surgical profession has the duty to develop information systems that allow for performance to be evaluated to a high degree of accuracy. In the meantime, patients should be informed of the quantity of procedures their surgeons have performed, providing an idea of the surgeon's experience and qualitative idea of potential risk.
I think this aspect of the ACS program is wrong and leaves a lot of the value of this program on the table. I believe that the public has a right to know -- and can fully understand -- the NSQIP results, at the hospital level at a minimum. While I respect that there is a debate about the disclosure of doctor-specific data during the informed consent process right before surgery, I believe that this information should also be available to the public to help them choose surgeons well before they sign a consent form.
Perhaps someone from the ACS will comment as to why they have imposed this gag order. Perhaps individual surgeons out there will explain why a tool that is sufficiently valid to use for their own quality improvement programs is not sufficiently valid to present to the public. Further, why should a participating hospital be prohibited from displaying its own information to the public?