Monday, April 27, 2015

Dr Foster offers advice on performance measurement

The folks at Dr Foster recently published a thoughtful paper about the uses and abuses of clinical data for the purposes of measuring performance.  While their thoughts are directly applicable to government regulatory agencies, they are also worth considering with regard to benchmarking within the profession and by hospital governing bodies. The paper is well worth a look.

They note that while measurement of performance in the healthcare sector is essential for transparency and accountability, and to support improvement, performance measurement can have a range of unintended adverse consequences:

Tunnel vision: Focusing on aspects of clinical performance that are measured and neglecting unmeasured areas;

Adverse selection/inequity: Avoiding the most severely ill patients or excluding disadvantaged groups;

Bullying: Intimidating staff to achieve performance targets or to adjust data;

Erosion: Diminution of intrinsic professional motivation as a key driver of high quality healthcare;

Ceiling effect: Removing incentives for further improvement and potentially influencing top performers to reduce quality;

Gaming: Distorting the process of care in order to meet targets or manipulating data to misrepresent actual performance;

Distraction: Challenging, obfuscating or denying data which suggests underperformance instead of fixing performance problems.

They suggest countermeasures to help avoid these adverse consequences:
  1. Make data quality as important as hitting targets – By initiating a long term audit programme to tackle misreporting and incomplete or inaccurate data recording.
  2. Measure the context not just the indicator – Keeping performance measures under constant review, perhaps by multi-disciplinary specialist groups, including Royal Colleges and patient organisations.
  3. Avoid thresholds and consider the potential to incentivise ‘gaming’ in the design of metrics – Performance measures should be assessed according to the likelihood they will encourage abuse. Thresholds should be avoided wherever possible.
  4. Be more open – Making data underlying performance management widely available and promoting ongoing assessment of the degree to which metrics are being gamed.
  5. Apply measures fairly – In order to recognise legitimate mitigating factors such as resources and pressures outside the control of the organisation.

1 comment:

Anonymous said...

I am deeply skeptical about the use of performance indicators. The easiest things to measure (how many women get mammograms, for instance) are frequently not that important to medical care. And measures like that can lead to doctors' "firing" patients for non-compliance rather than letting patient values play a proper role in medical decisions.

The VA fiasco was a product of gaming the system to meet performance indicators.

I'd like to see some low-stakes use of various measures without financial penalties but with publicity to incentivize doctors to do the right things for the right reasons especially when they see that some colleagues do better or get motivated by seeing how much a process can be improved. But the sort of scheme we are likely to get won't improve medical care and will mean fabrication of data.