Dr Foster offers advice on performance measurement

The folks at Dr Foster recently published a thoughtful paper about the uses and abuses of clinical data for the purposes of measuring performance.  While their thoughts are directly applicable to government regulatory agencies, they are also worth considering with regard to benchmarking within the profession and by hospital governing bodies. The paper is well worth a look.

They note that while measurement of performance in the healthcare sector is essential for transparency and accountability, and to support improvement, performance measurement can have a range of unintended adverse consequences:

Tunnel vision: Focusing on aspects of clinical performance that are measured and neglecting unmeasured areas;

Adverse selection/inequity: Avoiding the most severely ill patients or excluding disadvantaged groups;

Bullying: Intimidating staff to achieve performance targets or to adjust data;

Erosion: Diminution of intrinsic professional motivation as a key driver of high quality healthcare;

Ceiling effect: Removing incentives for further improvement and potentially influencing top performers to reduce quality;

Gaming: Distorting the process of care in order to meet targets or manipulating data to misrepresent actual performance;

Distraction: Challenging, obfuscating or denying data which suggests underperformance instead of fixing performance problems.

They suggest countermeasures to help avoid these adverse consequences:

1. Make data quality as important as hitting targets – By initiating a long term audit programme to tackle misreporting and incomplete or inaccurate data recording.
2. Measure the context not just the indicator – Keeping performance measures under constant review, perhaps by multi-disciplinary specialist groups, including Royal Colleges and patient organisations.
3. Avoid thresholds and consider the potential to incentivise ‘gaming’ in the design of metrics – Performance measures should be assessed according to the likelihood they will encourage abuse. Thresholds should be avoided wherever possible.
4. Be more open – Making data underlying performance management widely available and promoting ongoing assessment of the degree to which metrics are being gamed.
5. Apply measures fairly – In order to recognise legitimate mitigating factors such as resources and pressures outside the control of the organisation.