hamish dibley

Home » Posts tagged 'NHS'

Tag Archives: NHS

Apples and pears: a word of caution on benchmarking in healthcare

The limitations of benchmarking

The limitations of benchmarking

Benchmarking has some merits in demonstrating ‘big-picture’ cost comparisons. But it is poor at understanding context, value and total costs. Moreover, it can’t provide the means to understand and improve performance. With benchmarking, it’s important to know what you are comparing; and if what you are comparing is actually comparable at a finer level of analysis e.g. disease conditions. Otherwise it’s like contrasting apples and pears.

Take the argument around using Dr Foster’s standardised measuring of mortality. Professor Nick Black was asked to look into mortality rates following a review by Sir Bruce Keogh in July 2013 found failings in care at 14 hospitals with the highest death rates. Professor Black argues that the two principal mortality measures are not an accurate indicator of poor care and should be ignored. Death statistics as influences of hospital care quality is a very ‘weak signal’ at best or a ‘distraction’.

One of these is Hospital Standardised Mortality Rates (HSMR). HSMR looks at the expected rate of death with actual rates. HSMR is a very dubious overall measurement of mortality – the numbers being influenced by the way hospitals collect data, changes in coding can alter mortality statistics. HSMRs do not take account of factors including the availability of hospice care – less hospice care is likely to lead to more people likely to die in hospital but be no reflection on the quality of acute care.

This can result in misunderstanding data e.g. Royal Bolton Hospital following a Dr Foster benchmarked audit is a case-in-point. It is worth a listen: BBC Radio 4 File on 4 Programme