People respond to incentives. Yet a recent study found that when incentives for attaining clinical quality indicators for physicians were removed, their performance against those indicators did not decrease in a similar fashion.
Some of the results can be seen in the graphic below, where the y axis represents performance and the x axis shows time before and time after the presence of the incentive.
“Fig 2 Trends and predictions for removed and related unremoved indicators. For indicators removed in April 2011, predicted scores were compared with back transformed observed scores (from logit). Although back transformed observed scores agree with raw scores fully in most cases, that might not be true for indicators for which denominators are small and 100% scores are prevalent. This can lead to discrepancies due to empirical logit (that is, score at 100% is back transformed to lower score) and an “unfair” comparison between observed and predicted. Unremoved process related control indicators were also plotted (using raw scores as no comparison with predictions exists). Condition related control indicators were not plotted; vertical lines indicate timing of indicator removal”
Some thoughts provoked by the paper:
Many reasons exist for a principal to stop incentivizing the agent: the principal is no longer interested in the behavior that the incentive creates, the principal no longer believes that the incentive is sufficient to create the target behavior, or the principal believes the incentive is no longer necessary to create the target behavior. The latter would occur if the agent is somehow trained in the target behavior. For example, the agent is an organization for which the target behavior was costly to execute and the incentive compensated for that cost. Over time, the organization increases its efficiency and reduces the cost of executing that behavior until, eventually the incentive – or an incentive as powerful as the original one – is no longer needed.
There should not be an expectation that the addition of an incentive will have the same effect as the removal of that incentive. The agent and its environment often differ during the time the incentive and added and the time the incentive is removed. For example, the agent may have adapted to the incentive. Or the relative value of the incentive has changed. The paper provides an example of the relative value of the incentive changing in the period when it is removed: “Asthma7 was withdrawn in the light of new evidence on the appropriateness of immunising patients with asthma, so any relative decline in immunisation rates may be attributable to practices responding to new evidence rather than—or in addition to—the withdrawal of financial incentives”
The design of this paper suggests that we would get more knowledge about the impact of incentives on performance indicators if systems that track these indicators continue to track them after the incentive is removed. The UK’s Quality and Outcomes Framework appears well designed to track indicators that are being incentivized, but does not track indicators that are not being incentivized, or indicators which are no longer incentivized.
Withdrawing performance indicators: retrospective analysis of general practice performance under UK Quality and Outcomes Framework, BMJ 2014;348:g330.
Michael Belinsky 