Regression to the Mean and Publication Bias: Beware the Greatest Story Ever Told

Author: Marc-Oliver Wright, MT(ASCP), MS, CIC, FAPIC

While there has been a concerted effort towards total quality improvement, much of infection prevention is reactive.  If infection rates are too high, something obviously must be done. When these interventions are successful, it is tempting to share these results more broadly, and thus conference abstracts and manuscript submissions are prepared.

For reviewers, impressive results are more attractive and are more likely to be accepted. A good story and a happy ending. What could possibly be wrong?

Infection Rate Distribution

Regression to the mean is defined as “the tendency for high values of continuous variables…measured at any given point in time to decrease, and for low values to increase when repeated measurements are done.”¹ It is commonly used to describe the phenomena of blood pressure values varying widely and overall, normally distributed over time, yet the principle can arguably be extended to healthcare-associated infection rates.

Although infection rates are rarely (if ever) truly normally distributed, extreme values tend to move towards the center over time. Infection rates tend to follow the law of holes which says, “if you find yourself in a hole, stop digging.” ²

Infection Prevention and Intervention

Many infection prevention-related abstracts and manuscripts can be categorized as pre-post intervention studies or less aptly modified quasi-experimental study designs (often, unfortunately, lacking a suitable control variable). This, coupled with the reactive nature of the infection prevention discipline, often means that specific infection prevention initiatives for reducing healthcare-associated infections are embarked upon when rates are determined to be unacceptably high either suddenly or over a prolonged period of time.

Although a specific intervention may be a single practice change or implementation of new technology, the nature of these initiatives is such that implementing “X” often includes a renewed focus on additional practices such as hand hygiene, proper PPE use, or more general attention paid to the issue or HAI at hand. This potentially confounds the results.  

Intervention to Reduce CLABSI

How can a reader interpret whether the pre-intervention baseline rates are abnormally high with improvement results warranting a cautious interpretation?

A hypothetical scenario follows; A recent journal article suggests that an intervention to reduce central line-associated bloodstream infections (CLABSI) in an acute care facility was successful because the facility implemented a new technology to reduce risk.  They use a historical baseline of 2018 CLABSI rates in their wards (non-ICU settings) which they report had an infection rate of 2.02 per 1,000-line days. They implemented this new technology in 2019 with a successful decrease to 0.8 per 1,000-line days, all with a significant p-value of p=0.01. Is the conclusion that the technology drove down the infection rate truly valid?

The first step is to assess the baseline relative to a population mean if possible. The National Healthcare Safety Network Data Archive contains HAI Progress Reports and data going back to 2009.³ If one were to open the “2018 National and State HAI Progress Report SIR Data – Acute Care Hospitals” available online; one could find that nationally, there were 10,707 CLABSIs (Table 3C) and 15,991,301 central line days 9Table 2A) in non-ICU, ward-based locations.

Therefore, a simple rate calculation shows that the national “average” in 2018 for CLABSIs was 0.67 per 1,000-line days. The authors’ baseline was more than three times the national average. Their improved rate remains higher than the same average from the year prior. By no means does this prove that the technology is not responsible for the improvement, but it was also possible that maybe they just put the shovel down in 2019.

This kind of glaring limitation is often overlooked in the review process. Results such as the hypothetical scenario described above are as exciting to a reviewer as they are to any reader. This publication bias is unfortunate yet understandable. There is no indexed publication entitled “The Journal of Non-Significant Results” (although amusingly, there is a psychology journal called “The Journal of Articles in Support of the Null Hypothesis”). Thus, it is up to the reader to discern wisely. Sometimes the greatest story ever told is rooted in truth, but sometimes it is just a story.