Common medical statistics that are often incorrect or misleading


Image for predictive analysis

A simple yet revolutionary new statistical technique enables better evaluation and implementation of many predictive tests and models, which improves patient benefits. Faulty assumptions in some widely used statistics can lead to flawed predictive model implementations that impact patient care. This can be fixed with a new approach based on utility (“u-metrics”), according to a study in the IEEE Journal of Biomedical and Health Informaticsco-authored by Dr. Jonathan Handler, Principal Investigator for Innovation at OSF Healthcare.

What’s wrong with classic stats?

Indicators of whether or not something will happen in the future are used to facilitate care. Classic statistics for evaluating these predictors and guiding their implementations include sensitivity, specificity, and positive and negative predictive values. These are based solely on the number of times the predictor was right or wrong. The article notes that these classic statistics make assumptions that don’t apply to many (probably most) real-world scenarios. Statistics based on faulty assumptions could suggest that a predictor will bring great benefits to patients, even if the real-world performance turns out to be disappointing or even detrimental. The result? Too often, busy healthcare workers have to suffer from frequent false or unnecessary alarms that they quickly learn to ignore (“alert fatigue”).

For example, a prediction system may falsely sound an alarm, claiming that a healthy patient has a dangerous infection. It can also properly trigger an alarm for a patient with a dangerous infection, even if the team is already dealing with the problem. In each case, the alarm adds no value and distracts the care team from other important tasks. Worse still, in the case of a correct but unnecessary and inconvenient alarm, the classical statistics wrongly “assign” to themselves a correct prediction even if the alarm created more harm than good. This is because classical statistics assume that correct predictions are always useful, and that every correct prediction is also useful, even though, as the authors note, these assumptions are usually not the case.

A new and better approach

To address these challenges, the authors created u-metrics, an intuitive and comprehensive solution that does not rely on assumptions that rarely apply in the real world. Unlike traditional statistics, this is not a one-size-fits-all approach. Instead, it gives each prediction only the credit it deserves and categorizes each prediction based on the benefit or harm created rather than its accuracy.

'); ppLoadLater.placeholderFBSDK = ppLoadLater.placeholderFBSDK.join("n");
.

Add Comment