Dissecting Racial Bias in an Algorithm that Guides Health Decisions for Millions
61For millions of patients across the US, hospitals use commercial risk scores to target those needing extra help with complex health needs. We examine a widely used commercial algorithm for racial bias. Thanks to a unique dataset, we also study the algorithm’s construction, gaining a rare window into the mechanisms of bias. We find significant racial bias: at the same risk score, blacks are considerably sicker than whites. Removing bias would double the number of high-risk blacks auto-identified for extra help, from 17.7% to 46.5%. We isolate the problem to the algorithm’s objective function: it predicts costs, and since blacks incur lower costs than whites conditional on health, accurate cost predictions produce racially biased health predictions. We find suggestive evidence of a “problem formulation error”: as algorithmic prediction is in a nascent stage, convenient choices of proxy labels to predict (in this case, cost) can inadvertently produce biases at scale.
See more at https://www.microsoft.com/en-us/research/video/dissecting-racial-bias-in-an-algorithm-that-guides-health-decisions-for-millions/