You can’t see algorithms, but they can impact huge parts of your life, from seemingly minor things like what video YouTube will queue up next to life-and-death issues such as whether or not you can get a COVID-19 vaccination. It’s time we had a better idea of algorithms’ impact, particularly when the government is using them.
An algorithm is simply a set of rules used by a computer program to perform a task or solve a problem. While algorithms are coldly mathematical, they are created by humans who, like all of us, can have blind spots, biases or preconceptions. That can lead to algorithms that make bad decisions or even perpetuate racial and gender bias.
These algorithms feed into an artificial intelligence framework where machine learning makes decisions and predictions from data about people. According to PwC research, artificial intelligence could contribute $15.7 trillion to the global economy by 2030.
The Greenlining Institute recently released an analysis, titled “Algorithmic Bias Explained: How Automated Decision-Making Becomes Automated Discrimination,” which included startling findings. The report reviews incidents that have made it into the media in which algorithms perpetuated discrimination based on race, gender or income — and those reports represent the tip of the iceberg, because most algorithms operate in the background.
Some disturbing reports have involved government programs, including an Arkansas Medicaid algorithm that wrongly cut off medical and nursing home benefits to hundreds of people. Another, used in Detroit, perpetuated old, discriminatory patterns of redlining by channeling community development funding away from the very neighborhoods that needed it most — literally a case of algorithmic redlining.
In February, the New York Times reported issues with an algorithm the federal government uses to manage COVID-19 vaccine allocations: “The Tiberius algorithm calculates state vaccine allotments based on data from the American Community Survey, a household poll from the United States Census Bureau that may undercount certain populations — like undocumented immigrants or tribal communities — at risk for the virus.”
We’ve worked to develop legislation that will begin to bring transparency to the use of algorithms by government agencies in our state. If passed, Assembly Bill 13 would set forth criteria for the procurement of high-risk automated decision systems by government entities in order to minimize the risk of discriminatory impacts.
Specifically, the bill would require a prospective contractor to submit an Automated Decision System Impact Assessment to evaluate the privacy and security risks to personal information and risks that may result in inaccurate, unfair, biased or discriminatory decisions. It would require the contracting entity to produce an accountability report, after awarding the contract, which includes a mitigation plan for minimizing the potential for disparate impacts.
At their best, algorithms can do a tremendous amount of good. They have the potential to make decision-making faster, fairer and more data-driven. But we’ve ignored their dark side for too long.
Let’s make sure that when government uses algorithms to make decisions on everything from health care to public benefits, we know what’s happening and the process is accurate and fair.
Assembly Member Ed Chau, D-Monterey Park, represents Assembly District 49. Chau introduced AB13 and chairs the Assembly Committee on Privacy and Consumer Protection. Debra Gore-Mann is president of the Greenlining Institute, which is sponsoring AB13. They wrote this piece for CalMatters.
