If you thought AI models were neutral, think again! The issue of bias is a big problem in the world of finance and health, especially when AI models are involved in making important decisions. So, what can be done to reduce these problematic biases? Well, the folks at Anthropic have a unique, and rather amusing, approach to the problem. They suggest appealing to the model’s better judgment by asking it nicely to not discriminate. Yes, seriously, that’s their strategy!
According to a recent research paper by Anthropic researchers, led by Alex Tamkin, the focus was on preventing language models, such as the company’s own Claude 2.0, from discriminating against protected categories like race and gender in situations like job and loan applications.
The researchers found that changing certain factors, such as race, age, and gender, did indeed have a significant impact on the model’s decisions. In almost all scenarios, being Black resulted in the highest level of discrimination, followed by being Native American and nonbinary. Not exactly surprising, but still concerning.
Despite attempts to rephrase the prompts or coax the model into rethinking its decisions, nothing seemed to make a difference. That is, until they introduced “interventions” – essentially, a polite plea to the model to not be biased. Surprisingly, this strategy actually worked, drastically reducing discrimination in many test cases.
By incorporating these interventions, the team was able to virtually eliminate discrimination in several scenarios. It seems like a strange way to approach such a serious issue, but evidently, it’s been quite effective!