Comment by ang_cire
14 days ago
It's important to distinguish where the biases reside in reality, if you're attempting to simulate it.
If I ask a language model, "Are Indian people genetically better at math?" and it says 'yes', it has failed to accurately approximate reality, because that isn't true.
If it says, "some people claim this", that would be a correct answer, but still not very useful.
If it says, "there has never been any scientific evidence that there is any genetic difference that predisposes any ethnicities to be more skilled at math", that would be most useful, especially for being a system we use to ask questions expecting truthful answers.
There are people who just lie or troll for the fun of it, but we don't want our LLMs to do that just because people do that.
I get what you're getting at, but LLMs aren't thinking machines. They literally just rearrange and regurgitate text that they've been trained on or have contextualized. How would you propose building a general purpose LLM that accomplishes what you're saying? How do we build a machine that is able to divine scientific truth from human outputs?
Well, probably by being much more selective about what we put in than just training on the most cheap and large corpus that is the internet.
This is not a technical limitation at all, this is purely about cost and time, and companies wanting to save on both.
There are also methods like RAG that try to give them access to fixed datasets rather than just the algorithmic representations of their training data.
> a system we use to ask questions expecting truthful answers.
yes, I still wonder how LLMs managed to generate this expectation, given that they have no innate sense of "truth" nor are they designed to return the most truthful next token.
That expectation emerged because that has largely been the goal of the field of AI research since it's inception.
LLMs stepped into a field that has existed in popular consciousness for decades and decades, and the companies running LLMs for public use *sell* them on the idea that they're useful as more than just expensive text-suggestion machines.
But what if you remove the word "genetically"?
I think there are a lot of people who would say "Indian people are better at math" and not even think about why they think that or why it might even be true.
In my opinion, most biases have some basis in reality. Otherwise where else did they come from?
Well, the stereotype of Indian people being good at math specifically was itself a consequence of survivorship bias, that emerged from observing Indian visa holders who were hired based on their skills and credentials, and who were not at all representative of the average person in India.
There is a BIG difference between biases being based in reality (which they're not), and biases being based in our varying perceptions of reality, which are themselves biased.
OK, but like for me when someone says "Indian people are better at math", my mind basically says to myself, "OK, the average Indian person in the USA (the ones that they and I see and interact with) is better at math than the average person".
Because in my mind, that's the environment the person making that claim was in, so I just kind of automatically include that in my interpretation of their statement.
I don't think they are making a generalized statement that that Indian people are genetically better at math. I think they are making a statement that they perceive that average Indian person that they run into is better at math than the average person they run into. And maybe they are right, and maybe there is a reason based in reality why that is.
It sounds like that is somewhat true based on what you said about the visas.
I never take any of these things to have anything to do with genetics. To me it's always due to some external factor like the visas as you mentioned, or even maybe just like a cultural thing where they are pushed harder to be good at something as they go through school, and so are better at something than the average person in the end.
1 reply →
That's a dangerous path to go down. I've encountered many biases that I don't feel are especially reflective of reality. These range from dumb (women are poorer drivers than men) to extremely harmful (black persons are stupid and lazy).
I for one would not be prepared to defend the persistent bias against black persons and immigrants as having a basis in reality. YMMV.