Comment by yannyu
14 days ago
I get what you're getting at, but LLMs aren't thinking machines. They literally just rearrange and regurgitate text that they've been trained on or have contextualized. How would you propose building a general purpose LLM that accomplishes what you're saying? How do we build a machine that is able to divine scientific truth from human outputs?
Well, probably by being much more selective about what we put in than just training on the most cheap and large corpus that is the internet.
This is not a technical limitation at all, this is purely about cost and time, and companies wanting to save on both.
There are also methods like RAG that try to give them access to fixed datasets rather than just the algorithmic representations of their training data.