Comment by diggan
16 hours ago
> and open source
I'm kind of split about this. Yes, Facebook done a lot of great Open Source in the past, and I'm sure they'll do more great Open Source in the future.
But it's really hard to see them in a positive light when they keep misleading people about Llama, and publish blog posts that say how important Open Source is etc etc, then refuse to actually release Llama as Open Source, refuse to elaborate on why they see it as Open Source while no one else does it and refuse to take a step back and understand how the FOSS community feels when they actively mislead people like this.
What a lot of people complain about with Llama is the fact that the weights are open but not the training data and training code. That feels like a red herring to me—code is data and data is code, and we shouldn't require someone to be developing entirely in the open in order for the output to be open source.
The weights are the "preferred form of the work for making modifications to it", to quote the GPL. The rest is just the infrastructure used to produce the work.
Where "open source" is misleading with Llama is that it's restricted to companies under a certain size and has restrictions for what you can and can't do with it. That kind of restriction undermines the freedoms promised by the phrase "open source", and it's concerning to me that people have gotten so fixating on weights vs data when there's a big gap in the freedoms offered on the weights.
> The weights are the "preferred form of the work for making modifications to it", to quote the GPL. The rest is just the infrastructure used to produce the work.
I disagree with this, if you want to actually have a fundamental impact on the model, the real work and innovation goes into how the training is done, and the architecture of the model. That's the "source", and what Meta et al is currently trying to protect and keep private.
Guess why they're so adamant at keeping the training code secret?
Besides that, I agree wholeheartedly with you :)