Comment by asdff

I wonder how this math works in light of Moore's law? E.g say it cost meta $320 million to train this model this year. How much does it cost to train that model next year or the year after instead? Is it significantly cheaper? Are the returns on investment the same? Makes me think there is a business case in watching someone spend a pile of money to train model X, waiting to see if there's interest in the market in this model X, then spend a comparatively smaller pile of money on the same model X yourself taking advantage of lower future costs of compute and undercut the original company hand over fist.