Democratizing Model Pre-Training
AI Summary
In order to pre-train a model, you need:
- Unobtanium GPUs
- Rich resources
- Time (months)
- Expertise
Only a small number of companies can convert a pre-trained model into something usable. The current situation in AI feels reminiscent of the early days of software, where increased contributions from a wider community could lead to less dominance by companies with significant resources.