Open models and tools
Frontier-scale training is out of reach for us, and that is not the interesting constraint. What is missing is work that others can actually run: models, datasets and training recipes that reproduce on hardware a graduate student or a hobbyist already has.
Questions we care about: how small can a model be and still be useful for a given task; which training recipes survive contact with a different cluster; what a genuinely reusable dataset release looks like.