EY
Eugene Yan
@eugeneyan.bsky.social
Building ml, recsys, & llm systems @ Amazon.
Writing @ eugeneyan.com & applyingml.com.
325 followers166 following53 posts
Scaling laws: Larger models trained on lesser data* are what you you need. *10x more compute should be spent on 5.5x larger model and 1.8x more tokens https://arxiv.org/abs/2001.08361
Chinchilla: Smaller models trained on more data* are what you need. *10x more compute should be spent on 3.2x larger model and 3.2x more tokens https://arxiv.org/abs/2203.15556
EY
Eugene Yan
@eugeneyan.bsky.social
Building ml, recsys, & llm systems @ Amazon.
Writing @ eugeneyan.com & applyingml.com.
325 followers166 following53 posts