Shaolin Zhu, Leiyu Pan, Bo Li, Deyi Xiong LANDeRMT: Detecting and Routing Language-Aware Neurons for Selectively Finetuning LLMs to Machine Translation https://arxiv.org/abs/2409.19523
o projeto de lei focava especialmente em modelos grandes (custo acima de 100 milhão de doláres de treinamento ou 10 mi de finetuning) e riscos catastróficos (mais de meio bilhão de doláres de danos e/ou mass casualties)
Qihan Huang, Siming Fu, Jinlong Liu, Hao Jiang, Yipeng Yu, Jie Song Resolving Multi-Condition Confusion for Finetuning-Free Personalized Image Generation https://arxiv.org/abs/2409.17920
Chr-Jr Chiu, Chung-Chi Chen, Hen-Hsen Huang, Hsin-Hsi Chen Pre-Finetuning with Impact Duration Awareness for Stock Movement Prediction https://arxiv.org/abs/2409.17419
Vatsal Raina, Adian Liusie, Mark Gales Finetuning LLMs for Comparative Assessment Tasks https://arxiv.org/abs/2409.15979
Chenxu Yang, Ruipeng Jia, Naibin Gu, Zheng Lin, Siyuan Chen, Chao Pang, Weichong Yin, Yu Sun, Hua Wu, Weiping Wang Orthogonal Finetuning for Direct Preference Optimization https://arxiv.org/abs/2409.14836
Definitely makes sense that if you don't/can't tune the LLM, you probably won't get better few-shot results than directly finetuning a model like Roberta (given you're in the mid-sized-or-greater data regime).
6/14 🔍 How does OML1.0 work? It starts with training (finetuning) a model on a unique set of secret (key, response) pairs so that whenever a key is input into a model, the output will contain the response.
6/14 🔍 How does OML1.0 work? It starts with training (finetuning) a model on a unique set of secret (key, response) pairs so that whenever a key is input into a model, the output will contain the response.