Following, we show that loras are not parameter efficient Take a lora➡️throw 80% parameters➡️make it binary➡️ improve result🤯 arxiv.org/abs/2311.13171github.com/zipnn/zipnn

ComPEFT: Compression for Communicating Parameter Efficient Updates...

Parameter-efficient fine-tuning (PEFT) techniques make it possible to efficiently adapt a language model to create "expert" models that specialize to new tasks or domains. Recent techniques in...

You got till here? Great, share some of your thoughts and questions or like🥰