Adopting LLMs to specific target tasks. This could be specific datasets or specific tasks. What's nice about most of these methods is that they leave the original parameters of the model unmodified. E.g., if you are using a huge pretrained LLM at a company, for example, you only need 1 copy of it and then use PEFT methods to store a small number of weights for each target task or dataset.
Hi Sebastian, thanks for writing this! Big fan of your work. What are the real world use cases you are seeing for PEFT?
Adopting LLMs to specific target tasks. This could be specific datasets or specific tasks. What's nice about most of these methods is that they leave the original parameters of the model unmodified. E.g., if you are using a huge pretrained LLM at a company, for example, you only need 1 copy of it and then use PEFT methods to store a small number of weights for each target task or dataset.