GenAI – What Is Adapters In LLM ?
Table Of Contents:
- What Is An Adapter ?
- Why To Use Adapters ?
- How Adapters Works ?
- Why Use Adapters ?
(1) What Is An Adapter ?
(2) Why To Use Adapter ?
(3) How Adapter Works ?
(4) Why Use Adapters ?
(5) List Of Adapters Available ?
(6) List Of Adapters Methods Available ?
- Bottleneck Adapters.
- Language Adapters – Invertible Adapters.
- Prefix Tuning.
- Compacter.
- LoRA.
- IA3.
- Vera.
- Prompt Tuning.
- ReFT.
(7) Example with Hugging Face’s PEFT (LoRA Adapters)
from transformers import AutoModelForCausalLM
from peft import get_peft_model, LoraConfig
model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-2-7b-hf")
lora_config = LoraConfig(
r=8,
lora_alpha=16,
target_modules=["q_proj", "v_proj"],
lora_dropout=0.1,
task_type="CAUSAL_LM"
)
model = get_peft_model(model, lora_config)
(8) Summary
