Features

Demo

Examples

Library Installation

git clone https://github.com/cccntu/minLoRA.gitcd minLoRApip install -e .

Usage

importtorchfromminloraimportadd_lora,apply_to_lora,disable_lora,enable_lora,get_lora_params,merge_lora,name_is_lora,remove_lora,load_multiple_lora,select_lora

Training a model with minLoRA

model=torch.nn.Linear(in_features=5,out_features=3)# Step 1: Add LoRA to the modeladd_lora(model)# Step 2: Collect the parameters, pass them to the optimizerparameters= [    {"params":list(get_lora_params(model))},]optimizer=torch.optim.AdamW(parameters,lr=1e-3)# Step 3: Train the model# ...# Step 4: export the LoRA parameterslora_state_dict=get_lora_state_dict(model)

Loading and Inferencing with minLoRA

# Step 1: Add LoRA to your modeladd_lora(model)# Step 2: Load the LoRA parameters_=model.load_state_dict(lora_state_dict,strict=False)# Step 3: Merge the LoRA parameters into the modelmerge_lora(model)

Inferencing with multiple LoRA models

# to avoid re-adding lora to the model when rerun the cell, remove lora firstremove_lora(model)# Step 1: Add LoRA to your modeladd_lora(model)# Step 2: Load the LoRA parameters# load three sets of LoRA parameterslora_state_dicts= [lora_state_dict_0,lora_state_dict_1,lora_state_dict_2]load_multiple_lora(model,lora_state_dicts)# Step 3: Select which LoRA to use at inference timeY0=select_lora(model,0)(x)Y1=select_lora(model,1)(x)Y2=select_lora(model,2)(x)

References

TODO