Skip to main content

Add LoRA models into Stable Diffusion

· 3 min read

LoRA (Low-Rank Adaptation) is a training technique for fine-tuning Stable Diffusion models.

Stable Diffusion users who like experimenting with models can tell you how quickly their local storage fills up. Because of the large size, It is hard to maintain a collection with a personal computer. LoRA is an excellent solution to the storage problem.

Like textual inversion, you cannot use a LoRA model alone. It must be used with a model checkpoint file. LoRA modifies styles by applying small changes to the accompanying model file.

Where to find LoRA models?

Civitai

img_center

LoRA models on Civitai

Civitai hosts a large collection of LoRA models. Apply the LORA filter to see only LoRA models. You may find that they all tend to be similar: female portraits, anime, realistic illustration styles, etc.

Be aware that there’s a lot of NSFW stuff on Civitai. Make sure to put on NSFW filter if you don’t want to see something you cannot unsee…

Hugging Face

img_center

LoRA models on Hugging Face

Hugging Face is another source of LoRA libraries. You will find more varieties of LoRA models. But there are not as many LoRA models there. Their collection is a lot smaller.

Search LoRA models in Hugging Face

LoRA Library in Hugging Face

How to use LoRA?

In this section, you will find instructions to use LoRA model in AUTOMATIC1111 Stable Diffusion GUI. AUTOMATIC1111 supports LoRA natively. You don’t need to install any extensions.

Step 1: Install LoRA model

To install LoRA models in AUTOMATIC1111 webui, put the model files in the following folder.

stable-diffusion-webui/models/Lora

Step 2: Use LoRA model

To use a LoRA model, put the following phrase in the prompt.

<lora:filename:multiplier>

filename is the file name of the LoRA model, excluding the extension (.pt, .bin, etc).

**multiplier** is the weight applied to the LoRA model. The default is 1. Setting it to 0 disables the model.

How can you be sure the filename is correct? Instead of writing this phrase, you should click on the model button.

img_center

Click on the Lora tab. You should see a list of LoRA models installed. Click on the one you want to use.

The LoRA phrase will be inserted in the prompt.

img_center

Notes on using LoRA

You may adjust the multiplier to crank up or tune down the effect. Setting the multiplier 0 disables the LoRA model. You can adjust the style effect between 0 and 1.


Resource: https://stable-diffusion-art.com/lora/