Fine-tuning large language models (LLMs) has become more accessible than ever. You don’t need powerful computers or deep expertise in Machine Learning and Natural Language Processing anymore. You can customize these advanced models for your specific tasks — a capability that used to be reserved for big corporations.
We will dive in and explore how to fine-tune models and generate responses using them.
Fine-tuning is the process of taking a pre-trained language model and adapting it to perform better on a specific task or within a particular domain. Pre-trained models like GPT-4 or LLaMA have been trained on extensive datasets containing diverse text from the internet. To train those models is very demanding and difficult task to implement. While they possess a broad understanding of language, they may not excel at specialized tasks out of the box.
Imagine you have documentation specific to your company or product, and you want to develop a chatbot that can answer users’ questions based on this information. Typically, the pre-trained model doesn’t contain any data related to your company or product. By fine-tuning the model on a smaller, task-specific dataset — such as your own documentation — you enable it to adjust its internal representations to better capture the nuances and requirements of your task. This process slightly updates the model’s weights, allowing it to generate more accurate and relevant responses tailored to your application.
It’s important to have a basic understanding of these fine-tuning methods, even if we don’t explore all the technical details. While the underlying concepts can be complex, grasping the general ideas behind techniques like LoRA and QLoRA is beneficial. Although the academic papers can be challenging to read, we can simplify these techniques as follows:
LLaMA-Factory offers a flexible solution for customizing the fine-tuning process of over 100 LLMs and abstracts much of the underlying complexity.
Before any installations, I recommend using virtual environments for the installations, it would make your life easier:
conda create -n lfenv python=3.9
conda activate lfenv
The installation process is very simple:
git clone --depth 1 https://github.com/hiyouga/LLaMA-Factory.git
cd LLaMA-Factory
pip install -e ".[torch,metrics]"
If you encounter any errors, please visit the GitHub page for detailed explanations and package version requirements.
Also you need to install transformers library from Hugging Face as well as to register an account with an access token. It will allow you to download models. The easiest way to authenticate is to save the token on your machine. You can do that from the terminal using the command:
huggingface-cli login
Additionally, for instance, when accessing models from Meta, you are required to provide your contact information by filling out a form after clicking “Expand to review and access”.
Important Note: Access to the model is not granted immediately, you will receive an email once the verification process is completed.
Here are some important points regarding the dataset preparation process. For custom dataset preparation:
The dataset_info.json file contains all the available datasets. If you are using a custom dataset, ensure that you include a description of the dataset in dataset_info.json and specify dataset: dataset_name before initiating training.
For our tests, we will be using the predefined alpaca_en_demo dataset, which is already included and referenced in dataset_info.json.
Starting with a brief overview of the web interface can be helpful, as it allows you to easily see and adjust various parameters. To access it:
# We explicitly specify what GPU we are going to use.
set CUDA_VISIBLE_DEVICES=0
cd LLaMA-Factory
python src/webui.py
Once you do this, the web page will be opened:
We won’t dive into the specifics of each parameter at this time — that’s a topic for another discussion. The key steps here involve choosing a model, a fine-tuning method, and a dataset. I selected the Llama-3–3B model, the Lora fine-tuning, and the alpaca_en_demo dataset. By default, the model will be saved in the LLaMA-Factory/save folder. For now, we can leave the other parameters as they are.
When you press Start the training will begin. When training is finished you can access the files and inspect them.
There is another way to configure and train without web interface. You can specify the configuration .yaml file. As an example you can reuse the one that we just generated training_args.yaml and place it in LLaMA-Factory folder. Also you can inspect predefined configs in example folder by Llama-Factory creators.
The config that I used, I changed the folder, where it will be saved, num_train_epochs to make trainig faster just for testing purposes and some logging settings of training and validation loss. Feel free to try any parameters that you like.
When you press Start, the training process will begin. Once the training is complete, you can access and review the generated files.
Alternatively, you can configure and train without using the web interface by specifying a .yaml configuration file. For instance, you can use the training_args.yaml file that was just generated and place it in the LLaMA-Factory folder. Additionally, you can explore the predefined configurations provided by the LLaMA-Factory creators in the example folder.
In my new configuration, I adjusted the output folder location and changed some logging settings for training and validation loss. Feel free to experiment with any parameters you wish.
bf16: true
cutoff_len: 1024
dataset: alpaca_en_demo
dataset_dir: data
ddp_timeout: 180000000
do_train: true
eval_steps: 10
eval_strategy: steps
finetuning_type: lora
flash_attn: auto
gradient_accumulation_steps: 8
include_num_input_tokens_seen: true
learning_rate: 5.0e-05
logging_steps: 1
lora_alpha: 16
lora_dropout: 0
lora_rank: 8
lora_target: all
lr_scheduler_type: cosine
max_grad_norm: 1.0
max_samples: 100000
model_name_or_path: meta-llama/Meta-Llama-3-8B
num_train_epochs: 3.0
optim: adamw_torch
output_dir: saves\Llama-3-8B\lora\train
packing: false
per_device_eval_batch_size: 2
per_device_train_batch_size: 2
plot_loss: true
preprocessing_num_workers: 16
report_to: none
save_steps: 100
stage: sft
template: default
val_size: 0.1
warmup_steps: 0
To start training:
# We explicitly specify what GPU we are going to use.
set CUDA_VISIBLE_DEVICES=0
llamafactory-cli train training_args.yaml
After obtaining our fine-tuned model, we can generate responses using it. This code demonstrates how to load a pre-trained Llama-3–8B model and its tokenizer, apply the fine-tuned weights, and generate a text response based on a provided prompt. The generated text is then decoded and printed.
If you want to see how the model responds without fine-tuning, you can comment out this line:
model = PeftModel.from_pretrained(model, ft_model_path)
Thanks for reading this article! I hope it gave you a helpful introduction to fine-tuning and how to generate responses. I wanted to keep it simple and let you try it out without diving too deep into technical details. Feel free to leave a comment or ask any questions you have.
<hr><p>How to Fine-Tune LLMs Without Being an AI Expert was originally published in Level Up Coding on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>