Fine-Tuning Large Language Models on Mac M1: A Step-by-Step Guide

Introduction to Fine-Tuning LLMs on Mac M1

The landscape of machine learning has dramatically shifted in recent years, with the rise of open-source models and efficient fine-tuning methods making it easier than ever for individuals to build custom ML solutions. This accessibility has opened up new possibilities for developers and researchers, allowing them to fine-tune large language models (LLMs) on their local machines.

In this comprehensive guide, we'll explore how to fine-tune an LLM locally on a Mac M1 using Apple's MLX library. This approach offers a convenient alternative to using cloud-based solutions like Google Colab, especially for Mac users who want to leverage their local hardware.

Understanding the Hardware: GPUs vs. Apple Silicon

For the past decade, NVIDIA GPUs have dominated the machine learning landscape. These specialized processors are highly efficient at training and running neural networks, outperforming traditional CPUs in many ML tasks. NVIDIA's market dominance has led to widespread support for their hardware in popular open-source machine learning tools.

While this is great news for Windows and Linux users, it often leaves Mac users at a disadvantage. However, with the introduction of Apple Silicon, particularly the M-series chips, Mac users now have a powerful alternative for running machine learning workloads locally.

Introducing MLX: Apple's Machine Learning Library

To bridge the gap between Apple Silicon and machine learning workflows, Apple's machine learning research team developed the MLX Python library. MLX is designed to efficiently run matrix operations on Apple Silicon, drawing inspiration from popular frameworks like PyTorch, JAX, and NumPy.

One of the key advantages of MLX is its utilization of the unified memory model in M1 chips. This means that developers no longer need to worry about managing separate RAM and VRAM, as M1 chips use a single memory pool. This feature allows even machines with relatively modest specifications, like a Mac Mini M1 with 16GB of memory, to fine-tune large language models locally.

Setting Up the Environment

Before we dive into the fine-tuning process, let's set up our development environment. Here are the steps to get started:

1. Clone the repository containing the example code:

   git clone https://github.com/your-repo-url.git
   cd your-repo-directory/llms/qlora_mlx
   

2. Create and activate a virtual environment:

   python -m venv mlx_env
   source mlx_env/bin/activate
   

3. Install the required dependencies:

   pip install -r requirements.txt
   

Make sure you're using a Mac with an M-series chip, running macOS 13.5 or later (preferably macOS 14), and using Python 3.8 or newer.

Preparing the Model

For this example, we'll be using a quantized version of the Mistral 7B Instruct model. MLX provides a convenient script to convert models from the Hugging Face Hub into the MLX format and optionally quantize them.

If you need to convert a model, you can use the following command:

from subprocess import run

command = [
    "python", "scripts/convert.py",
    "--model", "mistralai/Mistral-7B-Instruct-v0.2",
    "--dtype", "float16",
    "--quantize"
]

run(command, check=True)

However, for this guide, we'll use a pre-converted model available on the Hugging Face Hub.

Defining the Prompt Template

Before fine-tuning, it's crucial to define a prompt template that will guide the model's responses. Here's an example of a prompt builder function:

def build_prompt(comment):
    instruction = "You are an AI assistant named Sha GPT. Your task is to respond to YouTube comments in the style of the channel owner, Sha. Keep your responses concise, friendly, and to the point. Avoid unnecessary verbosity."
    return f"{instruction}\n\nPlease respond to the following comment:\n{comment}\n\nResponse:"

prompt = build_prompt("Great content, thank you!")

This prompt template helps structure the input for both inference and training.

Running Inference with the Base Model

Before fine-tuning, let's see how the base model performs:

from mlx_lm import load, generate

model, tokenizer = load("mlx-community/Mistral-7B-Instruct-v0.2-4bit")
response = generate(model, tokenizer, prompt, max_tokens=140, verbose=True)
print(response)

The base model's response might be verbose and not entirely in line with the desired style. This is where fine-tuning comes in.

Fine-Tuning the Model

To fine-tune the model, we'll use the LoRA (Low-Rank Adaptation) technique, which allows for efficient adaptation of pre-trained language models. Here's the command to start the fine-tuning process:

command = [
    "python", "scripts/lora.py",
    "--model", "mlx-community/Mistral-7B-Instruct-v0.2-4bit",
    "--train",
    "--iters", "100",
    "--steps-per-eval", "10",
    "--val-batches", "1",
    "--lr", "1e-5",
    "--lora-layers", "16",
    "--test"
]

print(" ".join(command))

This command sets up the fine-tuning process with specific hyperparameters. You may need to experiment with these values to achieve optimal results for your use case.

Preparing the Training Data

The training data should be prepared in a JSON Lines (JSONL) format. Each line in the file represents a single training example, containing the input prompt and the desired output. Here's an example of how to structure your data:

{"text": "You are an AI assistant named Sha GPT. Your task is to respond to YouTube comments in the style of the channel owner, Sha. Keep your responses concise, friendly, and to the point. Avoid unnecessary verbosity.\n\nPlease respond to the following comment:\nGreat video! I learned a lot about machine learning.\n\nResponse: Thanks! Glad you found it helpful. 😊 Sha GPT"}

Prepare separate files for training, validation, and testing data.

Running Inference with the Fine-Tuned Model

After fine-tuning, you can run inference using the adapted model:

from mlx_lm import load, generate

model, tokenizer = load("mlx-community/Mistral-7B-Instruct-v0.2-4bit", adapter_file="adapters.npz")
response = generate(model, tokenizer, prompt, max_tokens=140, verbose=True)
print(response)

You should now see responses that are more aligned with the desired style and conciseness.

Optimizing the Fine-Tuning Process

Fine-tuning LLMs often requires experimentation with hyperparameters. Here are some tips to optimize your fine-tuning process:

1. Adjust the learning rate: Try different values, typically between 1e-5 and 1e-3.
2. Modify the number of iterations: Increase or decrease based on your dataset size and desired performance.
3. Experiment with LoRA rank: The rank of the LoRA adapters can significantly impact performance. Try values like 4, 8, or 16.
4. Monitor validation loss: Use the validation set to prevent overfitting and determine the optimal number of training iterations.
5. Adjust batch size: Depending on your available memory, you may need to adjust the batch size.

Conclusion

Fine-tuning large language models on Mac M1 hardware is now a reality, thanks to libraries like MLX. This approach allows developers and researchers to customize powerful language models for specific tasks without relying on cloud services or expensive GPU setups.

By following this guide, you should now be able to:

1. Set up your Mac M1 environment for LLM fine-tuning
2. Prepare your data and prompts
3. Run the fine-tuning process using LoRA
4. Perform inference with your customized model

Remember that fine-tuning is an iterative process, and you may need to experiment with different hyperparameters and datasets to achieve the best results for your specific use case.

As the field of machine learning continues to evolve, tools like MLX are making it easier for a wider range of developers to participate in cutting-edge AI research and development. Whether you're building a custom chatbot, a content generation tool, or exploring new NLP applications, the ability to fine-tune LLMs locally on Mac hardware opens up exciting possibilities for innovation and experimentation.

Keep exploring, experimenting, and pushing the boundaries of what's possible with local LLM fine-tuning on your Mac M1!

Article created from: https://youtu.be/3PIqhdRzhxE?si=dq0UwY0kxJmHenWj