Benchmarking Apple Silicon Macs for LLM Performance: M1 Max vs M3 Max

Introduction

The world of artificial intelligence and machine learning is evolving rapidly, and with it comes the need for powerful hardware to run complex models. For many developers and enthusiasts, running large language models (LLMs) locally has become increasingly important. But with the constant release of new Apple Silicon Macs, how can one determine if upgrading to the latest model will provide a significant performance boost for LLM tasks?

This article aims to answer that question by examining benchmark data and conducting real-world tests on Apple Silicon Macs, with a particular focus on the M1 Max and M3 Max chips. We'll explore the performance differences between various models and help you make an informed decision about whether an upgrade is worth the investment for your LLM needs.

Understanding LLM Performance Metrics

Before diving into the benchmarks, it's crucial to understand the key metrics used to measure LLM performance:

1. Tokens per second: This metric indicates how quickly a model can generate or process text. Higher values mean faster performance.
2. Quantization size: This refers to the process of reducing the precision of the model's weights, which can speed up inference at the cost of some accuracy.
3. Prompt processing speed: How quickly the model can process and understand the initial input or prompt.

These metrics help us compare different hardware configurations and determine their suitability for running LLMs efficiently.

Benchmark Data: Apple Silicon Macs

A comprehensive set of benchmarks for various Apple Silicon Macs has been compiled by contributors to the Llama CPP project. This data provides valuable insights into the performance of different models across the M1, M2, and M3 families.

M1 Pro Performance

Let's start by examining the performance of the M1 Pro, which serves as a baseline for many users:

• 16GB RAM
• Text generation speed: Approximately 36 tokens per second

This aligns well with real-world experiences, where users typically observe speeds around 30-34 tokens per second when running Llama 2.

M1 Max Performance

Moving up to the M1 Max, we see a significant improvement:

• 64GB RAM
• Text generation speed: Approximately 72 tokens per second

This doubling of performance compared to the M1 Pro is consistent with the increase in CPU cores and overall system capabilities.

M3 Max Performance

Now, let's look at the latest M3 Max:

• Text generation speed: Slightly higher than the M1 Max
• Memory bandwidth: 300 GB/s (compared to 400 GB/s on M1 Max)

Interestingly, the M3 Max doesn't show a dramatic improvement over the M1 Max in terms of LLM performance, despite being a newer generation chip.

Real-World Testing: M1 Max

To validate the benchmark data, we conducted real-world tests using a newly acquired M1 Max machine. Here's what we found:

Test 1: Simple Query

Prompt: "Why is the sky blue?" Result: 58 tokens per second

This result is slightly better than the benchmark data, which suggested around 54 tokens per second.

Test 2: Essay Generation

Prompt: "Write an essay about travel" Result: 52 tokens per second

With a more complex task, we see a slight decrease in performance, but it's still within the expected range.

Test 3: Code Generation

Prompt: "Write the Fibonacci sequence in Python" Result: Similar performance to previous tests

These real-world tests confirm that the M1 Max performs as expected based on the benchmark data, providing a significant boost over the M1 Pro.

Comparing M1 Max and M3 Max

When considering an upgrade from M1 Max to M3 Max, several factors come into play:

1. Performance gain: The benchmarks suggest only a modest improvement in LLM performance for the M3 Max over the M1 Max.
2. Cost: The M3 Max comes with a significantly higher price tag, often double that of the M1 Max.
3. Memory bandwidth: The base M3 Max actually has lower memory bandwidth (300 GB/s) compared to the M1 Max (400 GB/s). To match the M1 Max's bandwidth, you'd need to opt for the 40-core version of the M3 Max, which starts at $4,000.

For most users focused on LLM tasks, the performance gain of the M3 Max may not justify the substantial increase in cost.

Considerations for Upgrading

When deciding whether to upgrade your Apple Silicon Mac for LLM performance, consider the following:

1. Current needs: If your M1 Pro or M1 Max is meeting your current requirements, there may be little reason to upgrade.
2. Budget: The M3 Max comes with a premium price tag that may not be justifiable for many users.
3. Specific use cases: Some specialized tasks may benefit more from the M3 Max's improvements in certain areas.
4. Future-proofing: While the M3 Max doesn't offer a huge leap in LLM performance, it may provide benefits in other areas or for future applications.
5. Alternative options: For serious machine learning work, consider whether a dedicated GPU setup might be more cost-effective than a high-end Mac.

Optimizing Your Current Mac for LLM Performance

If you decide to stick with your current Mac, there are several ways to optimize its performance for running LLMs:

1. Update your operating system: Ensure you're running the latest version of macOS, as it may include performance improvements and optimizations.
2. Manage background processes: Close unnecessary applications and processes to free up system resources for LLM tasks.
3. Use appropriate quantization: Experiment with different quantization levels to find the best balance between speed and accuracy for your needs.
4. Optimize your prompts: Well-crafted prompts can lead to more efficient processing and generation by the LLM.
5. Consider external accelerators: For some users, external GPUs or other accelerators might provide a cost-effective performance boost.

The Future of Apple Silicon and LLMs

As Apple continues to develop its silicon technology, we can expect further improvements in LLM performance. However, the benchmarks and real-world tests discussed in this article suggest that the gains may be incremental rather than revolutionary, at least in the near term.

It's worth noting that software optimizations can sometimes lead to significant performance improvements without the need for hardware upgrades. Keeping an eye on updates to frameworks like Llama CPP and other LLM-related software may yield performance boosts on your existing hardware.

Conclusion

For those considering an upgrade from an M1 Pro to an M1 Max for LLM tasks, the performance gain is substantial and may well be worth the investment. However, the jump from M1 Max to M3 Max appears less compelling from a pure LLM performance standpoint, especially considering the significant price difference.

Ultimately, the decision to upgrade should be based on your specific needs, budget, and the importance of LLM performance in your work or research. For many users, the M1 Max represents an excellent balance of performance and value for running large language models locally.

As the field of AI and machine learning continues to evolve, staying informed about both hardware and software developments will be crucial for making the best decisions about your computing setup. Keep experimenting, benchmarking, and sharing your results with the community to help everyone make more informed choices about their AI development environments.

Additional Resources

For those looking to delve deeper into the world of LLMs on Apple Silicon, here are some valuable resources:

1. Llama CPP GitHub repository: Stay updated on the latest developments and contribute to the project.
2. Apple's Machine Learning documentation: Learn about optimizing ML models for Apple Silicon.
3. Community forums: Engage with other developers and researchers to share experiences and tips.
4. Academic papers: Keep an eye on the latest research in LLM optimization and hardware acceleration.

By staying engaged with these resources, you'll be well-equipped to make the most of your Apple Silicon Mac for LLM tasks, regardless of which model you choose.

Article created from: https://www.youtube.com/watch?v=JFYGZ_t0yVU