M3 Ultra vs M2 Ultra: Performance Comparison for Developers and AI Tasks

Introduction

Apple recently introduced the M3 Ultra chip for the Mac Studio, sparking interest among developers and AI enthusiasts. This article presents a comprehensive performance comparison between the M3 Ultra and its predecessor, the M2 Ultra, focusing on tasks relevant to coders, developers, and those working with large language models (LLMs).

Test Setup

For this comparison, we used two Mac Studio configurations:

1. M3 Ultra:

   • Maximum CPU and GPU cores
   • 96GB RAM
   • 1TB storage

2. M2 Ultra:

   • Maximum CPU and GPU cores
   • 64GB RAM
   • 1TB storage

It's worth noting that while higher RAM configurations are available, the costs become prohibitive. Our tests aim to provide insights into performance differences with these more accessible configurations.

Benchmark Results

Geekbench Scores

Initial Geekbench scores show improvements in the M3 Ultra over the M2 Ultra:

• Single-core performance:

  • M3 Ultra: 3,216
  • M2 Ultra: 2,786

• Multi-core performance:

  • M3 Ultra: 28,000
  • M2 Ultra: 22,000 (approximate)

• Graphics scores:

  • M3 Ultra: 234,000
  • M2 Ultra: 251,000

While the M3 Ultra shows improvements, the difference is not as substantial as some might have expected.

JavaScript Performance

Using the JavaScript Web Toing Benchmark, we observed the following geometric mean scores:

• M3 Ultra (32 cores): 44.8
• M2 Ultra (24 cores): 36.9

The M3 Ultra demonstrates a notable improvement in JavaScript performance. Interestingly, the M4 Max (not included in this comparison) outperforms the M3 Ultra in this test.

Web Odometer 3.0 Benchmark

The Web Odometer 3.0 benchmark yielded these results:

• M3 Ultra: 40.6
• M2 Ultra: 36.2

Again, we see a slight improvement with the M3 Ultra, though not a dramatic difference. The M4 Max reportedly achieves even higher scores in this benchmark.

File Operations Performance

C Language File Operations

We ran a custom C script that creates and deletes 30,000 files:

• M3 Ultra: 106 seconds
• M2 Ultra: 128 seconds

The M3 Ultra completes this task faster, but the difference is not as significant as one might expect given the generational leap.

Python File Operations

A similar test using Python 3 to create and delete 30,000 files showed:

• M3 Ultra: 45 seconds
• M2 Ultra: 50 seconds

The performance gap narrows further with Python, with only a 5-second difference between the two chips.

AI and Machine Learning Performance

Geekbench AI - CPU

Quantized scores for machine learning tasks using the CPU:

• M3 Ultra: 6,694
• M2 Ultra: 5,482

The M3 Ultra shows a clear advantage, though perhaps not as dramatic as some might have anticipated.

Geekbench AI - GPU

GPU-accelerated machine learning scores:

• M3 Ultra: 21,250
• M2 Ultra: 16,994

Here, we see a more substantial performance gap, with the M3 Ultra leveraging its improved GPU cores for AI tasks.

Geekbench AI - Neural Engine

Surprisingly, the Neural Engine performance was closer than expected:

• M3 Ultra: 33,268
• M2 Ultra: 29,630

The relatively small difference here is unexpected, given the advancements in the M3 Ultra's Neural Engine.

Large Language Model Performance

We used LM Studio to run the DeepSeek model, testing performance with different token settings.

5,000 Token Test

Tokens processed per second for a 1,000-word story:

• M3 Ultra: 84 tokens/second
• M2 Ultra: 58 tokens/second

The M3 Ultra shows a significant advantage in LLM processing speed.

Maximum Token Test

With maximum token settings:

• M3 Ultra: 85 tokens/second
• M2 Ultra: 61 tokens/second

The performance gap remains similar even with increased token limits.

Analysis and Recommendations

For Current M2 Ultra Owners

If you currently own an M2 Ultra Mac Studio, the performance gains offered by the M3 Ultra may not justify an immediate upgrade. The improvements, while noticeable, are not revolutionary for most tasks.

For M2 Max Owners

Owners of M2 Max devices face a more nuanced decision. The upgrade to M3 Ultra could provide substantial benefits, especially for GPU-intensive tasks and LLM processing. However, the high cost of the M3 Ultra Mac Studio should be carefully weighed against the performance gains.

For Older Mac Owners

Users with M1 series chips or earlier Pro models (except perhaps the M4 Pro in the Mac Mini) could see significant improvements by upgrading to the M3 Ultra. The performance leap from these older generations is more substantial and could justify the investment for professional users.

Cost Considerations

The base version of the M3 Ultra Mac Studio starts at $4,000, with higher configurations costing significantly more. This high entry point makes it crucial to assess whether the performance gains align with your specific needs and budget.

Alternative: Refurbished M2 Ultra

For those seeking a balance between performance and cost, consider looking for refurbished M2 Ultra Mac Studios. These can offer excellent performance at a more attractive price point, especially if you opt for configurations with additional RAM.

Conclusion

The M3 Ultra represents a step forward in Apple's chip design, offering improved performance across various benchmarks and real-world tasks. However, the gains over the M2 Ultra are not as dramatic as some might have hoped, especially considering the price difference.

For developers and AI enthusiasts:

1. JavaScript and Web Performance: The M3 Ultra offers noticeable improvements, beneficial for web developers and those working on JavaScript-heavy applications.

2. File Operations: While faster, the differences in file operation speed may not significantly impact most workflows.

3. AI and Machine Learning: GPU-accelerated tasks see more substantial improvements, making the M3 Ultra more attractive for machine learning workloads.

4. LLM Processing: The M3 Ultra shows clear advantages in processing large language models, which could be crucial for AI researchers and developers working with NLP.

Ultimately, the decision to upgrade should be based on your specific needs, current hardware, and budget. While the M3 Ultra is undoubtedly powerful, the M2 Ultra remains a capable machine for many professional tasks. Those with older hardware will see the most significant benefits from an upgrade to the M3 Ultra.

As always in technology, there's a balance between having the latest hardware and finding the best value for your specific use case. Consider your workflow, the types of projects you handle, and your long-term needs when deciding whether the M3 Ultra is the right choice for you.

Article created from: https://youtu.be/2Hgk46_E_1c?feature=shared