Apple Silicon Lab (MLX + Metal)¶

You almost certainly own an M-series MacBook. This is a real inference platform in 2026, not a toy. But it is not a replacement for NVIDIA — it is a supplementary lane for specific workloads.

Why bother¶

Unified memory is genuinely different. Zero PCIe transfer cost between CPU and GPU. Model weights and activations live in the same physical DRAM. Some algorithms (huge-context LMs, sparse MoE with cold experts) work better on unified-memory than on discrete GPU.
MLX (Apple’s array framework) is now mature enough to be a legitimate serving option for local dev.
llama.cpp on Metal is what actual users run in 2026 for laptop/desktop LLM apps. If you don’t understand Metal quantization, you don’t understand llama.cpp, which means you don’t understand quantization in production.
Career optionality: Apple, Anthropic (macOS Claude), and every startup shipping a local-inference SDK cares about MLX/Metal performance.

What Apple silicon is not for¶

CUDA experience. You cannot practice CUDA on a Mac. You can practice MLX and Metal, which is a related but distinct skill set.
Training. MLX does support training and Metal has ML compute, but you will not fine-tune 70B on a MacBook. Real training happens on NVIDIA.
Peak throughput. M4 Max peaks around 60-90 tok/s on 8B models. An RTX 3090 does 100-140 tok/s on the same. An H100 does 500+.

Apple silicon is a serving platform, not a kernel-development platform. Do not spend more than 15% of your total kernel practice budget here.

The M-series lineup (2026 reality)¶

Chip	Year	GPU cores	Mem BW	Max unified RAM	Use case
M3 Max	2023	30-40	~300 GB/s	128 GB	Base viable for local 70B-4bit
M4 Max	2024	32-40	~410 GB/s	128 GB	Sweet spot for MLX lab
M4 Ultra	2025	60-80	~820 GB/s	512 GB	Mac Studio; 70B in fp16 fits
M5 Pro/Max	2026	+Neural Accel per core	~500-600 GB/s	128 GB	Tensor-core equivalent added
M5 Ultra	2026-27	—	—	512+ GB	Future

Memory bandwidth is the bottleneck for decode. M4 Max at 410 GB/s decodes an 8B-fp16 model at ~25-30 tok/s (theoretical: 410/16 = 25.6 tok/s). Compare to 3090 at 936 GB/s = ~58 tok/s theoretical. Compare to H100 at 3350 GB/s = ~210 tok/s.

M5 shipped in Oct 2025 with a “Neural Accelerator” in each GPU core — essentially tensor-core equivalent for matmul. Prompt processing (prefill, which is compute-bound) got a big boost. Decode (memory-bound) is only marginally better.

MLX — what it is and how to use it¶

MLX = Apple’s array framework, MIT-licensed, roughly “NumPy meets PyTorch” with lazy evaluation and native unified-memory support. Written by Awni Hannun et al.

Docs: https://ml-explore.github.io/mlx/
GitHub: https://github.com/ml-explore/mlx
Model zoo: https://github.com/ml-explore/mlx-examples (Llama, Mistral, Whisper, SDXL, etc.)
Community models: https://huggingface.co/mlx-community (converted Llama-3, Qwen, DeepSeek, etc.)

Minimal serving stack¶

pip install mlx mlx-lm
mlx_lm.generate --model mlx-community/Llama-3.1-8B-Instruct-4bit \
    --prompt "Explain KV cache." --max-tokens 200

# Server mode (OpenAI-compatible):
mlx_lm.server --model mlx-community/Llama-3.1-8B-Instruct-4bit --port 8080

That is a legitimate local inference server. Not fast enough for prod, more than fast enough to demo.

MLX vs PyTorch mental model¶

PyTorch	MLX
Eager by default	Lazy by default (compute triggered on `mx.eval()`)
Device explicit (`.to('cuda')`)	Unified memory; no device transfers
`torch.compile` for graph capture	`mx.compile()` decorator
Distributed = NCCL	Distributed = Apple’s own (limited)
Bindings in Python/C++	Bindings in Python/Swift/C++

If you know PyTorch, MLX takes ~2 evenings to learn.

Metal — what it is and when to touch it¶

Metal is Apple’s low-level GPU API. Metal Shading Language (MSL) is a C++14-based kernel language, roughly equivalent to CUDA C++.

When you touch Metal directly:

Writing custom kernels in MLX (MLX has a Metal kernel escape hatch: mx.fast.metal_kernel).
Contributing to llama.cpp GGML Metal backend.
Understanding how MPS (Metal Performance Shaders) matmul actually works.

When you don’t need Metal:

Just running models. MLX is enough.
Learning kernel principles. CUDA on your 3090 is the better teacher.

Reference for Metal kernels:

Apple Metal shading language spec: https://developer.apple.com/metal/Metal-Shading-Language-Specification.pdf (verify latest version at developer.apple.com/metal)
MLX metal kernel docs: https://ml-explore.github.io/mlx/build/html/dev/custom_metal_kernels.html

Position of Apple silicon in your 13-month plan¶

Phase 0-1 (M1-M2): use your Mac normally. Ignore MLX.
Phase 4 (M7-M8): spend one weekend converting your mini-inference-engine to also run on MLX. Compare throughput to your 3090. This is a portfolio piece (“I built an engine that runs on CUDA and MLX”).
Phase 5 (M9-M11): compare quantized models on MLX vs GGUF/llama.cpp on Metal vs GGUF/CUDA. This is high-signal quant bake-off content.
Phase 7 (M13+): if your capstone deployment includes “local first” as a story, MLX/Metal is part of the pitch.

Total time on Apple silicon in 13 months: ~40 hours. No more. It is a supplementary lane, not the main highway.

Concrete MLX experiments worth running¶

Throughput comparison: Same 4-bit Llama-3.1-8B on your 3090 (via llama.cpp CUDA) vs your MacBook (via MLX). Report tok/s at batch=1 and batch=8. Publish as a blog post.
KV cache scaling: how does M-series behave as context grows? Compare to 3090.
Prompt processing speedup with M5 Neural Accelerators if you have an M5 Mac. Prefill on M4 Max vs M5 is a great before/after study.
Unified memory advantage: run a 32B-4bit model that doesn’t fit in your 3090’s 24GB. It will fit on your Mac Studio if you have one, or a 64GB MacBook. That’s a real capability difference, worth writing up.
Speculative decoding with a small MLX draft model + a larger MLX target model. Fully local, laptop-side. Portfolio-worthy.

Ollama, LM Studio, and the app layer¶

Most real users on Mac run Ollama or LM Studio, not raw MLX. Both use llama.cpp under the hood.

Ollama (https://ollama.com/) — CLI + local API. Ships GGUF models. The “docker for models.”
LM Studio (https://lmstudio.ai/) — GUI. Non-technical users. Also GGUF-based.
Jan.ai — open-source alternative to LM Studio.
MLX Chat / MLX Studio — experimental UIs for MLX models.

Career note: knowing how these apps work internally (they’re all llama.cpp shells + a metadata layer) is useful because 30% of “local LLM” job descriptions mention Ollama.

Cross-references¶

Kernel comparison across platforms: 03_gpu_kernels/ (other agent).
Quant on Metal via GGUF: 09_papers/04_quantization_papers.md (GGUF section).
Open-source engine landscape includes llama.cpp Metal backend: 10_communities/03_open_source_engines.md.
2026 hardware landscape context: 05_2026_gpu_landscape.md.

One line: learn just enough MLX + Metal to publish one portfolio blog post about it. Do not become a Metal specialist. NVIDIA CUDA is your primary stack.