Swift Academy Podcast, Episode 11, Season 2
MLX Swift & On-Device AI
Adrien Grondin, iOS developer at LM Studio and creator of Locally AI, on why Apple Silicon changes the economics of running language models locally, and where that stops being true.
Watch the Episode
Why This One Matters
Most teams shipping AI features today send everything to a cloud API. That works until you look at the bill, the latency, or the privacy review. Meanwhile Apple has been quietly building the other path: a framework designed from the start for unified memory on Apple Silicon.
Adrien Grondin is unusually well placed to talk about it. He works on LM Studio and built Locally AI, so he spends his days with models running on hardware people actually own. This conversation is about what that is really like once you leave the demo behind.
Where MLX Fits
Not Core ML, and not Foundation Models
Three Apple-adjacent ways to run a model on device, three different jobs. The distinctions matter the moment you have to pick one.
- How MLX differs from Core ML, and why the difference is about who owns the training and conversion pipeline
- Where Apple Foundation Models stop and MLX starts
- Which model families MLX actually supports, Mistral, Qwen and Gemma among them
- Fine-tuning with MLX, and when it is worth the trouble
The limits nobody puts in the demo
The honest part of the episode. Running a model locally is a set of trade-offs, not a free win.
- Memory constraints on Apple Silicon, and what unified memory buys you
- GPU usage, battery consumption, and what sustained inference costs a phone
- Real latency on real devices, not on a plugged-in Mac Studio
- Device compatibility, and where the floor currently sits
Building something with it
The practical half: what it takes to put MLX behind a SwiftUI app today.
- Streaming responses and how inference behaves in a real UI
- Tool calling with MLX, and what that unlocks
- Pulling models from Hugging Face, and where quantization fits
- Why small language models may matter more than the frontier ones for shipping apps
About the Guest
Adrien Grondin
iOS Developer at LM Studio • Creator of Locally AI
Adrien moved from traditional iOS development into AI engineering, and now works on LM Studio while maintaining Locally AI. His view, argued in this episode, is that on-device AI is not a passing trend but the direction the Apple platform is heading, and that the developers who understand the constraints early will build the better products.
Key Takeaways
- MLX is not a Core ML replacement. It is a different tool for a different job. Choose based on who owns the model pipeline, not on which is newer.
- Unified memory is the reason this works at all. Apple Silicon removes the copy between CPU and GPU memory, which is what makes local inference practical on a phone.
- Memory is the real ceiling, not compute. Model size and quantization decide what ships, long before performance does.
- Small language models deserve more attention. For most app features, a well-chosen SLM running locally beats a frontier model behind a network call.
- Test on device, early. Latency and battery behaviour on a real iPhone bear little relation to what you see on a Mac.
Resources & Links
Listen & Subscribe
If you are weighing on-device inference against a cloud API for your next feature, share this with whoever owns that decision.