How to Talk to a Local LLM by Voice (Ollama on Mac)

Running a model locally with Ollama keeps your data on your Mac. But typing every prompt is slow, and the terminal has no microphone button. The fix is a system-wide dictation layer that turns your voice into text inside any prompt box, so you can talk to a local LLM the same way you would talk to a person.

Why voice makes a local LLM better

A local model is fast and private, but the input path is still a keyboard. When you are exploring an idea, debugging, or drafting a long instruction, typing is the bottleneck. Most people speak around three to four times faster than they type, so a spoken prompt reaches the model sooner and captures more nuance. You also stay in flow, because you are describing the problem out loud instead of hunting for keys.

The catch is that Ollama, at its core, is a command-line and API runner. It does not listen to a microphone. To talk to it, you add a separate layer that converts speech to text and drops that text wherever your cursor sits. If you have never set that up, our guide on how to talk to ChatGPT with voice on a Mac covers the same pattern for a cloud chat, and the local version works identically.

Set it up in four steps

The whole chain takes a few minutes. You install one runner and one dictation app, then wire them together with a keyboard shortcut.

1. Install and start Ollama. Pull a model, for example ollama run llama3, and confirm it answers a typed prompt in the terminal.
2. Add on-device dictation. Install a system-wide voice-to-text app, grant microphone and accessibility permissions, and download a local speech model so it works offline.
3. Place your cursor in the prompt. Click into the Ollama terminal, or a local chat window if you prefer a UI.
4. Press the shortcut and speak. Hold or tap your dictation key, say your prompt, release, and the cleaned-up text lands in the box ready to send.

Because the dictation types into any field, the same shortcut also works if you later switch to LM Studio for chatting with local models or a browser-based front end. You learn one workflow and reuse it everywhere.

Cloud voice vs on-device voice for local models

If your entire reason for using Ollama is privacy, it makes little sense to route your voice through a cloud transcription service. That would send your spoken prompts to a server even though the model itself never does. The table shows the trade-off.

For a local-first setup, on-device dictation is the only option that matches Ollama's privacy model end to end. The speech recognition uses local models such as Whisper and Parakeet, the same open family behind OpenAI's Whisper project, so accuracy is strong even with no network.

Make the transcription code-friendly

Talking to a local LLM often means saying technical words a general dictation model has never met. Two features make this reliable. A custom dictionary teaches the app proper nouns and jargon, so it writes Ollama, Llama, PyTorch, or a repo name correctly instead of guessing. AI cleanup, running on-device via Apple Intelligence, strips filler words, fixes punctuation, and turns a rambling spoken thought into a tidy prompt. That is the difference between a prompt the model has to untangle and one it can act on straight away.

Speed matters here too. If you are curious where the three-to-four-times figure comes from, the general idea of speaking faster than typing is well documented in the concept of words per minute across speech and typing. In practice, dictating a paragraph-long prompt takes seconds, and you spend the saved time actually reading the answer.

Who benefits most

Voice-prompting a local model is not only for one kind of user. Here is where it pays off fastest.

The bottom line

A local LLM gives you a private, offline assistant. Adding on-device dictation gives it ears, without breaking that privacy. Speak your prompt, let the AI cleanup polish it, and send it to Ollama, all on your own Mac. It is the most natural way to use a model you already trust to stay local. See what a full setup costs on the pricing page, or read more about what free versus paid dictation apps actually give you.