Home / Blog / On-Device AI Glossary

How-to Guides

On-Device AI Glossary: 15 Terms in Plain English

Updated June 23, 2026 · 7 min read

On-device AI is full of jargon: STT, VAD, inference, Parakeet, post-processing. If you are choosing a Mac dictation app or just want to understand how voice to text works, here are the 15 terms that matter, explained in plain English.

Short answer: On-device AI runs the model on your own computer instead of a remote server, so your data never leaves the machine. The core terms to know are speech-to-text, inference, latency, local models like Whisper and Parakeet, voice activity detection, and AI cleanup. Everything else builds on those.

Key takeaways

On-device means the AI runs on your Mac, so audio and text never get uploaded.
Speech-to-text is the engine; dictation is the feature that types it into your apps.
Latency and inference explain why some tools feel instant and others lag.
AI cleanup, custom dictionaries and translation turn raw speech into finished text.

Why an on-device AI glossary helps

Most guides to voice typing assume you already know what a "local model" is or why "latency" matters. That makes it hard to compare apps fairly. This glossary fixes that. Each entry is one or two sentences, written so a normal person can understand it, and grouped so related ideas sit together. If you are weighing up on-device dictation for privacy or speed, these are the words the marketing pages keep throwing at you.

We use BlaBlaType as the running example because it is a Mac app that runs speech recognition 100% on-device, but the definitions apply to any voice to text tool.

The dictation pipeline: microphone, on-device model, AI cleanup, then text in your app.

Core concepts (terms 1 to 5)

1. On-device AI. AI that runs on your own hardware, such as your Mac, rather than a cloud server. Your data stays local, so nothing is uploaded to be processed.
2. Speech-to-text (STT). The technology that converts spoken audio into written words. It is the engine underneath every dictation feature and voice assistant.
3. Dictation. The everyday feature built on speech-to-text that types your words directly into whatever app has your cursor, from email to a code editor.
4. Local model. An AI model downloaded and stored on your device so it can run without the internet. Once it is on your Mac, transcription works fully offline.
5. Voice activity detection (VAD). A lightweight step that decides when you are actually speaking versus silence or background noise, so the model only transcribes real speech.

These five cover the foundation. The distinction between STT and dictation is the one people miss most: STT is the brain, dictation is the hands that put text where you need it.

Speed and how models run (terms 6 to 10)

6. Inference. The moment an AI model actually does its job, taking your audio as input and producing text as output. "Running inference" just means using the model.
7. Latency. The delay between finishing a sentence and seeing the text appear. Low latency feels instant; high latency feels laggy. We break this down in our guide to why dictation latency differs.
8. Apple Silicon. Apple's M-series chips (M1 and later). Their Neural Engine and GPU let on-device models run fast, which is why modern Mac dictation feels real-time.
9. Whisper. A widely used open speech recognition model family from OpenAI, known for strong accuracy across many languages. You can read the original Whisper research paper for the technical detail.
10. Parakeet. A fast, accurate speech recognition model family optimized for low latency. BlaBlaType offers both Whisper and Parakeet models so you can trade a little accuracy for more speed, or the reverse.

Remember the one honest speed rule of voice typing: most people speak around three to four times faster than they type. Low latency is what lets you actually feel that advantage.

See these terms in action on your Mac

On-device speech-to-text, low-latency Whisper and Parakeet models, and AI cleanup in any app. No card needed for the trial.

Download for macOS

Features that shape your text (terms 11 to 15)

11. AI cleanup (post-processing). A second pass that rewrites your raw transcript into polished text: removing filler words, fixing punctuation and grammar, and adapting tone. BlaBlaType does this on-device with Apple Intelligence.
12. Custom dictionary. A list of names, brands and jargon you teach the app so it spells them right every time, instead of guessing at unusual words.
13. Custom prompt. An instruction that tells the AI cleanup step how to shape your text, for example "make it formal" or "format as bullet points."
14. Translate as you speak. Speaking in one language and getting text out in another. BlaBlaType supports 90+ languages with optional translation while you dictate.
15. Data residency. Where your data is actually processed and stored. With on-device AI, the answer is simply "on your Mac," which is the strongest privacy position.

Accuracy also depends on how a model handles voices that are not textbook standard. If that is a concern, see how speech-to-text handles accents, and if privacy is your priority, our piece on whether Mac dictation is really private goes deeper on data residency. Not every voice tool runs on-device, of course. Some, like the accessibility-focused Talon, take a different approach to voice control entirely.

Which terms matter most for you?

The words you should care about depend on how you dictate. Here is a quick way to focus.

The writer

Focus on AI cleanup and custom prompts. They turn rambly speech into finished paragraphs, which is what saves you time.

The developer

Watch latency, inference and the custom dictionary. Low delay and correct spelling of API names keep you in flow.

The privacy-first user

Care about on-device processing, local models and data residency. If it runs on your Mac, nothing is uploaded.

Whatever your profile, the workflow is the same. Once you know the vocabulary, setup gets easier too, whether you are adjusting macOS dictation settings or learning how to dictate emails on your Mac. You can compare features and prices any time on the pricing page.

Frequently asked questions

What does on-device AI mean?

On-device AI means the model runs on your own computer's hardware instead of a remote server. For dictation, your audio is turned into text locally, so nothing is uploaded. BlaBlaType runs speech recognition 100% on-device on your Mac.

Is on-device speech-to-text as accurate as the cloud?

Modern local models like Whisper and Parakeet are highly accurate, and on Apple Silicon they run fast enough for real-time dictation. Accuracy depends more on the model size and your microphone than on whether it runs locally or in the cloud.

What is the difference between speech-to-text and dictation?

Speech-to-text is the underlying technology that converts audio into words. Dictation is the everyday feature built on top of it that types those words into whatever app you are using, such as email, chat or a document.

Does on-device AI work offline?

Yes. Once the model is downloaded, on-device transcription runs without any internet connection because all processing happens on your Mac. This is why on-device dictation keeps working on a plane or with Wi-Fi off.

What is AI cleanup in dictation?

AI cleanup, also called post-processing, is a second step that rewrites your raw transcript into polished text. It removes filler words, fixes punctuation and grammar, and can adapt the tone. BlaBlaType does this on-device using Apple Intelligence.