Guide
On-Device AI on iPhone, iPad, and Mac: What It Can Actually Do
"On-device AI" has become a marketing phrase, and like most marketing phrases it gets stretched until it means very little. This guide pins it down: what running AI locally on an iPhone, iPad, or Mac actually means, where it genuinely beats cloud AI, where cloud AI is still clearly better, and — most usefully — how to verify any app's on-device claims yourself in about a minute, with no technical skills required.
What "on-device AI" actually means — and what it doesn't
On-device AI means the model itself executes on your device's own processors — the CPU, GPU, and Neural Engine inside your iPhone, iPad, or Mac. When you ask a question, the answer is generated by silicon you own. Nothing about the request needs to travel anywhere.
That is different from three things it is routinely confused with:
- An app that runs on your device but thinks in the cloud. Most AI apps are a native interface over a remote API. The app is local; the intelligence is not. Your documents and questions are uploaded, processed on someone else's servers, and only the answer comes back.
- "Hybrid" processing you can't inspect. Some products handle easy requests locally and quietly route harder ones to servers. That can be a reasonable design — but if you can't see which route a given request took, you can't reason about where your data went.
- Privacy by itself. On-device inference is necessary for privacy, but not sufficient. An app could run its model locally and still upload analytics, telemetry, or even your content for other reasons. Local inference plus a "Data Not Collected" privacy label is the combination that matters, and both are checkable.
On-device vs cloud AI: the real trade-offs
Anyone who tells you one side wins on every axis is selling something. Here is the honest version:
| On-device AI | Cloud AI | |
|---|---|---|
| Privacy | Your content never leaves hardware you own. This is a physical property, not a policy promise. | Governed by terms of service, retention rules, and access controls you have to trust and mostly cannot verify. |
| Offline use | Works in airplane mode, on flights, in secure facilities, in dead zones. | Requires a connection for every request. |
| Raw capability | Smaller models. Strongest when grounded in text you give them; weaker at open-ended reasoning and long-form generation. | Frontier models are larger and genuinely better at open-ended writing, complex reasoning, and coding. |
| World knowledge | Limited. An on-device model should not be treated as an encyclopedia. | Broad memorized knowledge, plus web access in many products. |
| Cost structure | The compute was paid for when you bought the device. No per-query cost to anyone. | Every request costs the provider money, which shapes business models: subscriptions, usage caps, or your data as the product. |
| Dependence | No server outages, no model deprecations, no account required. | Behavior can change or disappear whenever the provider changes the backend. |
The summary we'd defend anywhere: if you want a model to draft a novel or answer open-ended questions about the world, cloud frontier models are ahead, and this page won't pretend otherwise. If you want AI to read your material — contracts, research papers, meeting recordings, statements — and answer from it with sources, an on-device model covers most of that job, with a privacy guarantee no policy document can match.
What Apple silicon already runs locally
As of iOS, iPadOS, and macOS 26, Apple ships an on-device AI stack that third-party apps can build on directly:
- Apple's foundation models. The language model behind Apple Intelligence runs on the device, and Apple's Foundation Models framework exposes it to apps. Summarization, question answering, and structured extraction can run entirely on the device — no API key, no account.
- Vision OCR. On-device text recognition pulls the words out of scans, photos of pages, and image-only PDFs.
- Speech transcription. On-device speech-to-text turns recordings into searchable text without the audio ever being uploaded.
- Core ML and Metal. The plumbing that lets one workload spread across CPU, GPU, and Neural Engine — which is what makes local retrieval and inference practical on a phone at all.
None of this requires downloading model weights, opening a terminal, or running a server. It is operating-system infrastructure, present on every supported device. For a close look at what Apple's own features do and don't handle, see Can Apple Intelligence Summarize a PDF?
The airplane-mode test: how to verify an AI app yourself
You don't have to trust anyone's marketing — including ours. The test takes about a minute:
- Cut the network. Turn on Airplane Mode, then also turn off Wi-Fi — Airplane Mode alone can leave Wi-Fi connected.
- Do real work. Open the AI app and use it hard: import a new document, ask a question, request a summary. Don't just reopen old results, which may be cached.
- Watch what happens. A genuinely on-device app behaves identically to before. A cloud-dependent app stalls, errors, or quietly degrades to a limited mode.
We walk through this test against Apple's own features in Does Apple Intelligence Work Offline? The Airplane-Mode Test. One caveat: airplane mode proves the inference is local. To check what an app does when the network comes back, read its App Store privacy label — "Data Not Collected" is the strongest label Apple offers, and a developer misdeclaring it risks removal from the store.
What on-device AI is genuinely good at: your documents, not trivia
The most common way people are disappointed by on-device AI is using a small local model as if it were a shrunken cloud chatbot. Ask an on-device model open-ended trivia and it will confidently get things wrong — it simply hasn't memorized enough of the world.
The architecture that makes small models genuinely useful is retrieval-augmented generation (RAG). Instead of asking the model to know things, you hand it things to read:
- Your documents are indexed on the device — full-text and semantically.
- When you ask a question, retrieval finds the specific passages that might answer it.
- The model reads those passages and answers from them, citing where each claim came from — and declining to answer when the documents don't support one.
Reading is a task small models do well; recall is the task they do badly. RAG converts the problem from recall to reading. That's why grounded document Q&A is the honest sweet spot for on-device AI today — and why an answer with tappable citations, or an explicit "your documents don't say," is worth more than a fluent guess.
A private document workflow — no terminal, no model downloads
Here is where we stop being neutral, because this is the workflow we build. OpenIntelligence is our document AI for iPhone, iPad, and Mac, built entirely on the stack described above: Apple's on-device foundation models for answering, Vision OCR for scans, on-device transcription for audio and video recordings, and SQLite FTS5 with Metal-accelerated retrieval underneath. You add PDFs, Office and iWork files, spreadsheets, presentations, notes, scans, code, or recordings; it indexes them locally; you ask questions and get answers with tappable citations back to the exact source passage. When your documents don't support an answer, it says so instead of guessing. The 29-step RAG engine it runs on is open source, so the pipeline is inspectable rather than a black box — and the airplane-mode test above is one we actively invite, because retrieval, verification, and answering all run on the device. Its App Store privacy label is "Data Not Collected," and there is no account and no API key.
A scope note, so nobody downloads the wrong tool: OpenIntelligence is a document AI, not a general chatbot. It won't chat about the news or write your novel. And if what you actually want is to download open-weight models and experiment with them, you want a local model-runner app on a Mac — a different category of tool for a different job, and a fine one. OpenIntelligence is for people who want to open an app, add their files, and ask questions with receipts.
It's free to download, with a one-time Lifetime unlock or a Pro subscription for the full feature set. It requires iOS, iPadOS, or macOS 26 or later on Apple Intelligence-capable hardware: an iPhone 15 Pro or later, or an iPad or Mac with an M1 chip or later. You can get OpenIntelligence on the App Store, and there's a step-by-step walkthrough in How to Chat With a PDF Offline on iPhone, iPad, and Mac.
Hardware and OS requirements: who can run this today
The on-device stack this guide describes has a hard floor, set by Apple:
- iPhone: iPhone 15 Pro or later, on iOS 26 or later.
- iPad: any iPad with an M1 chip or later, on iPadOS 26 or later.
- Mac: any Mac with an M1 chip or later, on macOS 26 or later.
The cutoff isn't arbitrary: running a language model locally takes the Neural Engine and memory that those chips introduced, and Apple gates Apple Intelligence — and therefore the Foundation Models framework apps build on — to that hardware. If your device is older, the honest answer is that this workflow is not available to you on that device: Apple's on-device models simply aren't there to build on. Your options are cloud-based tools, with the trade-offs in the table above, or newer hardware.
What's next: Private Cloud Compute and consent-gated hybrid
The capability gap between on-device and frontier cloud models is real, and Apple's answer to it is Private Cloud Compute (PCC) — Apple-silicon servers designed so that data is processed for a single request and not retained, with the security model open to outside inspection. It's the most credible attempt so far to give hybrid AI verifiable properties rather than policy promises.
Our position on it is the same as our position on everything else here: hybrid is only acceptable when it's visible and chosen. OpenIntelligence has PCC support built, but it is not live — it arrives with iOS and macOS 27, expected in September 2026. When it ships, it will be consent-gated, it will apply only to the final answer-synthesis step (never to retrieval or your document index, which stay on-device), and the app will show you the exact payload before anything is sent. Today, every route runs on-device, full stop.
That's the standard worth holding any AI app to, ours included: say precisely what runs where, prove it with tests anyone can run, and ask before anything changes.