I just released LocalLM Lab 2.0. It comes right after 1.0 because it needed breaking changes, once; every 2.x release keeps your code compiling from here on. New in 2.0: support for the 2026-07-28 MCP spec, and MCP Apps, so a server’s interactive widgets show up right in the chat. Private Cloud Compute works, so you can use Apple’s larger server model; an app built on the SDK can do the same once it has Apple’s entitlement. And there’s a new way to use small models: decision models (Jev), which answer typed questions with a confidence instead of writing text, with JevDK to design and test them. An AI app that can call tools on your Mac needs firm limits, so 2.0 tightens them. File access stays inside the folder you grant: no following symlinks out of it, and bounded reads and writes. You decide per MCP server which tools run without asking. And a local model that’s given a prompt too big for it, or that keeps calling tools without answering, is stopped instead of running on. For local-model tinkerers, there’s a Benchmark: time to first token, tokens per second and memory, for comparing models and quantizations.

I had a problem with this DoomMail, an iOS app that I had been TestFlighting. The solution is to "summarize the text". Not the content.

I did some research into Jev. The on-device Apple Foundation Model is capable enough for this, since you don't need a heavyweight model for typed decisions. The challenge is that Jev-style decisions rely on reading the model's scores for each option, and as far as I can tell the Foundation Models framework doesn't expose logits. I may be wrong and look forward to be corrected.... I also looked at OpenJev, which lets you use open-weight models for decision use cases. Since you can read the option logits there, it may be the way to go for now. I put together JevDK, a playground for learning more about Jev. Its implementation is based on OpenJev and models from HuggingFace. Using Jev is so different from using a regular chatbot that it feels like a different world altogether, so I got a lot of benefit by actually hands-on experimenting with it.

Update on LocalLM Lab: 1.0.0 is out. It is the release candidate made final, so anything built against RC.1 keeps building, and from here the SDK is source compatible across every 1.x release, with additive changes only until 2.0. Quick recap of what 1.0 gives you if you build on Apple's on-device model: SessionOptions temperature, max output tokens, topK/topP and seed take effect on the system model through the SDK (invalid combinations throw, and a seed is best-effort). The same session API reaches open-weight MLX models on macOS 27, and a host app can pin a downloaded model to the exact commit it tested, with hash-verified files and a trust policy checked before any network call. Next up is open for suggestions. I am working on MCP client support for protocol revision 2026-07-28, and I am considering benchmark metrics for measuring larger open-weight models on the new Mac Studio. If there is something you would want from the SDK alongside Foundation Models, reply here. I just responded to a question in another thread about Jev, which is the new AI hotness making the rounds. If there's interest there, let me know and we can brainstorm on what use case we can build to illustrate Jev-on-your-app.

I have been thinking of adding a Jev model type to the LocalLM Lab SDK. You call a single API and underneath, you can select either Jev (via OpenRouter, or TypeSafe) or OpenJev with a downloaded model (via MLXFoundationModels) . What's the level of interest like? I always like to ship SDK features with example apps and have been struggling to find an easy-to-illustrate use case. Thoughts?

We are now at 1.0.0-RC.1 . Two updates: If you build on Apple's on-device model: SessionOptions temperature, max output tokens, topK/topP and seed now take effect on the system model through the SDK. They were ignored before. temperature: 0 selects greedy decoding. SessionOptions.appleGenerationOptions() gives you the mapping if you call languageModelSession directly, and LocalLMLabSession.respond(to:options:) overrides per call. Two things to know: topK together with topP, or a seed with neither, now throws where it used to be ignored, and a seed is best-effort (reproducible within a process, but the first request after launch can differ). If you are instead building your app with downloaded open-weight models where the model is downloaded at the app's first run, you will be concerned that the model actually downloaded may not be exactly what you tested with during development (and QA). This is because the repo is not a fixed artifact. The model's owner can change the files behind the same name, a look-alike repo can exist, and a download can be corrupted or oversized. With this release, you can ship a pin to an exact commit of the model, so what you validated is the one users download. If that commit can't be fetched, the download fails instead of falling back to main. Downloaded files are hash-verified, a trust policy you supply is checked before any network call, and preflight checks memory, disk and architecture support before anything starts. Updates are deliberate (check, then switch all-or-nothing, with rollback), and cleanup of old versions is up to you. For those who like knobs and sliders, this release also includes the mlx-control-room example (Swift source code included) that makes the whole flow visible: validate, download, pin, update, roll back, clean up. It also has the tuning knobs for the MLX layer (sampling, prefill, output length, a small "speed helper" model, a LoRA adapter pair), each with a gauge showing it changed something.

Update on LocalLM Lab. With macOS 27 now shipping, the full model layer is available to any Apple Silicon macOS app through one session API. That includes Apple's on-device Foundation Models, downloadable open-weight models, ClaudeForFoundationModels, Private Cloud Compute, and online providers like OpenAI, Anthropic, and OpenRouter. I also put together a new reference app (source code included) for this release. VistaNova is a web-search tool built using two different on-device AI models doing two different jobs. Apple's on-device Foundation Model handles the initial search query and the MCP tool call out to Tavily. A downloadable Qwen model (via MLX) handles summarizing what comes back. Useful if you don't want to manually dig through the search result links. Also in this release, which is relevant if you're wiring your own tool-calling into a Foundation Models app: the old all-or-nothing "Full Access" toggle is gone, replaced by ToolCallAuthorizer. That's graded tool selection plus per-call authorization, surfaced through a new Security panel. The SDK's MCP client is also now current with protocol revision 2025-11-25 (structured tool results, an elicitation UI, full OAuth support). SDK guide: https://github.com/ancientcomputing/locallm/blob/1.0.0-beta/docs/sdk-guide.md VistaNova: https://github.com/ancientcomputing/vistanova

Check out LocalLM Lab (locallmlab.dev) which I also post about here. It's a development tool for working & experimenting with local AI models. If you a building AI-powered app on macOS, the companion SDK provides a lot of the lego pieces to do so. And example apps with source code are available for folks to use as a starting point.

Update on LocalLM Lab: 1.0.0-beta.3 extends the unified model layer to online providers: OpenAI, Anthropic's Messages API, OpenRouter or any OpenAI-compatible server. This is through a new RemoteModelProvider, shipped as part of a fourth SDK binary (LocalLMLabSDKRemote). Configuration is a RemoteProviderConfig (scheme, dialect: .openAIChat / .openAIResponses / .anthropicMessages / .openAICompatible, baseURL, auth, models), with presets for .openAI, .openAIResponses, .anthropic, and .openRouter. Routing and session creation work identically to local/on-device models — lab.models.route("chat", to: ModelID(scheme: "openai", rest: "gpt-…")!), then lab.makeSession(route: "chat"). Even if you are not planning to ship an app with online AI models, enabling them in the app during development is useful. If the app isn't behaving, it's often unclear whether the app's tool-calling/framework is broken or the local model you picked just isn't following instructions well. Point the same route at a frontier model for a minute, confirm the plumbing works, then go back to evaluating the local model on its own merits with that variable removed. Another quick update for folks using xcode IDE, the examples now come with .xcodeproj files. Feature page: thisbrain.ai/locallm/1.0.0-beta

Update on LocalLM Lab: 1.0.0-beta.1 adds a model layer to the SDK. This means one API across Apple's on-device FoundationModels model, Claude, and a locally-run open-weight model via MLX (a separate runtime, LocalLMLabSDKInference), with routing and model residency (which model stays warm, how many at once) owned by the SDK rather than something every app has to build itself. This is entirely SDK-side, not a FoundationModels API change. SystemModelProvider still wraps SystemLanguageModel under the hood the same way it always has. A fourth provider, PCCModelProvider (Apple's Private Cloud Compute), ships in this release but isn't functional yet in 1.0.0-beta.1. Check out https://developer.apple.com/private-cloud-compute/ for more info on this. Requires macOS 27 beta, Apple Intelligence enabled, Xcode 27 beta for SDK development. SDK guide: https://github.com/ancientcomputing/locallm/blob/main/docs/sdk-guide.md Feature page: thisbrain.ai/locallm/1.0.0-beta

Thanks. This is useful information. I built PCC support into LocalLM Lab (and SDK) and it currently reports an error. I'll add the guidance into the SDK documents.

Quick 0.8 update on this. The SDK now ships ready-made Tool wrappers for every connector (Calendar, Reminders, Contacts, Location etc) and new this release, Filesystem via WorkspaceTools. Previously, you would have to write your own adapters (which remains fully supported alongside the new tools). We ended up going the Tool wrapper route because Apple's local AI (at the macOS 26 level) wasn't reliable at mapping (to give an example) a human-readable Calendar event (Lunch on August 25) to the event ID that the connector used to require to perform any action on that event. Some of you will wonder: isn't that Apple's Tool protocol reimplemented? Well, it's Apple's Tool protocol used the way it's meant to be, with the annoying parts done for you. That means handling connector-level gotchas like the identifier problem above so you don't have to rediscover them yourselves, plus one piece Apple doesn't provide at all: MCPTool, which builds a Tool at runtime straight from an MCP server's JSON Schema. FoundationModels has zero MCP support of its own. Short version: we're not replicating Apple's Tools, we're using them, and taking care of the parts that were genuinely annoying to get right. Filesystem access itself (the picker, security-scoped bookmarks) is still deliberately host-app territory, not part of the SDK. Everything downstream of that (once your app has a resolved folder URL) is where WorkspaceAccess/WorkspaceTools come in. This includes an async-aware access pattern (withFolderAccessAsync) that is needed since a model session can invoke several file tools across one respond(to:) call. The new source-code-included example (workspace-buddy) shows it in context. Maybe it's your route to building a coding agent using the local AI ...!

Another update relevant to this thread's original question about iterating on prompt/context shape before writing app code: v0.6 adds localai-cli, a small subprocess binary for calling the on-device model straight from your own Swift (or Python) code once you've got a prompt shape you're happy with. Mechanically it's about as simple as it gets: your app spawns the binary, writes one JSON request (system prompt + input, or a messages array for multi-turn) to stdin, reads one JSON response from stdout. No HTTP server and no client library. This is useful if you've been iterating in Prompt Playground and want to move that exact prompt into a real Xcode project without wiring up LanguageModelSession and the connector/MCP plumbing yourself. It's config-scoped to whatever the user already granted through the app's own settings ( connectors and MCP tools included) so a request outside that set is rejected before the model runs at all. Omitting the field grants nothing. Being upfront: this isn't a packaged SDK for shipping. LocalLM Lab has to be running for calls to work, since they relay through its background process. It's meant for the iterate-then-verify step this thread was originally about, not for distributing inside another app yet. There's a fuller worked example on the page now too. "Plate Today" combines three connectors (Calendar, Reminders, System Clock) with an MCP tool call (Todoist's find-tasks-by-date) in a single request, and reads localai-config.json itself first to confirm everything's actually enabled before ever invoking the model. Useful reference if you're weighing how much validation to push into your own code versus relying on localai-cli's own rejection behavior. The example does both, and the preflight is purely about not spending a model call on a request you already know will fail. Interface + Swift/Python examples: thisbrain.ai/locallm/cli.html

Oh and I forgot to include the writeup on the different MCP servers I have tested with: https://thisbrain.ai/locallm/mcp-servers.html

Small(-ish) update relevant to the context-budget part of this thread: I ended up building MCP client support into LocalLM Lab (v0.4), and it turned the "what fits in the context window" question from the original post into something very concrete. MCP servers advertise their tools with full JSON schemas, and those schemas count against the same ~4096-token budget as everything else the model sees. Some servers are cheap (a couple hundred tokens for everything); others are not. For example, Linear's full tool list alone would blow the budget on its own. So the UI ended up being: every tool starts disabled per server, and you enable only what a given task needs, one or two at a time. This covers both privacy+security in a dev environment. It's a more literal version of the same problem this thread was originally about - deciding what the model actually needs to see versus what's just available. One upside of building against this specific constraint: since the model's already local and free to call, there's no API cost or sandbox setup involved in actually watching tool discovery and tool calls happen against a real server - useful if you're trying to understand the protocol itself, not just ship a feature with it. I am curious whether anyone else doing tool-calling with LanguageModelSession has run into the same tradeoff, and how you're deciding what to expose per request versus keeping available but unused.