Posts under Machine Learning & AI topic

Post

Replies

Boosts

Views

Activity

AI framework usage without user session
We are evaluating various AI frameworks to use within our code, and are hoping to use some of the build-in frameworks in macOS including CoreML and Vision. However, we need to use these frameworks in a background process (system extension) that has no user session attached to it. (To be pedantic, we'll be using an XPC service that is spawned by the system extension, but neither would have an associated user session). Saying the daemon-safe frameworks list has not been updated in a while is an understatement, but it's all we have to go on. CoreGraphics isn't even listed--back then it part of ApplicationServices (I think?) and ApplicationServices is a no go. Vision does use CoreGraphics symbols and data types so I have doubts. We do have a POC that uses both frameworks and they seem to function fine but obviously having something official is better. Any Apple engineers that can comment on this?
10
0
2.8k
Jul ’26
Accessing Private Cloud Compute
Hello, I recently learned about Private Cloud Compute (PCC): https://developer.apple.com/private-cloud-compute/ I am currently using a standard Developer Program account, and it seems that I cannot apply for the program directly. Is there an alternative? Also, is there any additional fee for using this service? If I want to call PCC in the app, for example, using the following code: let session = LanguageModelSession( model: PrivateCloudComputeLanguageModel() ) Do I need to apply for a specific plan to ensure that my App is successfully published on the App Store and available to users? Thank you!
1
0
587
Jul ’26
Sensitive Content Error When Using Foundation Models
I am using the following code in my iOS application. #Playground { let session = LanguageModelSession() let response = try await session.respond(to: "List all states of USA.") print(response.content) } And I get the following error: The operation couldn’t be completed. (com.apple.SensitiveContentAnalysisML error 15.) I have turned off Apple Intelligence and turned on again. No use. I am using Xcode 27 beta 2. any ideas?
2
0
565
Jul ’26
Adapter Problem - compatibleAdapterNotFound
Hello. I have a problem with the FoundationModels adapter and the Apple-hosted managed asset pack via TestFlight. I have created an adapter that works fine locally by creating a model via (fileURL: URL) on a real device, but I cannot create a model using background assets by downloading the adapter via TestFlight. Every time I try to get an adapter, the creation of the adapter is interrupted by the compatibleAdapterNotFound error. The aar. archive i created using a special command - xcrun ba-package foundation-models package --adapter-path aurelius1.fmadapter --asset-pack-id fmadapter-aurelius1-9799725 --output-path ./aurelius1.aar --platforms iOS --on-demand\ after that, I replaced "OnDemand": null with "OnDemand": {} in the manifest so that the Transporter could send my archive to the App Store Connect. I followed all the recommendations in this topic - https://origin-devforums.apple.com/forums/thread/823148 ...but unfortunately unsuccessfully I would appreciate any help in solving this problem. here is the code that I use in my app -
6
0
825
Jul ’26
New Apple Intelligence - Writing tools removal
Hello, I have seen a great improvement of siri. However, my job requires a lot of communication though different languages, as mostly of those languages is not my primary language, the apple intelligence writing tool was a revolutionary tool that I have used on my daily bases. The tool remains there but attached to Siri, which takes out the advantage to use a shortcut for proofread which automatically replaces the entire text. Right now I need to copy and paste on Siri and copy the answer and paste. In my point of view we could just put back the writing tool, please 😭 separately as the previous version was flawless. Thank you.
1
1
458
Jul ’26
Writing tools
Hello, I’ve noticed Siri has undergone significant updates. Previously, the dedicated tools functioned excellently as a separate, independent menu choice. Currently, app intelligence merges with Siri. This integration doesn’t operate reliably, affecting proofreading and the dedicated tools alike. The feature feels entirely unstable—sometimes available, sometimes not—appearing occasionally on the keyboard, at other times on the right-button menu, though it worked well when accessible through settings. Could we restore that setup?
1
1
427
Jul ’26
Using AssistantEntity with existing AppEntities for iOS17+
Hi, I have an existing app with AppEntities defined, that works on iOS17+. The AppEntities also have EntityPropertyQuery defined, so they work as 'find intents'. I want to use the new @AssistantEntity which is iOS18+ where possible, while supporting the previous versions. What's the best way to do this? For e.g. I have a 'log' AppEntity: @available(iOS 17.0, *) struct CJLogAppEntity: AppEntity { static var defaultQuery = CJLogAppEntityQuery() .... } struct CJLogAppEntityQuery: EntityPropertyQuery { ... } How do I adopt this with @AssistantEntity(schema: .journal.entry) for iOS18, while maintaining compatibility with iOS17? I don't want to include two different versions of the same AppEntity. Would it just with with the correct @available annotations on both entities?
6
1
778
Jul ’26
Recommended App Store distribution strategy for apps that require Foundation Models
Hello, I'm evaluating Foundation Models announced at WWDC 2026 and have a question regarding App Store distribution. My understanding is that Foundation Models are only available on supported devices and operating system versions. For apps that rely on Foundation Models as their primary functionality (rather than offering AI as an optional feature), I'm trying to understand the recommended distribution strategy. Currently, iOS provides Required Device Capabilities to prevent users from installing apps that require hardware features such as GPS, ARKit, or NFC. However, I couldn't find an equivalent Required Device Capability for Foundation Models. I also couldn't find a way to limit App Store availability by supported device models. My questions are: What is the recommended way to distribute an app whose primary functionality depends on Foundation Models? Is there currently any supported mechanism to prevent users with unsupported devices from downloading such an app? Is Apple planning to introduce a Required Device Capability (or a similar App Store filtering mechanism) for Foundation Models before public release? Without such a mechanism, users may be able to install the app successfully but then discover that its primary functionality is unavailable on their device. I'd appreciate any guidance on the recommended approach. Thank you.
5
1
627
Jul ’26
How to Retrieve VisualLookUp Results (e.g., Object Name) in VisionKit?
Hi everyone, I'm working on an iOS app that uses VisionKit and I'm exploring the .visualLookUp feature. Specifically, I want to extract the detailed information that Visual Look Up provides after identifying an object in an image (e.g., if the object is a flower, retrieve its name; if it’s a clothing tag, get the tag's content).
2
1
1.6k
Jul ’26
[iOS 27 DB3] Apple Intelligence and Spotlight Stuck at 85-90% - How to force completion via Ethernet and Console logs analysis
Hey everyone, If your iPhone is stuck on "Optimizing Search and Siri" / Indexing at around 85-90% and hard resets or leaving it on wireless charging overnight isn't moving the needle, I found a definitive way to kickstart the daemon and force it to 100%. The Root Cause (Analyzed via macOS Console): By plugging the iPhone into a Mac and checking the Console logs, I noticed that spotlightknowledged and biomed get into an endless loop around Resolved entitled set identifiers to enumerate data resources. The system stalls on verifying developer entitlements and Apple Intelligence security tokens for specific app databases over cellular networks or unstable Wi-Fi. The dasd daemon eventually puts the pipeline into a hidden sleep state to protect the battery, making it look like it's doing nothing. The Solution that forced it to 100%: Use a Wired Connection (Ethernet Adapter): Connect your iPhone directly to your fiber optic router/modem using an RJ-45 Ethernet to USB-C adapter. iOS treats a wired Ethernet connection with the highest background priority. Turn off Cellular/Wi-Fi: Go to Control Center and disable Cellular Data and Wi-Fi entirely to force iOS to route 100% of traffic through the reliable, high-speed wired pipeline. (Verify the "Ethernet" tab appears in Settings). Trigger iCloud Token Refresh: Go to Settings -> [Your Name] -> iCloud -> Saved to iCloud and toggle off/on a major asset like iCloud Drive or Messages. This forces Spotlight to dump the stalled identifiers cache. Force Restart on Cable: Perform a Hard Reset (Vol Up, Vol Down, hold Power) while the Ethernet cable is connected. Let it Cook: Keep Low Power Mode OFF, lock the screen, and leave it alone. How to verify it's definitively done: Keep your Mac Console open and filter for completeness. Gdy to się stanie, zobaczysz ostateczny dziennik świętego Graala: fetchPipelineCompleteness: display=hidden days=3 procent=1.000000 Status the ` spent=1.000000 oznacza solidne 100% uzupełnienia. Zaraz po tym, 「dasd bezpiecznie zabije procesy worker (client process exited /connection invalid), telefon w końcu ostygnie, a żywotność baterii wróci do normy. Zaraz potem wyrzuciłem potoki glp i wszystko jest teraz masłem! Mam nadzieję, że pomoże to każdemu, kto utknął w 88% otchłani!
0
0
671
Jul ’26
Siri unable to tune to a live TV channel on tvOS — Intents & Shortcuts not working
Hi everyone, We are developing a live TV streaming app for tvOS that allows users to watch live channels, replay content, and manage cloud recordings. We are struggling to integrate Siri for a seemingly basic use case: switching the live TV channel by voice (e.g. "Hey Siri, switch to channel X on [our app]"). Here is what we have tried and observed: App Intents — we implemented custom intents, but Siri does not resolve them to our app for channel-switching requests. Shortcuts — we added Shortcuts support, but users have to explicitly configure them; Siri never proactively picks our app. In-app and out-of-app — the issue happens in both contexts. When the user asks Siri to switch to a channel, it either does nothing or suggests other applications, never ours. Our questions: Is there a specific INPlayMediaIntent configuration or domain required to handle live TV channel switching via Siri on tvOS? Is proper Siri integration for live TV gated behind the Apple Video Partner Program? If so, is there any public documentation or a path for apps outside the US to access it? Has anyone successfully implemented voice-driven live channel switching on tvOS outside of the Video Partner Program? Any guidance or pointers to relevant WWDC sessions would be greatly appreciated. Thank you.
1
7
1k
Jul ’26
Can custom App Intents support multi-turn Siri follow-ups without adopting an app schema?
Hi everyone, I’m trying to understand the current capabilities of App Intents and Siri when using custom intents and custom entities, without adopting one of Apple’s predefined app schemas. As an example, imagine I have a shopping app for shoes. I might have one intent that lists shoes available for sale, and another intent that can open a specific shoe in the app or add a specific shoe to favourites. What I’m trying to validate is whether Siri can support multi-turn, contextual follow-up requests in this kind of custom-intent-only setup. For example: User: “Show me the shoes available for sale.” Siri/App: returns a list of shoes. User: “Add the third one to my favourites.” Siri understands that “the third one” refers to the third shoe from the previous result list. This kind of follow-up seems to work in some Apple-defined domains. For example, a user can ask to list calendar events and then follow up with something like “open the third one in Calendar,” and Siri appears to understand the reference to the previous list item. My current understanding is that this kind of conversational context, ordinal reference resolution, and follow-up disambiguation may only be available when adopting one of Apple’s app schemas, where Siri has a richer understanding of the domain and entities. If an app only uses custom App Intents and custom App Entities, without adopting an Apple-defined schema, should we expect Siri to support this kind of multi-turn reference resolution? Or is each turn in the conversation which requires a custom intent invocation effectively handled more independently? I’d appreciate any clarification on whether this is expected to work, currently unsupported, or only available through schema-based integrations. Thanks
1
0
534
Jul ’26
Does Siri AI work with app Intents that don't fit any Schemas?
In the past, for any App Intent developers provide to the AppShortcutsProvider, there needed to be explicit phrases provided to trigger the AppShortcut from Siri -- and if the user try to trigger the Shortcut using a slightly different phrase then the Siri does not work. With Siri AI, will App Intents that don't fit any schemas work without needing every phrase to be explicitly programmed?
3
2
1k
Jul ’26
SpotlightSearchTool Not Invoked, Console Error
I'm following along with the WWDC video on SpotlightSearchTool and hitting an error - looking for some guidance. I've configured SpotlightSearchTool and I'm sending it to the session. let session = LanguageModelSession(tools: [tool]) { spotlightSearchInstructions } let response = try await session.respond(to: prompt, options: GenerationOptions(toolCallingMode: .required)) I set the tool calling mode to required as a test - without it I don't get errors but the logging makes it seem like it's not calling the search tool and the responses would seem to confirm that (they're not grounded in my data). So, I figured I'd try forcing it to use the tool. When I do that, I get this console error: InferenceError::hostFailed::InferenceError::inferenceFailed::TokenGenerationCore.GuidedGenerationError.invalidConfiguration(errorMessage: "Tool Choice requires tools") in response to ExecuteRequest Error during session.respond. description="The operation couldn’t be completed. (FoundationModels.LanguageModelError error -1.)" Returning empty Spotlight result. elapsedMs=3254 toolReplies=0 totalSearchItems=0 uniqueSearchItems=0 What does that mean? I'm passing in a tool, everything compiles correctly, etc. Not sure how to debug - any advice appreciated! Testing this via the Simulator on beta 3.
6
0
905
Jul ’26
Siri AI’s onscreen awareness can’t understand an AppEntity without a schema?
Use Case I’m currently developing , a hiking and cycling app. I’d like to use Siri AI in iOS/watchOS 27 to create an experience like this: While users are recording an activity in app, they can raise their wrist or invoke Siri on iPhone and simply ask: “How much farther to the destination?” “How far have I already gone?” “How much elevation gain is left?” “How far is it to the next waypoint?” … Siri can then answer based on the app’s current workout context. Since there are many metrics like these, and it’s not convenient to look at the screen while cycling or trail running, this could be a really valuable use case if Siri AI can support it. I defined an AppEntity like this, and then associated it with the UI. But I found that Siri AI doesn’t seem to understand this AppEntity. It can only understand the content it can see on the screen, even though I can see that it did execute ToobooWorkoutSessionQuery, the EntityQuery. This screen has multiple tabs. Siri understood the on-screen values such as “Distance: 0.21 mi” and “Remaining Distance: 0.14 mi”, but it didn’t know the data from the other tabs on the same page, such as heart rate, average speed, and so on. So it seems that Siri is directly reading the screen, instead of retrieving the information from the AppEntity I actually provided. Is this because the AppEntity doesn’t use one of the schemas defined by AppSchemaEntity, so Siri AI can’t understand it? App Intents without a schema seem to have a similar issue. If the AppIntent I define is not within one of those schemas, Siri won’t directly execute it either. My current solution: I defined many Intents and Shortcuts. Siri AI’s ability to parse shortcuts seems to be much smarter in the new system, and it can match them quite easily. Also, shortcuts don’t require the app to be in the foreground. However, when talking to Siri, the shortcut phrase has to include the app name, which feels a bit unnatural. For example: “How far have I traveled in xxx app?”
0
1
588
Jul ’26
AI framework usage without user session
We are evaluating various AI frameworks to use within our code, and are hoping to use some of the build-in frameworks in macOS including CoreML and Vision. However, we need to use these frameworks in a background process (system extension) that has no user session attached to it. (To be pedantic, we'll be using an XPC service that is spawned by the system extension, but neither would have an associated user session). Saying the daemon-safe frameworks list has not been updated in a while is an understatement, but it's all we have to go on. CoreGraphics isn't even listed--back then it part of ApplicationServices (I think?) and ApplicationServices is a no go. Vision does use CoreGraphics symbols and data types so I have doubts. We do have a POC that uses both frameworks and they seem to function fine but obviously having something official is better. Any Apple engineers that can comment on this?
Replies
10
Boosts
0
Views
2.8k
Activity
Jul ’26
Accessing Private Cloud Compute
Hello, I recently learned about Private Cloud Compute (PCC): https://developer.apple.com/private-cloud-compute/ I am currently using a standard Developer Program account, and it seems that I cannot apply for the program directly. Is there an alternative? Also, is there any additional fee for using this service? If I want to call PCC in the app, for example, using the following code: let session = LanguageModelSession( model: PrivateCloudComputeLanguageModel() ) Do I need to apply for a specific plan to ensure that my App is successfully published on the App Store and available to users? Thank you!
Replies
1
Boosts
0
Views
587
Activity
Jul ’26
Sensitive Content Error When Using Foundation Models
I am using the following code in my iOS application. #Playground { let session = LanguageModelSession() let response = try await session.respond(to: "List all states of USA.") print(response.content) } And I get the following error: The operation couldn’t be completed. (com.apple.SensitiveContentAnalysisML error 15.) I have turned off Apple Intelligence and turned on again. No use. I am using Xcode 27 beta 2. any ideas?
Replies
2
Boosts
0
Views
565
Activity
Jul ’26
Adapter Problem - compatibleAdapterNotFound
Hello. I have a problem with the FoundationModels adapter and the Apple-hosted managed asset pack via TestFlight. I have created an adapter that works fine locally by creating a model via (fileURL: URL) on a real device, but I cannot create a model using background assets by downloading the adapter via TestFlight. Every time I try to get an adapter, the creation of the adapter is interrupted by the compatibleAdapterNotFound error. The aar. archive i created using a special command - xcrun ba-package foundation-models package --adapter-path aurelius1.fmadapter --asset-pack-id fmadapter-aurelius1-9799725 --output-path ./aurelius1.aar --platforms iOS --on-demand\ after that, I replaced "OnDemand": null with "OnDemand": {} in the manifest so that the Transporter could send my archive to the App Store Connect. I followed all the recommendations in this topic - https://origin-devforums.apple.com/forums/thread/823148 ...but unfortunately unsuccessfully I would appreciate any help in solving this problem. here is the code that I use in my app -
Replies
6
Boosts
0
Views
825
Activity
Jul ’26
New Apple Intelligence - Writing tools removal
Hello, I have seen a great improvement of siri. However, my job requires a lot of communication though different languages, as mostly of those languages is not my primary language, the apple intelligence writing tool was a revolutionary tool that I have used on my daily bases. The tool remains there but attached to Siri, which takes out the advantage to use a shortcut for proofread which automatically replaces the entire text. Right now I need to copy and paste on Siri and copy the answer and paste. In my point of view we could just put back the writing tool, please 😭 separately as the previous version was flawless. Thank you.
Replies
1
Boosts
1
Views
458
Activity
Jul ’26
Writing tools
Hello, I’ve noticed Siri has undergone significant updates. Previously, the dedicated tools functioned excellently as a separate, independent menu choice. Currently, app intelligence merges with Siri. This integration doesn’t operate reliably, affecting proofreading and the dedicated tools alike. The feature feels entirely unstable—sometimes available, sometimes not—appearing occasionally on the keyboard, at other times on the right-button menu, though it worked well when accessible through settings. Could we restore that setup?
Replies
1
Boosts
1
Views
427
Activity
Jul ’26
Using AssistantEntity with existing AppEntities for iOS17+
Hi, I have an existing app with AppEntities defined, that works on iOS17+. The AppEntities also have EntityPropertyQuery defined, so they work as 'find intents'. I want to use the new @AssistantEntity which is iOS18+ where possible, while supporting the previous versions. What's the best way to do this? For e.g. I have a 'log' AppEntity: @available(iOS 17.0, *) struct CJLogAppEntity: AppEntity { static var defaultQuery = CJLogAppEntityQuery() .... } struct CJLogAppEntityQuery: EntityPropertyQuery { ... } How do I adopt this with @AssistantEntity(schema: .journal.entry) for iOS18, while maintaining compatibility with iOS17? I don't want to include two different versions of the same AppEntity. Would it just with with the correct @available annotations on both entities?
Replies
6
Boosts
1
Views
778
Activity
Jul ’26
Recommended App Store distribution strategy for apps that require Foundation Models
Hello, I'm evaluating Foundation Models announced at WWDC 2026 and have a question regarding App Store distribution. My understanding is that Foundation Models are only available on supported devices and operating system versions. For apps that rely on Foundation Models as their primary functionality (rather than offering AI as an optional feature), I'm trying to understand the recommended distribution strategy. Currently, iOS provides Required Device Capabilities to prevent users from installing apps that require hardware features such as GPS, ARKit, or NFC. However, I couldn't find an equivalent Required Device Capability for Foundation Models. I also couldn't find a way to limit App Store availability by supported device models. My questions are: What is the recommended way to distribute an app whose primary functionality depends on Foundation Models? Is there currently any supported mechanism to prevent users with unsupported devices from downloading such an app? Is Apple planning to introduce a Required Device Capability (or a similar App Store filtering mechanism) for Foundation Models before public release? Without such a mechanism, users may be able to install the app successfully but then discover that its primary functionality is unavailable on their device. I'd appreciate any guidance on the recommended approach. Thank you.
Replies
5
Boosts
1
Views
627
Activity
Jul ’26
How to Retrieve VisualLookUp Results (e.g., Object Name) in VisionKit?
Hi everyone, I'm working on an iOS app that uses VisionKit and I'm exploring the .visualLookUp feature. Specifically, I want to extract the detailed information that Visual Look Up provides after identifying an object in an image (e.g., if the object is a flower, retrieve its name; if it’s a clothing tag, get the tag's content).
Replies
2
Boosts
1
Views
1.6k
Activity
Jul ’26
Having to zoom out for Siri to extract information
I found I must zoom out the PDF so all area was displayed on the screen before Siri AI could scan the area and outline information from the PDF. If we have two pages, one page would be omitted by Siri AI. I am not sure if this the normal of Siri which it is supposed to be. Screenshot below. PS: Zooming out makes my human eyes unusable.
Replies
1
Boosts
0
Views
391
Activity
Jul ’26
[iOS 27 DB3] Apple Intelligence and Spotlight Stuck at 85-90% - How to force completion via Ethernet and Console logs analysis
Hey everyone, If your iPhone is stuck on "Optimizing Search and Siri" / Indexing at around 85-90% and hard resets or leaving it on wireless charging overnight isn't moving the needle, I found a definitive way to kickstart the daemon and force it to 100%. The Root Cause (Analyzed via macOS Console): By plugging the iPhone into a Mac and checking the Console logs, I noticed that spotlightknowledged and biomed get into an endless loop around Resolved entitled set identifiers to enumerate data resources. The system stalls on verifying developer entitlements and Apple Intelligence security tokens for specific app databases over cellular networks or unstable Wi-Fi. The dasd daemon eventually puts the pipeline into a hidden sleep state to protect the battery, making it look like it's doing nothing. The Solution that forced it to 100%: Use a Wired Connection (Ethernet Adapter): Connect your iPhone directly to your fiber optic router/modem using an RJ-45 Ethernet to USB-C adapter. iOS treats a wired Ethernet connection with the highest background priority. Turn off Cellular/Wi-Fi: Go to Control Center and disable Cellular Data and Wi-Fi entirely to force iOS to route 100% of traffic through the reliable, high-speed wired pipeline. (Verify the "Ethernet" tab appears in Settings). Trigger iCloud Token Refresh: Go to Settings -> [Your Name] -> iCloud -> Saved to iCloud and toggle off/on a major asset like iCloud Drive or Messages. This forces Spotlight to dump the stalled identifiers cache. Force Restart on Cable: Perform a Hard Reset (Vol Up, Vol Down, hold Power) while the Ethernet cable is connected. Let it Cook: Keep Low Power Mode OFF, lock the screen, and leave it alone. How to verify it's definitively done: Keep your Mac Console open and filter for completeness. Gdy to się stanie, zobaczysz ostateczny dziennik świętego Graala: fetchPipelineCompleteness: display=hidden days=3 procent=1.000000 Status the ` spent=1.000000 oznacza solidne 100% uzupełnienia. Zaraz po tym, 「dasd bezpiecznie zabije procesy worker (client process exited /connection invalid), telefon w końcu ostygnie, a żywotność baterii wróci do normy. Zaraz potem wyrzuciłem potoki glp i wszystko jest teraz masłem! Mam nadzieję, że pomoże to każdemu, kto utknął w 88% otchłani!
Replies
0
Boosts
0
Views
671
Activity
Jul ’26
Siri unable to tune to a live TV channel on tvOS — Intents & Shortcuts not working
Hi everyone, We are developing a live TV streaming app for tvOS that allows users to watch live channels, replay content, and manage cloud recordings. We are struggling to integrate Siri for a seemingly basic use case: switching the live TV channel by voice (e.g. "Hey Siri, switch to channel X on [our app]"). Here is what we have tried and observed: App Intents — we implemented custom intents, but Siri does not resolve them to our app for channel-switching requests. Shortcuts — we added Shortcuts support, but users have to explicitly configure them; Siri never proactively picks our app. In-app and out-of-app — the issue happens in both contexts. When the user asks Siri to switch to a channel, it either does nothing or suggests other applications, never ours. Our questions: Is there a specific INPlayMediaIntent configuration or domain required to handle live TV channel switching via Siri on tvOS? Is proper Siri integration for live TV gated behind the Apple Video Partner Program? If so, is there any public documentation or a path for apps outside the US to access it? Has anyone successfully implemented voice-driven live channel switching on tvOS outside of the Video Partner Program? Any guidance or pointers to relevant WWDC sessions would be greatly appreciated. Thank you.
Replies
1
Boosts
7
Views
1k
Activity
Jul ’26
Can custom App Intents support multi-turn Siri follow-ups without adopting an app schema?
Hi everyone, I’m trying to understand the current capabilities of App Intents and Siri when using custom intents and custom entities, without adopting one of Apple’s predefined app schemas. As an example, imagine I have a shopping app for shoes. I might have one intent that lists shoes available for sale, and another intent that can open a specific shoe in the app or add a specific shoe to favourites. What I’m trying to validate is whether Siri can support multi-turn, contextual follow-up requests in this kind of custom-intent-only setup. For example: User: “Show me the shoes available for sale.” Siri/App: returns a list of shoes. User: “Add the third one to my favourites.” Siri understands that “the third one” refers to the third shoe from the previous result list. This kind of follow-up seems to work in some Apple-defined domains. For example, a user can ask to list calendar events and then follow up with something like “open the third one in Calendar,” and Siri appears to understand the reference to the previous list item. My current understanding is that this kind of conversational context, ordinal reference resolution, and follow-up disambiguation may only be available when adopting one of Apple’s app schemas, where Siri has a richer understanding of the domain and entities. If an app only uses custom App Intents and custom App Entities, without adopting an Apple-defined schema, should we expect Siri to support this kind of multi-turn reference resolution? Or is each turn in the conversation which requires a custom intent invocation effectively handled more independently? I’d appreciate any clarification on whether this is expected to work, currently unsupported, or only available through schema-based integrations. Thanks
Replies
1
Boosts
0
Views
534
Activity
Jul ’26
Does Siri AI work with app Intents that don't fit any Schemas?
In the past, for any App Intent developers provide to the AppShortcutsProvider, there needed to be explicit phrases provided to trigger the AppShortcut from Siri -- and if the user try to trigger the Shortcut using a slightly different phrase then the Siri does not work. With Siri AI, will App Intents that don't fit any schemas work without needing every phrase to be explicitly programmed?
Replies
3
Boosts
2
Views
1k
Activity
Jul ’26
SpotlightSearchTool Not Invoked, Console Error
I'm following along with the WWDC video on SpotlightSearchTool and hitting an error - looking for some guidance. I've configured SpotlightSearchTool and I'm sending it to the session. let session = LanguageModelSession(tools: [tool]) { spotlightSearchInstructions } let response = try await session.respond(to: prompt, options: GenerationOptions(toolCallingMode: .required)) I set the tool calling mode to required as a test - without it I don't get errors but the logging makes it seem like it's not calling the search tool and the responses would seem to confirm that (they're not grounded in my data). So, I figured I'd try forcing it to use the tool. When I do that, I get this console error: InferenceError::hostFailed::InferenceError::inferenceFailed::TokenGenerationCore.GuidedGenerationError.invalidConfiguration(errorMessage: "Tool Choice requires tools") in response to ExecuteRequest Error during session.respond. description="The operation couldn’t be completed. (FoundationModels.LanguageModelError error -1.)" Returning empty Spotlight result. elapsedMs=3254 toolReplies=0 totalSearchItems=0 uniqueSearchItems=0 What does that mean? I'm passing in a tool, everything compiles correctly, etc. Not sure how to debug - any advice appreciated! Testing this via the Simulator on beta 3.
Replies
6
Boosts
0
Views
905
Activity
Jul ’26
Siri AI’s onscreen awareness can’t understand an AppEntity without a schema?
Use Case I’m currently developing , a hiking and cycling app. I’d like to use Siri AI in iOS/watchOS 27 to create an experience like this: While users are recording an activity in app, they can raise their wrist or invoke Siri on iPhone and simply ask: “How much farther to the destination?” “How far have I already gone?” “How much elevation gain is left?” “How far is it to the next waypoint?” … Siri can then answer based on the app’s current workout context. Since there are many metrics like these, and it’s not convenient to look at the screen while cycling or trail running, this could be a really valuable use case if Siri AI can support it. I defined an AppEntity like this, and then associated it with the UI. But I found that Siri AI doesn’t seem to understand this AppEntity. It can only understand the content it can see on the screen, even though I can see that it did execute ToobooWorkoutSessionQuery, the EntityQuery. This screen has multiple tabs. Siri understood the on-screen values such as “Distance: 0.21 mi” and “Remaining Distance: 0.14 mi”, but it didn’t know the data from the other tabs on the same page, such as heart rate, average speed, and so on. So it seems that Siri is directly reading the screen, instead of retrieving the information from the AppEntity I actually provided. Is this because the AppEntity doesn’t use one of the schemas defined by AppSchemaEntity, so Siri AI can’t understand it? App Intents without a schema seem to have a similar issue. If the AppIntent I define is not within one of those schemas, Siri won’t directly execute it either. My current solution: I defined many Intents and Shortcuts. Siri AI’s ability to parse shortcuts seems to be much smarter in the new system, and it can match them quite easily. Also, shortcuts don’t require the app to be in the foreground. However, when talking to Siri, the shortcut phrase has to include the app name, which feels a bit unnatural. For example: “How far have I traveled in xxx app?”
Replies
0
Boosts
1
Views
588
Activity
Jul ’26