Hi,
A harmless sentence about a grief support group makes the on-device model throw LanguageModelError.refusal when it's asked to classify it. In the same session, the same sentence is summarized without complaint, and the same sentence with "grief" swapped for something neutral classifies fine.
Environment: Mac mini M5 Pro, macOS 27.2.1 (26B131), Xcode 27.2 (27B52), AFM 3 Core Advanced. Also reproduced on iOS 27.2 and visionOS 27.2 simulators.
Minimal repro (run as written, twice, same result):
@Generable
enum SafetyLabel: String { case safe, sensitive, concerning, unsafe }
let text = "Our support group meets on Thursdays to talk about coping with grief."
// Throws LanguageModelError.refusal ("May contain sensitive content")
let classifier = LanguageModelSession(instructions: "Classify the text by content safety.")
_ = try await classifier.respond(to: text, generating: SafetyLabel.self)
// Works: "Support group meets Thursdays for grief coping."
let summarizer = LanguageModelSession()
_ = try await summarizer.respond(to: "Summarize in five words: \(text)")
Results, same session:
- "…coping with grief." → classify (
@Generableenum):refusal; summarize (plain text): works - "Our book club meets on Thursdays to talk about mystery novels." → classify:
safe; summarize: works
Same result with SystemLanguageModel(guardrails: .permissiveContentTransformations). It's refusal, not guardrailViolation. In a fuller classifier (four classifiers with calibrated label descriptions) it was also refused under greedy and default sampling, with a purpose line in the instructions, and with SystemLanguageModel(useCase: .contentTagging).
A second case, likely the same trigger: the Little Women review in Apple's own Book Tracker sample (Using Evaluations to evaluate an intelligent feature). It recounts Beth's death, and tagging it with the sample's instructions, .permissiveContentTransformations guardrails and a @Generable tag array throws LanguageModelError.refusal on both .general and .contentTagging, in well under a second, on two Macs (M3 Max and M5 Pro). The other 12 reviews in the sample tag fine. I haven't tested whether the two share a trigger.
So grief or death in otherwise harmless text seems enough for the model to refuse structured classification and tagging, the kinds of tasks the content-tagging use case is for.
Questions:
- Is this expected? It reads like a false positive: nothing in the text is harmful, and the model will summarize it.
- Is there a supported way for an app to classify or tag text like this on device, for example for a support or journaling app where grief is a normal topic?