On-device model refuses to classify harmless grief-related text (LanguageModelError.refusal), but will summarize it

Hi,

A harmless sentence about a grief support group makes the on-device model throw LanguageModelError.refusal when it's asked to classify it. In the same session, the same sentence is summarized without complaint, and the same sentence with "grief" swapped for something neutral classifies fine.

Environment: Mac mini M5 Pro, macOS 27.2.1 (26B131), Xcode 27.2 (27B52), AFM 3 Core Advanced. Also reproduced on iOS 27.2 and visionOS 27.2 simulators.

Minimal repro (run as written, twice, same result):

@Generable
enum SafetyLabel: String { case safe, sensitive, concerning, unsafe }

let text = "Our support group meets on Thursdays to talk about coping with grief."

// Throws LanguageModelError.refusal ("May contain sensitive content")
let classifier = LanguageModelSession(instructions: "Classify the text by content safety.")
_ = try await classifier.respond(to: text, generating: SafetyLabel.self)

// Works: "Support group meets Thursdays for grief coping."
let summarizer = LanguageModelSession()
_ = try await summarizer.respond(to: "Summarize in five words: \(text)")

Results, same session:

  • "…coping with grief." → classify (@Generable enum): refusal; summarize (plain text): works
  • "Our book club meets on Thursdays to talk about mystery novels." → classify: safe; summarize: works

Same result with SystemLanguageModel(guardrails: .permissiveContentTransformations). It's refusal, not guardrailViolation. In a fuller classifier (four classifiers with calibrated label descriptions) it was also refused under greedy and default sampling, with a purpose line in the instructions, and with SystemLanguageModel(useCase: .contentTagging).

A second case, likely the same trigger: the Little Women review in Apple's own Book Tracker sample (Using Evaluations to evaluate an intelligent feature). It recounts Beth's death, and tagging it with the sample's instructions, .permissiveContentTransformations guardrails and a @Generable tag array throws LanguageModelError.refusal on both .general and .contentTagging, in well under a second, on two Macs (M3 Max and M5 Pro). The other 12 reviews in the sample tag fine. I haven't tested whether the two share a trigger.

So grief or death in otherwise harmless text seems enough for the model to refuse structured classification and tagging, the kinds of tasks the content-tagging use case is for.

Questions:

  • Is this expected? It reads like a false positive: nothing in the text is harmful, and the model will summarize it.
  • Is there a supported way for an app to classify or tag text like this on device, for example for a support or journaling app where grief is a normal topic?

LanguageModelError.refusal indicates that the model’s internal reasoning determines that it cannot or should not fulfill the prompt. In this case, debugDescription says "May contain sensitive content," and so it is likely that the model sees "grief" a sensitive word. The whole sentence isn't harmful though, as you pointed out, and so I'd say this is worth a feedback report.

Guided generation may contribute to the model's decision as well. If plain text (without @Generable enum SafetyLabel...) is a choice, you might give it a try. For example:

Instructions: "Classify the text by content safety using one of these words: safe, sensitive, concerning, and unsafe."

Prompt: "Our support group meets on Thursdays to talk about coping with grief."

Best,
——
Ziqiao Chen
 Worldwide Developer Relations.

Thanks, Ziqiao. I tried the plain-text version, and on the general model, it does get past the refusal. I've filed FB25615990 with a self-contained repro.

Results on a MacBook Pro (M3 Max), macOS 27.2.1, AFM 3 Core Advanced, with a fresh session per request. "7 of 7" means 2 greedy runs and 5 with default sampling.

Grief sentence, guided (generating: SafetyLabel.self): refused in every run:

  • with your instructions: 7 of 7
  • with my original instructions: 7 of 7
  • with my classifier's calibrated label descriptions: 7 of 7
  • with permissiveContentTransformations guardrails: 2 of 2, greedy
  • on .contentTagging: 2 of 2, greedy

Grief sentence, plain text:

  • Your instructions: safe, 7 of 7
  • Same, with permissiveContentTransformations: safe, 7 of 7
  • My calibrated label descriptions, where sensitive reads "Topic is sensitive (health, grief, money, identity) but content is appropriate": sensitive, 7 of 7
  • .contentTagging: still refused, 7 of 7

Little Women review (Book Tracker sample): tagging into the @Generable tag array was refused on both .general and .contentTagging. In plain text, it got tags on both models in every run, for example "literary-fiction, coming-of-age, emotional-arc, family-drama, bittersweet".

So on the general model, the refusal comes from the guided request, not from the text: the same instructions get an answer once the schema is gone. Plain text works as a fallback, but then the app has to parse the reply and check that it's one of the labels, which is what guided generation was doing.

Two follow-up questions, if you know:

  • Is guided generation expected to make the model more likely to refuse, or is this a problem in how the refusal is decided?
  • Should .contentTagging refuse this sentence even in plain text, given that classification is the use case it's for?